Method, apparatus, medium and terminal for determining motion information of target object

Abstract

This application relates to the field of computer vision technology. Specifically, it relates to a method for determining motion information of a target object, a device for determining motion information of a target object, a computer-readable medium, and a terminal. The method includes: determining a target object from N frames of detection data, where N is a positive integer; obtaining the i-th data set corresponding to the target object in the i-th frame, and obtaining the j-th data set corresponding to the target object in the j-th frame, where i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i is less than j; performing a matching calculation on the i-th data set and the j-th data set to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame; and determining the motion information of the target object according to the relative position information. The technical solution of this application can improve the accuracy of the motion information of the target object.

Description

Technical Field

[0001] This application relates to the field of computer vision technology. Specifically, it relates to a method for determining motion information of a target object, a device for determining motion information of a target object, a computer-readable medium, and a terminal. Background Art

[0002] In the process of using lidar or depth cameras to map a target, it is often necessary to perform a motion calculation and determination on moving targets in the point cloud to obtain the motion characteristics of the target object, and then evaluate the motion intention or motion impact of the target.

[0003] However, the error of the motion information determination scheme for the target object provided by the related art is relatively large. Therefore, the accuracy of the motion information determination scheme for the target object needs to be improved.

[0004] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of this application. Therefore, it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a method for determining motion information of a target object, a device for determining motion information of a target object, a computer-readable medium, and a terminal, which can improve the accuracy of determining the motion information of the target object to a certain extent.

[0006] Other features and advantages of this application will become apparent through the following detailed description, or be learned in part through the practice of this application.

[0007] According to one aspect of the embodiments of this application, a method for determining motion information of a target object is provided, including: determining a target object from N frames of detection data, where N is a positive integer; obtaining the i-th data set corresponding to the target object in the i-th frame, and obtaining the j-th data set corresponding to the target object in the j-th frame, where i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i is less than j; performing a matching calculation on the i-th data set and the j-th data set to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame; and determining the motion information of the target object according to the relative position information.

[0008] According to another aspect of the embodiments of the present application, there is provided a device for determining motion information of a target object, including: a target object determination module, configured to: determine a target object from N frames of detection data, where N is a positive integer; a data set acquisition module, configured to: acquire the i-th data set corresponding to the target object in the i-th frame, and acquire the j-th data set corresponding to the target object in the j-th frame, where i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i < j; a matching module, configured to: perform a matching calculation on the i-th data set and the j-th data set to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame; a motion information determination module, configured to: determine the motion information of the target object according to the relative position information.

[0009] According to still another aspect of the embodiments of the present application, there is provided a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, it implements the method for determining motion information of a target object as described in the above embodiments.

[0010] According to yet another aspect of the embodiments of the present application, there is provided a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the method for determining motion information of a target object as described in the above embodiments.

[0011] The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

[0012] In the technical solutions provided by some embodiments of the present application, a target object is determined from N frames of detection data. Then, the i-th data set corresponding to the target object in the i-th frame is acquired, and the j-th data set corresponding to the target object in the j-th frame is acquired. Further, a matching calculation is performed on the i-th data set and the j-th data set to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame. Compared with the relative position information reflecting a certain point or certain points, the above relative position information can more accurately reflect the actual motion situation of the target object, which is beneficial to obtaining a more accurate target position observation value, and thus the motion information of the target object determined according to the technical solutions of the present application has a higher accuracy.

[0013] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. Description of the Drawings

[0014] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 Schematically shows a flowchart of a method for determining the motion information of a target object according to an exemplary embodiment of the present application.

[0016] Figure 2 Schematically shows a flowchart of a method for determining a target object according to an exemplary embodiment of the present application.

[0017] Figure 3 Schematically shows a comparison diagram of inter-frame data sets for the same target object according to an exemplary embodiment of the present application.

[0018] Figure 4 Schematically shows a flowchart of a method for determining the motion information of a target object according to another exemplary embodiment of the present application.

[0019] Figure 5 Shows a relative change diagram of inter-frame data sets for the same target object according to an exemplary embodiment of the present application.

[0020] Figure 6 Schematically shows a flowchart of a method for determining relative position information according to an exemplary embodiment of the present application.

[0021] Figure 7 Schematically shows a structural diagram of a device for determining the motion information of a target object according to an exemplary embodiment.

[0022] Figure 8 Schematically shows a structural diagram of a device for determining the motion information of a target object according to another exemplary embodiment.

[0023] Figure 9 Schematically shows a block diagram of a terminal suitable for implementing the method for determining the motion information of the above-mentioned target object. Detailed implementation manners

[0024] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail in conjunction with the drawings.

[0025] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0026] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances. In addition, in the description of the present application, unless otherwise specified, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0027] This technical solution can be applied to related fields such as lidar, depth cameras and other point cloud sensors for surveying and mapping detection, such as autonomous driving obstacle detection, point cloud mapping, and human pose detection.

[0028] Exemplarily, during the process of autonomous driving, the motion information of the target object around the vehicle (such as motion speed information and direction, etc.) is generally obtained through sensors. Further, the driving path and driving speed and other information of the vehicle are planned according to the obtained motion information of the target object.

[0029] In the existing motion information determination scheme of the target object, for the first scheme in the related art, that is, the scheme of directly detecting the motion speed information of the target object through a millimeter-wave radar, etc. The relevant radar has low ranging and angular resolution for the target object, resulting in inaccurate description of both the position characteristics and the shape characteristics of the target object. That is to say, this existing scheme sacrifices the detection performance of other attribute characteristics of the target to obtain the direct measurement ability of the motion information such as the speed of the target object. Therefore, the accuracy of the motion information determined by this scheme is not high.

[0030] In the second scheme in the above-mentioned related art, it is often applied to sensors such as lidar and depth cameras that cannot perform Doppler measurement on the target. For example, lidar can obtain the accurate position of the target, and estimate the target speed according to the position and the time difference between frames.

[0031] Specifically, the position of the target object at different frames (i.e., at different times) is observed by a point cloud sensor such as a lidar or a depth camera, and a certain feature point of the point cloud (e.g., the centroid point of the point cloud, the nearest point of the point cloud, the corner point of a bounding box calculated from the point cloud, or the geometric center point, etc.) is selected as the position reference point of the target. Then, the motion information of the entire target object is estimated based on the position change of the position reference point and the time difference between frames.

[0032] However, the actual point cloud observation target may have a relatively large volume (such as a vehicle), and may also be an object with a changing shape (such as a pedestrian). For such target objects, if the position reference point is selected in the above manner, the position relationship reflected, by selecting a single feature point as the position representation of the target, that is, inherently taking the position change of this feature point as the position change of the target, will not be able to accurately describe the shape characteristics of the target object, resulting in a large deviation between the observed value of the target object and the actual position transformation of the object's motion. Especially in the case of a small cycle (i.e., when the time interval between frames is short, such as the common lidar frame interval is 50 ms to 100 ms), the above position estimation error will be doubled and amplified.

[0033] Therefore, the accuracy of the motion information determined by this scheme also needs to be improved.

[0034] In view of the technical problems existing in the above related technologies, the technical solution provided by this application can reduce the error in determining the motion information of the target object, thereby improving the accuracy.

[0035] Next, the method for determining the motion information of the target object provided in the embodiments of this application will be introduced in detail with reference to the Figure 1 - appended Figure 5 , in which, Figure 1 FIG. schematically shows a flowchart of a method for determining the motion information of a target object according to an exemplary embodiment of this application. Referring to Figure 1 , the method includes the following steps:

[0036] S110, determine the target object from N frames of detection data, where N is a positive integer;

[0037] S120, obtain the i-th dataset corresponding to the target object in the i-th frame, and obtain the j-th dataset corresponding to the target object in the j-th frame, where i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i is less than j;

[0038] S130, perform a matching calculation on the i-th dataset and the j-th dataset to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame; and,

[0039] S140, determine the motion information of the target object according to the relative position information.

[0040] Figure 1 In the technical solution provided by the illustrated embodiment, for the same target object (for example, an obstacle during the intelligent driving of a vehicle, etc.), a data set corresponding to the target object in one frame of detection data is obtained, and another data set corresponding to the target object in another frame of detection data is obtained. Further, the data sets corresponding to the above two frames of detection data are respectively subjected to matching calculation to obtain relative position information that can reflect the overall position change of the above target object. In the related art, the information determined reflects the position change of a certain point or certain points (such as the centroid of the point cloud, the nearest point of the point cloud, the corner points of a bounding box calculated by the point cloud, or the geometric center point, etc.). Compared with directly using the relevant points as the position reference points of the target in the related art, the relative position information that can reflect the overall position change of the above target object in this technical solution can more accurately reflect the actual movement situation of the target object. Therefore, this technical solution is more conducive to obtaining accurate position observation values of the target object. It can be seen that the motion information of the target object determined according to the technical solution of this application has high accuracy.

[0041] The following Figure 1 introduces in detail the specific implementation manners of each step included in the illustrated embodiment:

[0042] In S110, a target object is determined from N frames of detection data, where N is a positive integer.

[0043] In an exemplary embodiment, the above detection data includes point cloud data obtained by lidar scanning, and may also include an image with depth information of the relevant surrounding environment collected by a depth camera. For example, the above multi-frame point cloud data is collected by setting a lidar of an autonomous vehicle to collect the environmental information around the autonomous vehicle. Further, the collected multi-frame point cloud data is used to analyze the motion information of the obstacles around the autonomous vehicle.

[0044] In an exemplary embodiment, in an autonomous driving scenario, the above target object may be an obstacle during the driving of an autonomous vehicle, such as other vehicles, pedestrians, buildings, etc. that affect driving.

[0045] As a specific implementation manner of step S110, in an exemplary embodiment, Figure 2 The flowchart of the method for determining a target object in an exemplary embodiment of the present application is schematically shown. Specifically, S210 - S240 are used to predict the detection target in each frame of detection data based on a machine learning model, and S250 and S260 are used to determine the above target object according to the detection target obtained from the above prediction.

[0046] Refer to Figure 2, in S210, multi-frame detection data containing tag information is obtained to determine a training set and a test set, where the tag information includes the near-field detection targets included in each frame of detection data.

[0047] In an exemplary embodiment, a frame of detection data and its corresponding tag information are used as a set of samples, and the above training set / test set is constituted by multiple sets of samples.

[0048] Taking a mechanical lidar as an example, the near-field recognition range refers to the area range of a circle surrounded by a recognition distance with the lidar as the center. At the same time, the near-field ranges corresponding to lidars with different numbers of beams are different, and the determination of the near-field range can be preset. For example, the near-field recognition range of a 32-line mechanical lidar can be set as the area of a circle surrounded by a radius of 60 meters with the radar as the center. For another example, the near-field recognition range of a 128-line lidar can be set as the area of a circle surrounded by a radius of 100 meters with the lidar as the center. Exemplarily, the mechanical lidars with other numbers of beams can be set with a scanning radius between 60 meters and 100 meters. It can be understood that the above near-field detection target refers to a target object within the near-field range.

[0049] Exemplarily, the above tag information includes: near-field containing detection targets and near-field not containing detection targets. For example, in an unmanned driving scenario, the above detection targets can be obstacles such as other vehicles, pedestrians, and buildings that affect driving. For example, a set of positive samples includes detection data S and the tag information of detection data S: near-field containing detection targets, and the detection targets are pedestrians and vehicles. For another example, a set of negative samples includes detection data S' and the tag information of detection data S': near-field not containing detection targets.

[0050] In S220, a machine learning algorithm is trained through the training set.

[0051] In an exemplary embodiment, in order to improve the recognition accuracy of the detection object, the above machine learning algorithm adopts a deep learning algorithm suitable for processing detection data, such as point cloud segmentation pointnet or an upgraded version pointnet++ of pointnet, etc.

[0052] In S230, the machine learning classification model in the training process is tested through the test set, and the machine learning classification model that meets the preset evaluation index is determined as the target recognition model.

[0053] In an exemplary embodiment, one or more of the following model evaluation metrics can be used: accuracy, recall, and the area under the receiver operating characteristic curve (ROC) (AUC, a model evaluation metric specifically used to evaluate the prediction value of a model; short for Area Under Curve) to evaluate the above machine learning classification model. And the machine learning classification model that meets the budget evaluation conditions is determined as the target recognition model and used for identifying the target object.

[0054] In S240, the k-th (k takes values of 1, 2,..., N) frame of detection data is input into the trained target recognition model, and the detection target in the k-th frame of detection data is determined according to the output of the target recognition model.

[0055] In an exemplary embodiment, for the above N frames of detection data, they are respectively input into the above target recognition model. After being processed by the trained machine learning algorithm, it can output whether the detection data currently input into the model contains a detection target. Exemplarily, after the first frame of detection data is input into the above target recognition model, the prediction result of the model for the first frame of detection data is "no detection target in the near field". After the 10th frame of detection data is input into the above target recognition model, the prediction result of the model for the 10th frame of detection data is "detection target a in the near field". After the 15th frame of detection data is input into the above target recognition model, the prediction result of the model for the 15th frame of detection data is "detection target a in the near field". After the 20th frame of detection data is input into the above target recognition model, the prediction result of the model for the 20th frame of detection data is "detection target b in the near field".

[0056] It can be seen that through S240, it is possible to obtain whether each frame of detection data contains a near-field detection target and which detection target it contains. Further, through S250 and S260, the detection targets detected between different frames are associated and matched to determine the above target object.

[0057] It should be noted that the identification of the detection target in the detection data is not limited to the method provided in the above embodiment. In an exemplary embodiment, a point cloud segmentation and clustering algorithm can also be used to implement the identification of the detection target in the detection data.

[0058] Continue to refer to Figure 2 , in S250, calculate the feature similarity between the detection target corresponding to the m-th frame and the detection target corresponding to the n-th frame. And, in S260, the detection target corresponding to the feature similarity that meets the budget requirement is determined as the target object.

[0059] In an exemplary embodiment, the above-mentioned feature similarity includes the similarity of shape features, the similarity of position features, and the similarity of reflectivity. m is not equal to n, and m and n are positive integers not greater than N. Exemplarily, by comparing the feature similarity between the detected objects in two frames of detection data, those with relatively similar features are considered to be the same object. For example, for the detected object a in the m-th frame and the detected object a' in the n-th frame, if the similarity of the shape features between the detected object a and the detected object a' meets the first preset value, the similarity of the position features meets the second preset value, and the similarity of the reflectivity meets the third preset value, it indicates that the detected object a and the detected object a' are the same object; otherwise, the detected object a and the detected object a' do not belong to the same object.

[0060] Specifically, taking the detected object a as an example. Calculate the similarity between each retrieved object other than the detected object a itself and the detected object a, and then obtain a similarity list corresponding to the detected object a. Determine the retrieved object with the largest similarity value in the similarity list as the same object as the detected object a.

[0061] Continue to refer to Figure 1 , in S120, obtain the i-th data set corresponding to the target object in the i-th frame, and obtain the j-th data set corresponding to the target object in the j-th frame. i is a positive integer less than N (i takes values 1, 2, 3... N - 1), j is a positive integer greater than 1 and not greater than N (i takes values 2, 3... N - 1, N), and i is less than j. It should be noted that "i" and "j" reflect the front-back relationship between frames, and the "i-th frame" and the "j-th frame" may or may not be two consecutive frames.

[0062] Different from the related art that directly uses a certain point corresponding to the target object (such as the centroid of the point cloud, the nearest point of the point cloud, the corner point of a bounding box calculated from the point cloud, or the geometric center point, etc.) as the position reference point of the target, this technical solution will use the relative position information between different frames regarding the whole of the same target object to determine the position reference. Since the above relative position information can accurately reflect the actual movement of the target object, this technical solution is more conducive to obtaining accurate position observation values regarding the target object.

[0063] To obtain the above relative position information, in S120, obtain the data sets corresponding to the target object in different frames. Among them, to improve the calculation accuracy, the data sets corresponding to the target object in consecutive frames can be obtained. For example, the i-th data set corresponding to the target object in the i-th frame, and obtain the j-th data set corresponding to the target object in the j-th frame.

[0064] In an exemplary embodiment, Figure 3 Schematically shows a comparison diagram of the inter-frame data sets regarding the same target object according to an exemplary embodiment of the present application. Refer toFigure 3 , which specifically shows a schematic diagram of the comparison of the inter-frame data sets for the same target object (rabbit). Specifically, the data sets corresponding to the ear A, back B, and tail C of the target object belong to the detection data of the previous frame (shown as darker colored points in the figure), and the data sets corresponding to the ear A', back B', and tail C' of the target object belong to the detection data of the subsequent frame (shown as lighter colored points in the figure). It can be seen that the target object has made relative movement during the two frames before and after. The following will introduce the determination of the movement information of the moving object based on the above relative movement. Continuing to refer to Figure 1 , in S130, the i-th data set and the j-th data set are matched and calculated to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame. And, in S140, the movement information of the target object is determined according to the relative position information.

[0065] As a specific implementation manner of step S130 and step S140, in an exemplary embodiment, Figure 4 schematically shows a flowchart of a method for determining the movement information of a target object according to another exemplary embodiment of the present application. Referring to Figure 4 , the embodiment shown in this figure includes S410 - S450.

[0066] In S410, the relative displacement information and relative attitude angle information of the target object are calculated based on the i-th data set and the j-th data set to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame.

[0067] Among them, the coincidence degree of the i-th data set and the j-th data set after the transformation of the relative position information is greater than a preset value. That is to say, the above relative position information can make the coincidence degree of the position of the target object in the i-th and its position in the j-th greater than the preset value.

[0068] In an exemplary embodiment, the relative displacement information and displacement registration error, as well as the relative attitude angle information and angle registration error, of the i-th data set and the j-th data set can be solved through Euclidean transformation until the position registration error and the angle registration error converge, or the relative displacement information / relative attitude angle information reaches within a preset error range, then the relative displacement information and relative attitude angle information are used as the relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame. Exemplarily, the above relative position information can be a rotation translation matrix including relative displacement information and relative attitude angle information.

[0069] Based on the above relative displacement information, the i-th data set is translated, and based on the above relative attitude angle information, the translated i-th data set is rotated, so that the coincidence degree of the i-th data set and the j-th data set is greater than the preset value.

[0070] Schematic Figure 5 shows a relative change diagram of an inter-frame data set for the same target object in an exemplary embodiment according to the present application. Refer to Figure 5 , that is, when the coincidence degree between the above-mentioned i-th data set and the j-th data set is greater than a preset value, the target object (rabbit) in the previous frame and the target object in the subsequent frame are almost coincident. Specifically, the data set coincidence degree between ear A in the previous frame and ear A' in the subsequent frame is greater than the preset value or reaches the best match, so that the data set coincidence degree between back B in the previous frame and back B' in the subsequent frame is greater than the preset value or reaches the best match, and the data set coincidence degree between tail C in the previous frame and tail C' in the subsequent frame is greater than the preset value or reaches the best match.

[0071] In an exemplary embodiment, in order to further improve the inter-frame coincidence degree of the target object, which is beneficial to improving the accuracy of determining the motion information of the target object. The present technical solution also provides the following embodiments for determining relative position information:

[0072] Figure 6 Schematically shows a flowchart of a method for determining relative position information in an exemplary embodiment according to the present application. Refer to Figure 6 , the embodiment shown in this figure includes S610 - S650.

[0073] S610, segment the target object into multiple sub-blocks.

[0074] Exemplarily, after determining the target object, the target object is segmented into multiple sub-blocks according to actual needs. Among them, the same target object can be segmented multiple times, and the following calculation process is performed each time, and the segmentation method corresponding to the maximum inter-frame target repeatability is used as the final segmentation method. It can be understood that the size of the sub-block is also related to the set accuracy requirement, and the data sub-block can be adjusted according to the accuracy requirement. For example, the higher the accuracy requirement, the smaller the split data blocks. When the accuracy requirement is high enough, it can even be split into each data point (if it is point cloud data, each point in the point cloud).

[0075] S620, for each sub-block, obtain the i-th sub-data set corresponding to the i-th frame, and obtain the j-th sub-data set corresponding to the j-th frame.

[0076] Exemplarily, if the target object (rabbit) in Figure 5 is segmented into three sub-blocks, then: for the first sub-block, obtain its i-th 1 sub-data set in the i-th frame, and obtain its j-th 1 sub-data set in the j-th frame; for the second sub-block, obtain its i-th 2Sub-dataset, and obtain its jth in the jth frame 2 sub-dataset; for the third block, obtain its ith in the ith frame 3 sub-dataset, and obtain its jth in the jth frame 3 sub-dataset.

[0077] S630. For each block, perform matching calculations on the ith sub-dataset and the jth sub-dataset to obtain the relative displacement and coincidence degree information corresponding to each block.

[0078] Still taking the above embodiment as an example, for the first block, perform matching calculations on the ith 1 sub-dataset and the jth 1 sub-dataset to obtain the coincidence degree information about the first block; for the second block, perform matching calculations on the ith 2 sub-dataset and the jth 2 sub-dataset to obtain the coincidence degree information about the second block; for the third block, perform matching calculations on the ith 3 sub-dataset and the jth 3 sub-dataset to obtain the coincidence degree information about the third block. Among them, the specific implementation manner of the matching calculation can refer to the above embodiment.

[0079] S640. Determine the block corresponding to the maximum value among multiple coincidence degrees as the first target block.

[0080] S650. Determine the rotation displacement matrix determined according to the ith sub-dataset and the jth sub-dataset corresponding to the first target block as the relative position information reflecting the overall position change of the target object between the jth frame and the ith frame.

[0081] Exemplarily, if the coincidence degree value of the second block is the largest, then determine the second block as the first target block. Further, determine the rotation displacement matrix determined according to the ith 2 sub-dataset and the jth 2 sub-dataset as the relative position information reflecting the position change of the target object (the whole rabbit) between frames.

[0082] Exemplarily, in an optional embodiment of the present application, S640 can also be: obtain the coincidence degree information corresponding to each block, and integrate the blocks with the coincidence degree information greater than the preset value into the first target block. Among them, S650 can also be: determine the rotation displacement matrix determined according to the ith sub-dataset and the jth sub-dataset corresponding to the first target block as the relative position information reflecting the overall position change of the target object between the jth frame and the ith frame.

[0083] Among them, still taking the above embodiment as an example, if Figure 5The target object (rabbit) in it is divided into three blocks. The first block is the rabbit ear part, the second block is the rabbit head part, and the third block is the rabbit body part. Each block is calculated separately to obtain the relative displacement and coincidence degree information corresponding to each block. It can be understood that if the coincidence degree value of the second block and the third block is greater than a preset value (for example, 90% can be adjusted according to the actual situation), then the second block and the third block are used as the first target block, and the rotation displacement matrix is determined again according to the sub-dataset corresponding to the i-th frame data and the sub-dataset corresponding to the j-th frame data corresponding to the first target block. The rotation displacement matrix is used as the relative position information reflecting the overall position change of the target between the j-th frame and the i-th frame.

[0084] Exemplarily, taking a pedestrian in motion as the target object, the collected point cloud data of the pedestrian is processed in blocks to obtain multiple block data corresponding to the limbs, and multiple block data corresponding to the head and torso parts. By calculating the coincidence degree values corresponding to each block through matching, it can be known that the coincidence degree value between the sub-displacement rotation matrices of the multiple blocks corresponding to the head and torso parts is relatively high. That is to say, the head and torso parts are the parts with relatively small inter-frame motion differences. Then in this embodiment, each block corresponding to the head and torso parts is determined as the first target block set, and finally, matching calculations are performed based on the data sets corresponding to the first target block set in the i-th frame and the j-th frame respectively to obtain the relative position information of the target object between frames.

[0085] It can be understood that in an alternative solution of this embodiment, when performing S610 to divide the target object into multiple blocks, it may further include: obtaining the block matching results of the same type of target objects, and correcting the segmentation processing rules of the current target object according to the block matching results of the same type of target objects.

[0086] For example, when the target object is determined to be a rabbit, obtain the first target block set of previous rabbits. Still taking the above embodiment as an example, if Figure 5 the target object (rabbit) in it is divided into three blocks. The first block is the rabbit ear part, the second block is the rabbit head part, and the third block is the rabbit body part. After multiple calculations, the probability that the first target block is the combined part of the second block and the third block is greater than the preset value, then the point cloud data of the current target object is segmented according to the first target block set of previous rabbits. That is, when the target object is recognized as a rabbit, the point cloud data corresponding to the body and head parts is preferentially used as the first target block subset for matching calculations to obtain the relative position information.

[0087] In this embodiment, determining the blocks corresponding to the sub-displacement rotation matrices with relatively high similarity as the above-mentioned target block set is beneficial for capturing the parts (blocks) with relatively small inter-frame motion differences of the target object, and using the parts with relatively small inter-frame motion differences of the target object to calculate the relative position information reflecting the overall change of the target object between frames is beneficial for improving the accuracy of determining the relative position information, and further beneficial for improving the accuracy of the motion information of the target object.

[0088] In an exemplary embodiment, after determining the above-mentioned relative position information, the velocity information of the target object can be determined based on the time difference between the i-th frame and the j-th frame. Thus, the motion information of the target object is determined. It should be noted that the motion information of the target object can also be determined according to the following embodiments. Specifically:

[0089] In S410’, determine the first position information corresponding to the target object in the i-th frame.

[0090] In an exemplary embodiment, using the determination of the position state of the entire target to replace the position transformation relationship of selecting a single feature point or multiple feature points of the target to characterize the actual motion of the target, so as to obtain a more accurate determination of the position transformation, and finally a more accurate motion information can be obtained. This application aims to use the method of point cloud registration to replace the selection of feature points to achieve the observation of more accurate position changes, and then obtain more accurate information on the motion state of the target. The invention can significantly improve the accuracy of determining the motion state of the target.

[0091] In the above technical solution, the above-mentioned relative position information can reflect the position change of the entire target. Therefore, for any point in the i-th data set, its relative change amount from the i-th frame to the j-th frame is the same. Therefore, any point in the i-th data set can be selected as the above-mentioned target reference point.

[0092] In S420, superimpose the relative position information and the first position information to obtain the second position information of the target object in the j-th frame.

[0093] In an exemplary embodiment, the rotation displacement matrix including the above-mentioned relative displacement information and relative attitude angle information and the coordinate information reflecting the first position information can be superimposed to obtain the coordinate information of the target object in the j-th frame, that is, the above-mentioned second position information is obtained. Specifically, superimposing the relative displacement information on the coordinate information of the above-mentioned first position information can translate the i-th data set, and based on the relative attitude angle superimposed on the coordinate information of the above-mentioned first position information, the above-mentioned i-th data set can be rotated, and finally the second position information corresponding to the target object in the j-th frame is obtained.

[0094] In S420’, obtain the time difference between the i-th frame and the j-th frame. Further, in S430, based on a Kalman filter or a similar tracking filtering algorithm, determine the moving speed, moving acceleration, and moving direction of the target object according to the time difference, the first position information, and the second position information.

[0095] In this actual example, it is necessary to assume that both the motion equation and the observation equation of the object are linear and the errors follow a normal distribution, and then obtain the optimal estimation information of the moving speed, moving acceleration, and moving direction of the target object.

[0096] In an exemplary embodiment, in order to focus on the motion information of a certain block in the target object, this technical solution can also perform segmentation processing on the target object after determining the target object to obtain a target block (to distinguish it from the above-mentioned “first target block”, the target block here is denoted as “second target block”), and further determine the motion information of the second target block. For example, if the user hopes to focus on determining the motion information of the ears of the rabbit in the above embodiment, after determining the target object rabbit in the multi-frame detection data, the data set corresponding to the target object is segmented, and the second target block corresponding to the rabbit ear part is obtained. Further, determining the motion information of the second target block specifically includes:

[0097] Obtain the i-th sub-data set corresponding to the second target block in the i-th frame, and obtain the j-th sub-data set corresponding to the second target block in the j-th frame; perform matching calculation on the i-th sub-data set and the j-th sub-data set to obtain relative position information reflecting the overall position change of the second target block between the j-th frame and the i-th frame; finally, determine the motion information of the second target block according to the relative position information. Thus, information such as the moving speed, moving acceleration, and moving direction of the ear part of the rabbit is determined.

[0098] Regarding the specific implementation manner of determining the motion information of the above-mentioned second target block, it is similar to the embodiment of determining the motion information of the above-mentioned target object, and will not be elaborated here.

[0099] The above-mentioned relative position information determined by this technical solution reflects the overall position change of the target object (more specifically, it can also be the second target block after segmentation processing of the target object), rather than the position change of a certain point in the data set involved in the related art. The above-mentioned relative position information in this solution can reduce the error in determining the motion information of the target object / second target block. Therefore, this solution can use the determination of the position state of the entire target to replace the position transformation relationship of a single feature point or multiple feature points of the selected target to characterize the motion information of the target, thereby reducing the error in determining the motion information of the target object / second target block, and finally improving the estimation accuracy of the target object / second target block.

[0100] Exemplarily, the technical solution can significantly improve the following situations: 1. Since the relative perspective changes, the sensor observes different parts of the target in different frames. For example, when a vehicle overtakes the lidar, the observed point cloud of the target changes from the front part of the overtaking vehicle to the rear part of the overtaking vehicle. 2. The shape change of the target point cloud caused by the swinging arm movement of pedestrians when walking. 3. The local shape change of the point cloud caused by the bending of the front of a two-wheeled vehicle rider when turning. 4. The large local shape change of the point cloud caused by the bending of the front of an extra-long trailer when turning. 5. The local occlusion of the target by the sensor at a certain moment results in the loss of local point cloud of the target.

[0101] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For the details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0102] Among them, Figure 7 Schematically shows the structural diagram of the device for determining the motion information of a target object according to an exemplary embodiment. Please refer to Figure 7 , the device 700 for determining the motion information of the target object shown in this figure can be implemented as all or part of the terminal through software, hardware, or a combination of both, and can also be integrated as an independent module on the server.

[0103] The device 700 for determining the motion information of the target object in the embodiment of the present application includes: a target object determination module 710, a data set acquisition module 720, a matching module 730, and a motion information determination module 740, where:

[0104] The above-mentioned target object determination module 710 is used to: determine the target object from N frames of detection data, where N is a positive integer; the above-mentioned data set acquisition module 720 is used to: acquire the i-th data set corresponding to the above-mentioned target object in the i-th frame, and acquire the j-th data set corresponding to the above-mentioned target object in the j-th frame, where i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i is less than j; the above-mentioned matching module 730 is used to: perform a matching calculation on the above-mentioned i-th data set and the above-mentioned j-th data set to obtain relative position information reflecting the overall position change of the above-mentioned target object between the above-mentioned j-th frame and the above-mentioned i-th frame; and, the above-mentioned motion information determination module 740 is used to: determine the motion information of the above-mentioned target object according to the above-mentioned relative position information.

[0105] In an exemplary embodiment, Figure 8 Schematically shows the structural diagram of the device for determining the motion information of a target object according to another exemplary embodiment. Please refer to Figure 8 :

[0106] Exemplarily, the matching module 730 in the motion information determination device 700 provided by an embodiment of the present application is specifically configured to: calculate the relative displacement information and relative attitude angle information of the target object according to the above-mentioned i-th data set and the above-mentioned j-th data set to obtain the above-mentioned relative position information, where the coincidence degree between the i-th data set and the j-th data set after the transformation of the relative position information is greater than a preset value.

[0107] Exemplarily, the motion information determination device 700 of the above-mentioned target object further includes: a segmentation processing module 760.

[0108] Among them, the above-mentioned segmentation processing module 760 is used to: after the target object determination module 710 determines the target object from N frames of detection data, segment the target object into multiple blocks; the data set acquisition module 720 is further used to: for each block, acquire the corresponding i-th sub-data set in the i-th frame and acquire the corresponding j-th sub-data set in the j-th frame; the matching module 730 is specifically used to: for each block, perform matching calculation on the i-th sub-data set and the j-th sub-data set to obtain a plurality of coincidence degrees; determine the block corresponding to the maximum value among the plurality of coincidence degrees as the first target block; and determine the rotation displacement matrix determined according to the i-th sub-data set and the j-th sub-data set corresponding to the first target block as the relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame.

[0109] Exemplarily, the motion information determination module 740 in the motion information determination device 700 provided by an embodiment of the present application includes: a position information determination unit 7401, a position information calculation unit 7402, and a motion information determination unit 7403. Among them:

[0110] The above-mentioned position information determination unit 7401 is used to: determine the first position information corresponding to the target object in the i-th frame; the above-mentioned position information calculation unit 7402 is used to: superimpose the above-mentioned relative position information and the above-mentioned first position information to obtain the second position information corresponding to the target object in the j-th frame; and the above-mentioned motion information determination unit 7403 is used to: determine the motion information of the target object according to the above-mentioned first position information and the above-mentioned second position information.

[0111] Exemplarily, the relative position information includes relative displacement information and relative attitude angle information; the above-mentioned position information calculation unit 7402 is specifically used to: superimpose the relative displacement information on the first position information to translate the i-th data set, and superimpose the relative attitude angle on the first position information to rotate the i-th data set to obtain the second position information corresponding to the target object in the j-th frame.

[0112] Exemplarily, the above-mentioned motion information determination unit 7403 is specifically configured to: obtain the time difference between the above-mentioned i-th frame and the above-mentioned j-th frame; and, based on a Kalman filter or a similar tracking filtering algorithm, determine the moving speed, moving acceleration, and moving direction of the above-mentioned target object according to the time difference, the above-mentioned first position information, and the above-mentioned second position information.

[0113] Exemplarily, the target object determination module 710 in the target object motion information determination device 700 provided by the embodiments of the present application includes: a detection target recognition unit 7101 and an association matching unit 7102. Among them:

[0114] The above-mentioned detection target recognition unit 7101 is configured to: obtain N frames of detection data and recognize the detection targets in each frame of detection data. And, the above-mentioned association matching unit 7102 is configured to: perform association matching on the detection targets between different frames to obtain one or more target objects.

[0115] Exemplarily, the above-mentioned detection target recognition unit 7101 is specifically configured to: input the k-th frame of detection data into a pre-trained target recognition model, and determine the detection target in the k-th frame of detection data according to the output of the above-mentioned target recognition model, where k is a positive integer not greater than N.

[0116] Exemplarily, the target object motion information determination device 700 provided by the embodiments of the present application further includes a model training module 750. Among them:

[0117] The above-mentioned model training module 750 is configured to: obtain multiple frames of detection data containing marking information to determine a training set and a test set, where the above-mentioned marking information includes the near-field detection targets included in each frame of detection data; train a machine learning algorithm through the above-mentioned training set; test the machine learning classification model during the training process through the above-mentioned test set, and determine the machine learning classification model that meets the preset evaluation index as the above-mentioned target recognition model.

[0118] Exemplarily, the above-mentioned association matching unit 7102 is specifically configured to: calculate the feature similarity between the detection target corresponding to the m-th frame and the detection target corresponding to the n-th frame, where the above-mentioned feature similarity includes the similarity of shape features, the similarity of position features, and the similarity of reflectivity, m is not equal to n, and m and n are positive integers not greater than N; determine the detection target corresponding to the feature similarity that meets the budget requirement as the above-mentioned target object.

[0119] Exemplarily, the above-mentioned segmentation processing module 760 is further configured to: after the target object determination module 710 determines the target object from the N frames of detection data, perform segmentation processing on the above-mentioned target object to obtain a second target block of the above-mentioned target object;

[0120] The above-mentioned data set acquisition module 720 is specifically configured to: acquire the i-th sub-data set corresponding to the above-mentioned second target block in the i-th frame, and acquire the j-th sub-data set corresponding to the above-mentioned second target block in the j-th frame; the above-mentioned matching module 730 is specifically configured to: perform matching calculation on the above-mentioned i-th sub-data set and the above-mentioned j-th sub-data set to obtain relative position information reflecting the overall position change of the above-mentioned second target block between the j-th frame and the i-th frame; and, the above-mentioned motion information determination module 740 is specifically configured to: determine the motion information of the above-mentioned second target block according to the above-mentioned relative position information.

[0121] Since each functional module of the motion information determination device of the target object in the exemplary embodiment of the present application corresponds to the steps in the exemplary embodiment of the above-mentioned motion information determination method of the target object, for details not disclosed in the device embodiment of the present application, please refer to the embodiment of the above-mentioned motion information determination method of the target object of the present application.

[0122] It should be noted that when the above-mentioned motion information determination device of the target object provided in the above-mentioned embodiment executes the motion information determination method of the target object, only the above-mentioned division of each functional module is used for illustration. In actual application, the above-mentioned functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the above-mentioned motion information determination device of the target object provided in the above-mentioned embodiment and the embodiment of the motion information determination method of the target object belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0123] The serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0124] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method in any of the foregoing embodiments are implemented. Among them, the computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disk, optical disk, DVD, CD-ROM, microdrive, and magneto-optical disk, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory device, magnetic card or optical card, nanosystem (including molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0125] The embodiment of the present application also provides a terminal, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the steps of the method in any of the foregoing embodiments are implemented.

[0126] Figure 9A block diagram of a terminal suitable for implementing the method for determining the motion information of the above-mentioned target object is schematically shown. As Figure 9 shown, the terminal 900 includes: a processor 901 and a memory 902.

[0127] In the embodiment of the present application, the processor 901 is the control center of the computer system, which can be the processor of a physical machine or the processor of a virtual machine. The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one of the following hardware forms: DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0128] In the embodiment of the present application, the above-mentioned processor 901 is specifically configured to: determine a target object from N frames of detection data, where N is a positive integer; obtain the i-th data set corresponding to the target object in the i-th frame, and obtain the j-th data set corresponding to the target object in the j-th frame, where i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i is less than j; perform a matching calculation on the i-th data set and the j-th data set to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame; and determine the motion information of the target object according to the relative position information.

[0129] Further, the performing a matching calculation on the i-th data set and the j-th data set to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame includes: calculating relative displacement information and relative attitude angle information of the target object according to the i-th data set and the j-th data set to obtain the relative position information, where the coincidence degree between the i-th data set after being transformed by the relative position information and the j-th data set is greater than a preset value.

[0130] Further, after determining the target object from N frames of detection data, the method further includes: segmenting the target object into a plurality of blocks; for each block, obtaining the i-th sub-data set corresponding to the i-th frame and obtaining the j-th sub-data set corresponding to the j-th frame;

[0131] Perform matching calculations on the i-th dataset and the j-th dataset to obtain relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame, including: for each sub-block, perform matching calculations on the i-th sub-dataset and the j-th sub-dataset to obtain multiple degrees of coincidence; determine the sub-block corresponding to the maximum value among the multiple degrees of coincidence as the first target sub-block; determine the rotation displacement matrix determined according to the i-th sub-dataset and the j-th sub-dataset corresponding to the first target sub-block as the relative position information reflecting the overall position change of the target object between the j-th frame and the i-th frame.

[0132] Further, determining the motion information of the target object according to the above relative position information includes: determining the first position information corresponding to the target object in the i-th frame; superimposing the above relative position information and the above first position information to obtain the second position information corresponding to the target object in the j-th frame; determining the motion information of the target object according to the above first position information and the above second position information.

[0133] Further, the relative position information includes: relative displacement information and relative attitude angle information; superimposing the relative position information and the first position information to obtain the second position information corresponding to the target object in the j-th frame includes: superimposing the relative displacement information onto the first position information to translate the i-th dataset, and based on the relative attitude angle superimposed onto the first position information to rotate the i-th dataset to obtain the second position information corresponding to the target object in the j-th frame.

[0134] Further, determining the motion information of the target object according to the above first position information and the above second position information includes: obtaining the time difference between the i-th frame and the j-th frame; based on a Kalman filter or a similar tracking filtering algorithm, determining the moving speed, moving acceleration, and moving direction of the target object according to the time difference, the above first position information, and the above second position information.

[0135] Further, obtaining N frames of detection data and determining one or more target objects from the above N frames of detection data includes: obtaining N frames of detection data and identifying the detection targets in each frame of detection data; associating and matching the detection targets between different frames to obtain one or more target objects.

[0136] Further, identifying the detection targets in each frame of detection data includes: inputting the k-th frame of detection data into a pre-trained target recognition model and determining the detection targets in the k-th frame of detection data according to the output of the above target recognition model, where k is a positive integer not greater than N.

[0137] Further, the above method further includes: obtaining multiple frames of detection data containing marker information to determine a training set and a test set, where the above marker information includes near-field detection targets included in each frame of detection data; training a machine learning algorithm through the above training set; testing the machine learning classification model during the training process through the above test set, and determining the machine learning classification model that meets the preset evaluation metrics as the above target recognition model.

[0138] Further, the above-mentioned associative matching of detection targets between different frames to obtain one or more target objects includes: calculating the feature similarity between the detection target corresponding to the m-th frame and the detection target corresponding to the n-th frame, where the above feature similarity includes the similarity of shape features, the similarity of position features, and the similarity of reflectivity, m is not equal to n, and m and n are positive integers not greater than N; determining the detection target corresponding to the feature similarity that meets the budget requirement as the above target object.

[0139] Further, after determining the target object from the N frames of detection data, the above method further includes: performing segmentation processing on the above target object to obtain multiple second target blocks of the above target object;

[0140] Obtaining the i-th data set corresponding to the above target object in the i-th frame and obtaining the j-th data set corresponding to the above target object in the j-th frame includes: obtaining the i-th sub-data set corresponding to the above second target block in the i-th frame and obtaining the j-th sub-data set corresponding to the above second target block in the j-th frame;

[0141] Performing matching calculation on the above i-th data set and the above j-th data set to obtain relative position information reflecting the overall position change of the above target object between the above j-th frame and the above i-th frame, including: performing matching calculation on the above i-th sub-data set and the above j-th sub-data set to obtain relative position information reflecting the overall position change of the above second target block between the above j-th frame and the above i-th frame;

[0142] Determining the motion information of the above target object according to the above relative position information includes: determining the motion information of the above second target block according to the above relative position information.

[0143] The memory 902 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 902 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 902 is used to store one or more instructions, and the one or more instructions are used to be executed by the processor 901 to implement the method in the embodiments of the present application.

[0144] In some embodiments, the terminal 900 further includes: a peripheral device interface 903 and one or more peripheral devices. The processor 901, the memory 902, and the peripheral device interface 903 may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 903 through a bus, signal lines, or a circuit board. Specifically, the peripheral devices include at least one of a display screen 904, a camera 905, and an audio circuit 906.

[0145] The peripheral device interface 903 can be used to connect one or more peripheral devices related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments of the present application, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 can be implemented on a separate chip or circuit board. The embodiments of the present application do not make specific limitations in this regard.

[0146] The display screen 904 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 904 is a touch display screen, the display screen 904 also has the ability to collect touch signals on or above the surface of the display screen 904. The touch signals can be input to the processor 901 as control signals for processing. At this time, the display screen 904 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments of the present application, there may be one display screen 904, which is provided on the front panel of the terminal 900; in some other embodiments of the present application, there may be at least two display screens 904, which are respectively provided on different surfaces of the terminal 900 or in a foldable design; in still some other embodiments of the present application, the display screen 904 may be a flexible display screen, which is provided on a curved surface or a foldable surface of the terminal 900. Even, the display screen 904 can be set to an irregular non-rectangular shape, that is, a special-shaped screen. The display screen 904 can be prepared using materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0147] The camera 905 is used to collect images or videos. Optionally, the camera 905 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth camera, a wide-angle camera, and a telephoto camera, so as to implement the function of background blurring by fusing the main camera and the depth camera, panoramic shooting by fusing the main camera and the wide-angle camera, and VR (Virtual Reality) shooting function or other fused shooting functions. In some embodiments of the present application, the camera 905 may further include a flash. The flash may be a single-color temperature flash or a two-color temperature flash. The two-color temperature flash refers to a combination of a warm light flash and a cold light flash, and can be used for light compensation under different color temperatures.

[0148] The audio circuit 906 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 901 for processing. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively disposed at different parts of the terminal 900. The microphone may also be an array microphone or an omnidirectional collection type microphone.

[0149] The power supply 907 is used to supply power to each component in the terminal 900. The power supply 907 may be alternating current, direct current, a primary battery or a rechargeable battery. When the power supply 907 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charging technology.

[0150] The block diagram of the terminal structure shown in the embodiments of the present application does not limit the terminal 900. The terminal 900 may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.

[0151] In the present application, terms such as "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or order; the term "plurality" means two or more, unless otherwise clearly defined. Terms such as "mounted", "connected", "connected", and "fixed" should be understood in a broad sense. For example, "connected" may be a fixed connection, a detachable connection, or an integral connection; "connected" may be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0152] In the description of the present application, it should be understood that the orientation or positional relationship indicated by terms such as "upper" and "lower" is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or unit referred to must have a specific direction, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present application.

[0153] The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A method for determining the motion information of a target object, wherein, comprising: Performing segmentation processing on the point cloud data of the target object to obtain multiple blocks; For each of the blocks, obtaining the i-th sub-dataset corresponding to the block in the i-th dataset, and obtaining the j-th sub-dataset corresponding to the block in the j-th dataset, where the i-th dataset is the dataset corresponding to the target object in the i-th frame of detection data, the j-th dataset is the dataset corresponding to the target object in the j-th frame of detection data, i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i is less than j; For each of the blocks, performing matching calculation on the i-th sub-dataset and the j-th sub-dataset to obtain multiple degrees of coincidence; Determining the blocks corresponding to the degrees of coincidence greater than a preset value among the multiple degrees of coincidence as the first target blocks; Determining the rotation displacement matrix determined according to the i-th sub-dataset and the j-th sub-dataset corresponding to the first target block as the relative position information reflecting the overall position change of the target object between the j-th frame of detection data and the i-th frame of detection data; Determining the motion information of the target object according to the relative position information.

2. The method for determining the motion information of a target object according to claim 1, wherein, The step of determining the blocks corresponding to the degrees of coincidence greater than a preset threshold among the multiple degrees of coincidence as the first target blocks includes: Determining the block corresponding to the maximum degree of coincidence among the multiple degrees of coincidence as the first target block.

3. The method for determining the motion information of a target object according to claim 1, wherein, Before performing the segmentation processing on the point cloud data of the target object to obtain multiple blocks, the method further includes: Performing segmentation processing on multiple groups of point cloud data regarding the same target object respectively; For each group of segmented point cloud data, calculating the inter-frame target repeatability; Determining the group corresponding to the maximum inter-frame target repeatability as the target group, and using the segmentation method corresponding to the target group as the target segmentation method, where the target segmentation method is used to perform segmentation processing on the point cloud data of the target object to obtain the multiple blocks.

4. The method for determining the motion information of a target object according to claim 1, wherein, After performing the segmentation processing on the point cloud data of the target object to obtain multiple blocks, the method further includes: Judging whether the size of each block among the multiple blocks meets a preset accuracy requirement; In the case where there are blocks among the multiple blocks whose sizes do not meet the preset accuracy requirement, adjusting the segmentation processing; wherein the preset accuracy requirement is negatively correlated with the size of each block among the multiple blocks.

5. The method for determining the motion information of a target object according to claim 1, wherein, Before performing the segmentation processing on the point cloud data of the target object to obtain multiple blocks, the method further includes: Obtaining the block matching results of similar target objects, and correcting the segmentation processing of the current target object according to the block matching results of the similar target objects.

6. The method for determining the motion information of a target object according to any one of claims 1 to 5, wherein, After the point cloud data of the target object is segmented to obtain multiple chunks, the method further includes: Determining a second target chunk among the multiple chunks, where the second target chunk is determined according to the part of the target object that the user focuses on; Obtaining the ith sub-dataset corresponding to the second target chunk in the ith dataset, and obtaining the jth sub-dataset corresponding to the second target chunk in the jth dataset; Performing a matching calculation on the ith sub-dataset corresponding to the second target chunk and the jth sub-dataset corresponding to the second target chunk to obtain relative position information reflecting the overall position change of the second target chunk between the jth dataset and the ith dataset; Determining the motion information of the second target chunk according to the relative position information reflecting the overall position change of the second target chunk between the jth dataset and the ith dataset.

7. The method for determining the motion information of a target object according to claim 1, wherein, Before the point cloud data of the target object is segmented to obtain multiple chunks, the method further includes: Obtaining N frames of detection data and identifying the detection targets in each frame of detection data; Associating and matching the detection targets between different frames to obtain one or more of the target objects.

8. An apparatus for determining the motion information of a target object, wherein, comprises: A chunk processing module for segmenting the point cloud data of the target object to obtain multiple chunks; A sub-dataset acquisition module for, for each of the chunks, obtaining the ith sub-dataset corresponding to the chunk in the ith dataset, and obtaining the jth sub-dataset corresponding to the chunk in the jth dataset, where the ith dataset is the dataset corresponding to the target object in the ith frame of detection data, the jth dataset is the dataset corresponding to the target object in the jth frame of detection data, i is a positive integer less than N, j is a positive integer greater than 1 and not greater than N, and i is less than j; A calculation module for, for each of the chunks, performing a matching calculation on the ith sub-dataset and the jth sub-dataset to obtain multiple degrees of coincidence; A target chunk determination module for determining the chunk corresponding to the degree of coincidence greater than a preset value among the multiple degrees of coincidence as the first target chunk; A matching module for determining the rotation displacement matrix determined according to the ith sub-dataset and the jth sub-dataset corresponding to the first target chunk as the relative position information reflecting the overall position change of the target object between the jth frame of detection data and the ith frame of detection data; A motion information determination module for determining the motion information of the target object according to the relative position information.

9. A computer-readable medium having a computer program stored thereon, wherein, when the program is executed by a processor, it implements the steps of the method for determining the motion information of a target object according to any one of claims 1 to 7.

10. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method for determining the motion information of the target object according to any one of claims 1 to 7.

Images