Circuit and method for storing point cloud, chip and electronic device

Abstract

The present application relates to a circuit and method, chip and electronic device for storing point clouds, wherein the circuit for storing point clouds includes: a cache module, configured to cache point cloud data, the point cloud data including multiple point cloud frames; a recognition result acquisition module, coupled with the cache module, configured to obtain recognition results for multiple point cloud frames, and obtain recognition results for target objects; an assembly module, coupled with the recognition result acquisition module, configured to determine the association relationship between the target object and multiple point cloud frames in response to the recognition result characterizing the existence of the target object; an output module, coupled with the recognition result acquisition module and the assembly module, respectively, configured to output the recognition result and the multiple point cloud frames corresponding to the recognition result in association based on the association relationship, so as to write into a non-volatile memory. The present application can realize the targeted storage of valid data and improve the convenience of data retrieval.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a circuit and method for storing point clouds, a chip, and an electronic device. Background Art

[0002] With the rapid development of computer technology and artificial intelligence technology, automatic (assisted) driving technology has begun to enter people's daily lives. Radar can provide environmental perception information for automatic (assisted) driving technology in the form of point cloud data.

[0003] Point cloud data refers to a set of vectors in a three-dimensional coordinate system, which may contain color information (RGB) or reflection intensity information (Intensity), etc. Radar can generate tens of thousands of points per second, and the number of data points obtained in each scan can reach hundreds of thousands, millions or even more, with a data volume of tens of gigabytes (Gb) to thousands of Gb. Such a huge amount of data brings challenges to data storage and data retrieval. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a circuit and method, a chip and an electronic device for storing point clouds, which can store valid data and improve the convenience of data retrieval.

[0005] The first aspect of the present application provides a circuit for storing point clouds, including at least one of a cache module, a recognition result acquisition module, an assembly module, and an output module. The cache module is configured to cache point cloud data, and the point cloud data includes multiple point cloud frames; the recognition result acquisition module is coupled to the cache module, and is configured to obtain recognition results for multiple point cloud frames, and obtain recognition results for target objects; the assembly module is coupled to the recognition result acquisition module, and is configured to determine the association relationship between the target object and the multiple point cloud frames in response to the recognition result representing the existence of the target object; the output module is coupled to the recognition result acquisition module and the assembly module, respectively, and is configured to output the recognition result and the multiple point cloud frames corresponding to the recognition result in association based on the association relationship, so as to write them into a non-volatile memory.

[0006] The second aspect of the present application provides a chip, comprising a distance calculation unit and the above-mentioned circuit for storing point clouds. The distance calculation unit is configured to calculate the distance between the object and the mobile platform based on the time difference between the emission time point of the laser and the reception time point of the reflected laser, and generate point cloud data; the circuit for storing point clouds is configured to output multiple point cloud frames in the point cloud data in association with the target object.

[0007] The third aspect of the present application provides a board, including the circuit for storing point clouds as described above or the chip as described above. In addition, a non-volatile memory may also be included.

[0008] A fourth aspect of the present application provides a radar, comprising at least one of the circuits, chips or boards for storing point clouds as described above.

[0009] A fifth aspect of the present application provides a mobile platform, comprising the radar as described above.

[0010] A sixth aspect of the present application provides a method for storing point clouds, including: caching point cloud data, the point cloud data including multiple point cloud frames; obtaining recognition results for the multiple point cloud frames to obtain recognition results for a target object; in response to the recognition results indicating the existence of a target object, determining an association relationship between the target object and the multiple point cloud frames; and outputting the recognition results and the multiple point cloud frames corresponding to the recognition results in an associative manner based on the association relationship to write into a non-volatile memory.

[0011] A seventh aspect of the present application provides an electronic device, comprising: a processor; and a memory, on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method as described above.

[0012] An eighth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor executes the above method.

[0013] A ninth aspect of the present application provides a computer program product, comprising an executable code, which implements the above method when the executable code is executed.

[0014] The technical solution provided by this application may have the following beneficial effects:

[0015] In certain embodiments of the present application, after the cache module caches the multi-frame point cloud frames, it does not directly write the multi-frame point cloud frames into the non-volatile memory, but obtains the recognition result for the target object through the recognition result acquisition module, and the association relationship between the target object and the multi-frame point cloud frames is determined by the assembly module. In this way, the output module can output the recognition result and the multi-frame point cloud frames corresponding to the target object of the recognition result in an associative manner based on the association relationship, so as to write the useful (target object) point cloud frame and the corresponding recognition result into the non-volatile memory. The embodiment of the present application effectively reduces the waste of storage space caused by storing useless data, and facilitates the retrieval of the target point cloud frame based on the recognition result and the association relationship, thereby improving the retrieval efficiency and accuracy.

[0016] In certain embodiments of the present application, after obtaining the recognition result, the recognition result acquisition module determines the position of the target point cloud frame in the cached point cloud frame based on the timestamp of the target point cloud frame, which helps to improve the accuracy and efficiency of the retrieval result.

[0017] In some embodiments of the present application, the recognition model of the recognition result acquisition module outputs recognition results for each of the multiple preset target objects, and the point cloud frames corresponding to the target objects can be retained respectively. When the point cloud frame includes multiple target objects, the recognition results of the point cloud frame including the multiple target objects will not be lost, thereby improving the integrity of the stored valid data.

[0018] In some embodiments of the present application, on the basis of ensuring the integrity of valid data, deduplication is performed on point cloud frames that may be stored repeatedly. The deduplication operation avoids storing repeated point cloud frames, further improving the effective utilization of storage space.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0021] Figure 1 It is a schematic diagram of an application scenario of a circuit and method for storing point clouds, a chip, and an electronic device shown in an embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of another application scenario of a circuit and method for storing point clouds, a chip, and an electronic device shown in an embodiment of the present application;

[0023] Figure 3 is a block diagram of a circuit for storing a point cloud according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a recognition process for a point cloud frame according to an embodiment of the present application;

[0025] Figure 5 is a block diagram of a recognition result obtaining module shown in an embodiment of the present application;

[0026] Figure 6 is a block diagram of a target frame timestamp determination unit shown in an embodiment of the present application;

[0027] Figure 7 is a block diagram of a frame number statistics circuit shown in an embodiment of the present application;

[0028] Figure 8 is a schematic diagram of a recognition result for a point cloud frame shown in an embodiment of the present application;

[0029] Fig. 9 is a block diagram of a target object appearance frame timestamp determination unit shown in an embodiment of the present application;

[0030] Fig.10 is another block diagram of a circuit for storing a point cloud according to an embodiment of the present application;

[0031] Fig.11 is a schematic diagram of a point cloud frame to be written into a non-volatile memory according to an embodiment of the present application;

[0032] Fig.12 is a block diagram of a chip shown in an embodiment of the present application;

[0033] Fig.13 is a block diagram of a chip shown in another embodiment of the present application;

[0034] Fig.14 is a schematic diagram of the structure of a radar shown in an embodiment of the present application;

[0035] Fig.15 is a flow chart of a method for storing point clouds shown in an embodiment of the present application;

[0036] Fig.16 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0038] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "an" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0039] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0040] In order to facilitate the understanding of the present application, some concepts involved in the present application are first explained.

[0041] Point cloud: Point cloud is a data set of points in a specific coordinate system. Point cloud data includes a variety of information, such as three-dimensional coordinates (X, Y, Z), color, classification value, intensity value, time, etc. Point cloud data can be used in autonomous driving, assisted driving, mapping, machine vision and other fields. Point cloud data can be generated by Light Detection and Ranging (LIDAR).

[0042] Vehicle-mounted laser radar: By emitting outgoing light (such as a laser beam) with a wavelength of about 900nm, the outgoing light will be reflected by the obstacle after encountering it. The processing unit calculates the distance between the obstacle and the vehicle-mounted laser radar based on the time difference between the reflected light and the outgoing light. In addition, the processing unit can also estimate the reflectivity of the target based on the cross-section of the reflected light signal obtained after receiving the reflected light, so as to generate point cloud data. Vehicle-mounted laser radar has the advantages of small size and high degree of integration.

[0043] In the fields of smart transportation / autonomous driving, the key point is to quickly and accurately perceive the surrounding environment of roads / unmanned vehicles. According to the road, vehicle position and obstacle information obtained by the sensor equipment, the road signal control is coordinated to improve the quality and efficiency of road management. The decision-making of unmanned vehicles is controlled, and the safe distance between unmanned vehicles is adjusted to ensure that vehicles can travel safely and reliably on the road. As one of the most important autonomous driving sensors, LiDAR can provide the decision-making system with the most important information about the location / size / motion of traffic participants in the fields of smart transportation / autonomous driving.

[0044] In the autonomous driving scenario, the system architecture suitable for the autonomous driving scenario may include mobile devices, networks, and the cloud. The cloud may include a server cluster or a distributed system consisting of multiple physical servers. Electronic devices such as sensors may be provided on the mobile device to obtain information about obstacles in the surrounding environment of the mobile device. Electronic devices may include radars (such as lidars), image sensors, etc.

[0045] Compared with image data, point cloud data has distance information, reflectivity information, etc., and can more comprehensively represent spatial location information. However, as shown above, the amount of point cloud data is huge. When point cloud data needs to be stored as evidence or backup data, it takes up a huge amount of storage resources. This is unacceptable for application scenarios that continuously generate massive amounts of data (such as autonomous driving scenarios) and need to be stored on mobile platforms.

[0046] Specifically, the LiDAR main control chip is the core component of the LiDAR, which is responsible for the collection, processing and transmission of all LiDAR information. However, the point cloud map collected by the LiDAR contains redundant and invalid information most of the time, and the massive amount of invalid information also makes it difficult to search for effective information. For example, if a user wants to find the relevant point cloud data of a certain type of object (such as an ambulance) in the driving record, a lot of manual search is required and there is a possibility of missed detection. This will cause a huge waste of on-board storage resources and great difficulty in searching for effective information.

[0047] Point cloud data includes unordered point cloud data, and it is difficult to retrieve target point cloud frames (such as point cloud frames collected in specific scenarios) from massive point cloud data. For example, it is difficult for users to find point cloud frames when the mobile platform passes through dangerous sections from the stored multi-frame point cloud to review driving habits. For example, it is difficult for users to find point cloud frames when the vehicle overtakes or is overtaken from the stored multi-frame point cloud to review dangerous driving behaviors. For example, it is difficult for users to find point cloud frames collected in scenes related to specific types of objects (such as special vehicles passing by such as police cars and ambulances) from the stored multi-frame point cloud to view more relevant information. For example, it is difficult for users to find point cloud frames when vehicles collide from the stored multi-frame point cloud to use as evidence. For example, useful point cloud data stored by users in non-volatile memory is overwritten by useless point cloud data collected subsequently, resulting in the loss of useful information that cannot be recovered.

[0048] The embodiments of the present application provide a circuit and method, a chip, and an electronic device for storing point clouds. After collecting multiple point cloud frames, the multiple point cloud frames are not directly written into a non-volatile memory, but the recognition results for the target object in each point cloud frame are obtained, and the association relationship between the target object and the multiple point cloud frames is determined. In this way, the recognition results and the multiple point cloud frames corresponding to the recognition results can be output in association with each other based on the association relationship, so that the useful point cloud frames (with target objects) and the corresponding recognition results can be written into the non-volatile memory. In this way, the proportion of useful data stored can be effectively increased, and it is convenient to retrieve the target point cloud frame based on the recognition results and the association relationship, which improves the convenience and efficiency of retrieval.

[0049] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0050] Figure 1 It is a schematic diagram of an application scenario of a circuit and method for storing a point cloud, a chip, and an electronic device shown in an embodiment of the present application.

[0051] Figure 1 The hardware structure of the vehicle 10 supporting assisted driving or automatic driving functions is shown. For example, at least one LIDAR 11 is mounted on the roof and / or the side of the vehicle body of the vehicle 10. The detection area of ​​the LIDAR 11 can be fixed, such as a certain LIDAR 11 can be used only for detection of a preset area. The detection area of ​​the LIDAR 11 can be adjustable, such as the laser radar on the vehicle body can scan multiple detection areas by adjusting the posture, etc., or by adjusting the field of view angle range of the laser radar itself. In addition, the vehicle 10 can also be equipped with multiple millimeter-wave radars in a manner surrounding the vehicle 10. For example, the vehicle 10 is equipped with 4 millimeter-wave radars, with the left side in front of the vehicle, the right side in front of the vehicle, the left side behind the vehicle, and the right side behind the vehicle as the detection range. Through the millimeter-wave radar, the distance of the object existing in each detection area can be detected, and the relative speed of the object and the vehicle 10 can be detected.

[0052] Furthermore, the vehicle 10 may also be equipped with a positioning device 12 , such as a Beidou positioning device, a Global Positioning System (GPS), etc. The current position of the vehicle 10 can be determined by the positioning device 12 .

[0053] In addition, the vehicle 10 may also be equipped with an electronic control unit (ECU). The detection signal of at least one of the above-mentioned LIDAR 11, millimeter wave radar and positioning device 12 is sent to the ECU. The ECU can detect and identify obstacles (such as roadblocks, moving objects, trees, adjacent vehicles, etc. around the vehicle 10) based on these signals. In addition, the ECU can be physically divided into multiple units according to their functions, and this application will collectively refer to them as ECUs.

[0054] It should be noted that although the movable device is described as a car, such description is not restrictive and is applicable to a variety of movable devices, such as land robots, water robots, etc.

[0055] Figure 2 It is a schematic diagram of another application scenario of the circuit and method for storing point clouds, chip and electronic device shown in one embodiment of the present application.

[0056] Figure 2The aircraft 20 shown may be an aircraft with an assisted driving function and an automatic driving function. The aircraft 20 may include a sensor system 21 and a power mechanism 22 .

[0057] The sensing system 21 may include one or more sensors to sense at least one of the surrounding obstacles, spatial orientation, speed or acceleration of the aircraft 20. The types of sensors include, but are not limited to, ranging sensors, position sensors, motion sensors, inertial sensors or image sensors. The sensing data provided by the sensing system 21 can be used to control the spatial orientation, speed and / or acceleration of the aircraft 20. The sensing system 21 is used to collect relevant information of the aircraft 20. Different types of sensors can sense different types of signals or sense signals from different sources. For example, the sensor includes a radar, an inertial sensor, a GPS sensor, or a visual / image sensor (such as a camera).

[0058] The circuit and method for storing point clouds, the chip and the electronic device of the embodiment of the present application can be applied to Figure 1 or Figure 2 The arrangement shown is any one or more electronic devices that generate point clouds, such as LIDAR 11, millimeter wave radar, etc., and need to store point cloud data.

[0059] In the above-mentioned application scenarios such as assisted driving, autonomous driving, and smart transportation, quickly and accurately perceiving the surrounding environment of movable equipment is a key point.

[0060] Here, the vehicle's autonomous driving scenario is used as an example for illustrative explanation. According to the vehicle position information, obstacle information, and road information perceived by the sensor system, road signal control is coordinated to improve the quality and efficiency of road management. Specifically, the corresponding autonomous driving vehicle decision can be determined based on the information perceived by the sensor system, and the safe distance between autonomous driving vehicles can be adjusted, so that the vehicle can drive safely and reliably on the road.

[0061] As one of the most important sensors for autonomous driving, LiDAR can provide the decision-making system with particularly important information such as the location, size, and motion information of traffic participants in the fields of smart transportation, assisted driving, and autonomous driving. LiDAR is equivalent to human eyes for autonomous vehicles. Non-volatile memory is equivalent to the human brain and needs to store the collected point cloud data. However, the storage space of non-volatile memory is limited, and the time required to retrieve the target point cloud frame increases with the amount of stored point cloud data. How to ensure that the valid point cloud data is stored completely and that the target point cloud frame can be easily retrieved has become an urgent problem to be solved.

[0062] Based on the above reasons, the present application aims to provide a technical solution that can utilize limited storage space to store as much useful point cloud data as possible and improve the convenience of data retrieval.

[0063] Figure 3 It is a block diagram of a circuit for storing a point cloud according to an embodiment of the present application.

[0064] See also Figure 3 The circuit 300 for storing point clouds may include: a cache module 310 , a recognition result acquisition module 320 , an assembly module 330 and an output module 340 .

[0065] Among them, the cache module 310 is configured to cache point cloud data, and the point cloud data includes multiple frames of point cloud frames. The cache module 310 refers to a memory that can perform high-speed data exchange. It exchanges data with the processor before the memory, so the rate is very fast. The cache has the following main functions: pre-reading, writing, and temporary storage. The temporary storage and writing functions are at least partially involved in this application. The cache can be responsible for the storage and placement of point cloud data. This facilitates point cloud recognition, assembly of cached point cloud frames, and writing of target point cloud frames to non-volatile memory. In addition, the cache module 310 can also greatly reduce the time for data reading and writing to improve the use efficiency of the hard disk. At the same time, the use of cache can also reduce the frequent reading and writing of non-volatile memory, increase the service life of non-volatile memory, and save more power.

[0066] The recognition result acquisition module 320 is coupled with the cache module 310 and is configured to obtain recognition results for multiple point cloud frames and obtain recognition results for the target object. Specifically, the recognition result acquisition module 320 can call the recognition result, or obtain the recognition result by itself through the recognition algorithm. For example, the cached point cloud data can be sent to the processor or the cloud for point cloud recognition, and the recognition result is called by the recognition result acquisition module 320. For example, the main control chip of the laser radar can use a trained neural network to process the point cloud data to obtain the recognition result.

[0067] The assembly module 330 is coupled to the recognition result acquisition module 320, and is configured to determine the association relationship between the target object and the multiple point cloud frames in response to the recognition result indicating the existence of the target object. Specifically, the recognition result and the corresponding point cloud frame can be bound to establish an association relationship between the target object and the multiple point cloud frames. For example, if the recognition results of the i-th frame to the i+n-th frame indicate the existence of an ambulance, the recognition result "ambulance" is used as the label of the i-th frame to the i+n-th frame point cloud frame.

[0068] The output module 340 is coupled to the recognition result acquisition module 320 and the assembly module 330, respectively, and is configured to output the recognition result and the multiple point cloud frames corresponding to the recognition result in association with each other based on the association relationship, so as to write them into the non-volatile memory. The output module 340 can output the recognition result and the corresponding point cloud frame in association with each other to other electronic components, such as the non-volatile memory, the processor, etc. For example, the recognition result "ambulance" and the point cloud frames from the i-th frame to the i+n-th frame are written into the hard disk in association with each other.

[0069] It should be noted that, since the point cloud data includes accurate distance information, after determining that the recognition result indicates the existence of the target object, it is also possible to further determine whether the target object needs to be screened out based on the distance information between the radar and the target object. For example, the current vehicle of the radar is in lane 1, and the ambulance is in lane 4. Although the recognition result of the point cloud data obtained by the radar includes the ambulance, the ambulance is far away from the current vehicle, and the probability that the point cloud data needs to be retained is small. Therefore, there is no need to store the point cloud data, nor is there a need to build an association between the ambulance and the point cloud frame, thereby reducing the consumed computing resources. In addition, the relative speed between the mobile platform and the target object can be further considered to further improve the accuracy of the determined point cloud data to be stored. For example, although the two stationary cars are very close to each other, the relative speed is very small. If the distance between the two parties is not further reduced, there is no need to retain the point cloud data.

[0070] In addition, in addition to geometric position information, some point cloud data also have color information. Color information is usually obtained by acquiring color images through a camera, and then the color information (RGB) of the pixels at the corresponding position is assigned to the corresponding points in the point cloud. The acquisition of intensity information is the echo intensity collected by the laser scanner receiving device. This intensity information is related to the surface material, roughness, incident angle direction of the target, as well as the emission energy of the instrument and the laser wavelength. Therefore, the identification information of some signboards can also be identified from the point cloud data, such as dangerous road section information, no turn information, etc. That is, the target object can be a physical object, or it can be indication information, such as danger warning information, tidal lanes, etc.

[0071] Figure 4 It is a schematic diagram of a recognition process for a point cloud frame shown in an embodiment of the present application.

[0072] See also Figure 4 , you can first graph the point cloud data to obtain multiple point cloud frames, and then perform image recognition on each point cloud frame. You can also directly perform point cloud recognition on the point cloud data.

[0073] For example, the point cloud recognition process may include operations such as collecting point cloud data, point cloud filtering, sampling, feature extraction, feature matching, recognition, and outputting recognition results.

[0074] First of all, the raw data collected by the LiDAR generally contains noise and needs to be denoised. For example, radius filtering and statistical filtering can be used. Radius filtering: It searches for the number of adjacent point clouds within a given radius R. When the number is less than a given threshold, the point cloud is removed. Statistical filtering: For each point, a neighborhood statistical analysis is performed, and based on the statistical data, the sample points to be removed or retained are determined.

[0075] Since the amount of point cloud data is large, reaching several Gb or even Tb level, the data contains a lot of redundancy, which brings challenges to processing speed and storage. In addition, since the recognition speed is high during high-speed driving of the car, the filtered point cloud data can be sampled and processed to reduce the amount of calculation and improve the recognition speed. When sampling, it is necessary to ensure the boundary distribution of the original form of the point cloud, and the edge information cannot be destroyed, and edge preservation is required.

[0076] Next, feature vector extraction is performed. For example, feature extraction is performed on the sampled point cloud data. Features include but are not limited to: signature histogram of orientation (SHOT), fast point feature histogram (FPFH) descriptor, etc.

[0077] Then, feature matching is performed. The corresponding relationship is formed through feature matching, and the matching similarity is calculated from the matching point pairs to obtain the recognition result.

[0078] Figure 4 There are identified pedestrians riding bicycles, vehicles on the roadside, blast furnace isolation, etc. If pedestrians riding bicycles (motorcycles) belong to the target object, then the point cloud frame where the pedestrians riding bicycles appear can be retained. In addition, the distance information can be further integrated to determine whether the target object that appears needs to be screened out. For example, although a pedestrian riding a bicycle appears in the point cloud frame, the pedestrian riding a bicycle is still relatively far away from the vehicle. If the distance is greater than the preset threshold, the corresponding point cloud frame may not be output. If the distance between the pedestrian riding a bicycle and the vehicle is less than the preset threshold, the corresponding point cloud frame may be output.

[0079] The circuit for storing point clouds provided in this embodiment can intelligently filter point cloud frames of specific objects for storage. For example, edge computing based on artificial intelligence can filter the content after the laser radar collects point cloud data, and only retain and store valid point cloud data where specific objects appear. This greatly reduces the demand for on-board storage and the difficulty of retrieving key information.

[0080] Figure 5 It is a block diagram of a recognition result obtaining module shown in an embodiment of the present application.

[0081] See also Figure 5 The above-mentioned recognition result obtaining module 320 may include: a target frame timestamp determining unit 321 and a target frame position determining unit 322 .

[0082] Among them, the input end of the target frame timestamp determination unit 321 is coupled with the output end of the cache module 310, and is configured to determine the timestamp of the target point cloud frame. The target point cloud frame is a point cloud frame that includes the target object. In order to reduce the consumption of computing resources, a rough timestamp calculation method can be used. For example, if the frame rate of the point cloud frame is 30fps, then every 30 frames, add 1 second to the time, and the timestamp +1 can be used. In order to improve the accuracy of the stored point cloud frames and the accuracy of the retrieval results, a more sophisticated timestamp calculation method can also be used. For example, for each frame received, the timestamp is increased by 1. The timestamp is added to each point cloud frame by counting the number of frames to ensure the uniqueness of the identification of each point cloud frame.

[0083] The input end of the target frame position determination unit 322 is coupled to the output end of the target frame timestamp determination unit 321 , and is configured to determine the position of the target point cloud frame in the multi-frame point cloud frame based on the timestamp of the target point cloud frame.

[0084] For example, the target object ambulance appears in the current point cloud frame, and the timestamp of the point cloud frame is timestamp x. The target object ambulance appears in the subsequent 100 frames. The position of the target point cloud frame is: timestamp x to timestamp x+100. The target point cloud frame can be found by finding the point cloud frame corresponding to timestamp x to timestamp x+100.

[0085] Figure 6 It is a block diagram of a target frame timestamp determination unit shown in an embodiment of the present application.

[0086] See also Figure 6 The target frame timestamp determination unit 321 includes: a frame number counting circuit 3211, an identification circuit 3212 and a target object appearance frame timestamp determination unit 3213.

[0087] The input end of the frame number counting circuit 3211 is coupled to the output end of the buffer module 310, and is configured to count the frame numbers of multiple point cloud frames to determine the timestamp of the current point cloud frame, and transmit the multiple point cloud frames. Specifically, a timestamp can be generated for each point cloud frame by a counter, an accumulator, etc.

[0088] The input end of the recognition circuit 3212 is coupled to the output end of the frame number counting circuit 3211, and is configured to recognize each frame in the multi-frame point cloud frame to obtain the recognition result for each target object in the current point cloud frame. Specifically, the model topology and model parameters can be received to obtain the recognition model, so as to process the point cloud data or point cloud frame using the recognition model to obtain the recognition result for the target object.

[0089] The target object appearance frame timestamp determination unit 3213 is coupled to the frame number counting circuit and the recognition circuit respectively, and is configured to obtain the timestamp of the point cloud frame where each target object appears based on the timestamp of the current point cloud frame and the recognition result of each target object in the current point cloud frame. For example, the timestamp corresponding to the target object can be found through the search circuit, and then the point cloud frame corresponding to the timestamp can be found from the cache based on the timestamp.

[0090] The model parameters can be obtained by training a recognition model with a specific model topology using training data. The topology of the recognition model includes, but is not limited to: Linear Regression, Logistic Regression, Decision Trees, Naive Bayes, K-Nearest Neighbors, Support Vector Machines, Random Decision Forests or Bagging, Neural Networks, etc.

[0091] For example, the training data packet includes multiple point cloud frames, including a point cloud frame of a preset target object with a preset target object identifier. For example, point cloud frame 1 has a prohibition sign label, point cloud frame 2 has a close person label, point cloud frame 2 has a prohibition sign label and a close person label, etc. The training data can be divided into training data for multiple scenarios. The recognition model can be a recognition model composed of multiple recognition sub-models obtained by training multiple recognition sub-models respectively according to training data for different scenarios. The recognition model can be a model obtained by training the same recognition model using training data for different scenarios. Scenarios include but are not limited to: congested sections, highways, residential roads, national highways, high wind areas, slope areas, etc. This helps to improve the recognition accuracy of the recognition model.

[0092] Figure 7 It is a block diagram of a frame number statistics circuit shown in an embodiment of the present application.

[0093] See also Figure 7 , where the frame counting circuit is realized by an adder counter as an example, a binary adder counter can be formed by a trigger, such as a k-bit synchronous binary adder counter formed by a T trigger. For example, a 3-bit binary asynchronous adder composed of 3 falling edge triggered triggers T, the output signal of the trigger T is controlled by a pulse signal, and the counter starts counting from the state of Q2 Q1Q0 = 000.

[0094] Under the trigger of the falling edge of the counting input pulse CP, the output Q0 of the trigger FF0 will flip. 0 becomes 1 or 1 becomes 0. Since CP1 is taken from Q0, the output Q1 of the trigger FF1 will flip under the trigger of the falling edge of Q0. Similarly, since CP2=Q1, the output Q2 of the trigger FF2 will flip under the trigger of the falling edge of Q1. If the trigger T′ is triggered by the rising edge, an asynchronous binary addition counter can also be formed, but the carry pulse of each level of the trigger should be changed to the Qˉ terminal output. This is because when the output terminal Q of the low-order trigger changes from 1 to 0, the rising edge of the Qˉ terminal can just be used as a trigger pulse for the high-order.

[0095] Taking the subtraction counter as an example, if the triggers T′ are connected according to the binary subtraction rule, a binary subtraction counter can be obtained. According to the binary subtraction counting rule, if the low-order trigger is already 0, it should flip to 1 after inputting a subtraction counting pulse, and at the same time send a borrow signal to the high-order bit to flip the high-order bit.

[0096] In addition, it should be noted that counting can also be done by software. For example, the operation: n&(n–1) can be repeatedly performed until n becomes 0. The function of the operation: n&(n–1) is to remove the last 1 of n, that is, the last 1 of n becomes 0. For example, let n=11111, then the calculation process is as follows: n=11110, the count is 1; n=11100, the count is 2; n=11000, the count is 3; n=10000, the count is 4; n=00000, the count is 5.

[0097] In some embodiments, the target object appearance frame timestamp determination unit includes: a plurality of parallel timestamp determination subunits, each timestamp determination subunit corresponding to a target object.

[0098] Figure 8 It is a schematic diagram of the recognition result for a point cloud frame shown in an embodiment of the present application.

[0099] See also Figure 8 , taking the model topology of using a convolutional neural network (CNN) as a recognition model as an example for exemplary explanation. The recognition result of CNN may include confidence levels for multiple preset results. For example, the training data includes training samples for multiple target objects. If a point cloud frame includes a target object, a label of the target object may be set for the point cloud frame. Figure 8The recognition results shown in FIG. 1 include close-up vehicles, close-up people, prohibition signs, close-up animals, collisions, close-up vertical poles, etc. After a frame of point cloud is input into CNN, the confidence for each preset target object is obtained. If the confidence of a preset target object is higher than the preset confidence threshold, the recognition result representing the preset target object in the point cloud frame is output. It should be noted that multiple preset target objects can exist simultaneously in a frame of point cloud.

[0100] Figure 8 If the confidence of the prohibition sign in the image exceeds the confidence threshold, the recognition result corresponding to the prohibition sign is output. Since each preset target object has a corresponding timestamp determination subunit, the timestamp determination subunit corresponding to the prohibition sign obtains the timestamp from the frame number statistics circuit when receiving the recognition result, and sets the timestamp for the point cloud frame in which the prohibition sign currently exists. In this way, an accurate timestamp can be set for each point cloud frame in which a target object exists. In addition, since each timestamp determination subunit corresponds to a target object, when there are multiple target objects in a point cloud frame, the same timestamp can be set for each target object in the current point cloud frame. In this way, when retrieving a certain target object, all timestamps corresponding to each target object can be accurately found, and then all point cloud frames in which a certain target object exists can be found based on all timestamps.

[0101] In some embodiments, in order to reduce the chip area occupied by the target object frame timestamp determination unit and improve the signal processing speed, it can be achieved by multiplexing the timestamp determination subunit. Specifically, multiple preset target objects can reuse the same timestamp determination subunit. For example, the timestamp determination subunit may include a table lookup circuit. When the CNN output result is 00100000, indicating the presence of a prohibition mark, the lookup table (Look-Up-Table, LUT for short) circuit selects the channel corresponding to the prohibition mark to be turned on, so as to set the timestamp for the current point cloud frame with the prohibition mark.

[0102] The target object appearance frame timestamp determination unit can achieve: determining the point cloud frame where the target object exists from multiple preset target objects, and setting an accurate and unique timestamp for the point cloud frame where the target object exists. Figure 8 , setting a timestamp for the current point cloud frame where the prohibition sign appears. For example, the target object appearance frame timestamp determination unit may include a multiplexer (MUX for short). The inputs of the multiplexer each correspond to a preset target object, and are configured to determine the existing target object from multiple preset target objects, so as to set a timestamp for the point cloud frame including the target object. Fig. 9 , the multiplexer may be a selector. It should be noted that, in order to switch the path of the preset target object, at least one of a wiring switch, a lookup table circuit, etc. may also be used.

[0103] For example, the implementation logic of the two-choice MUX can be shown as formula (1).

[0104]

[0105] Among them, Y is the signal output by MUX, S is the control signal, NOT S, A and B are the two signals input to the MUX. In addition, a buffer (such as a single NOT gate) can be connected to the output of the multiplexer to provide a positive logic output (logic "1", high) at one terminal and a complementary negative logic output (logic "0", low) at the buffer output.

[0106] It should be noted that the logic operation circuits and combinations thereof shown above are only illustrative examples and are not limited here. It should be noted that logic operations include but are not limited to: "OR", "AND", "NOT", "NOR", "NAND", "XOR", etc. Logic operation circuits include but are not limited to: "OR operation circuit", "AND operation circuit", "NOT operation circuit", "OR NOR operation circuit", "NAND operation circuit", "XOR operation circuit", etc. These logic operation circuits may also be referred to as logic operation units. In order to realize functions such as counting, threshold comparison, and determination of target point cloud frames, a combination of multiple logic operation circuits may be used to achieve this. Accordingly, a combination of multiple logic operation circuits may be used to achieve this, which is not limited here.

[0107] Fig.10 It is another block diagram of a circuit for storing a point cloud according to an embodiment of the present application.

[0108] See also Fig.10 The target frame position determination unit may include: a plurality of parallel target object position determination subunits. Each target object position determination subunit has a one-to-one corresponding and coupled timestamp determination subunit, and is coupled to the cache module. Each of the plurality of target object position determination subunits is respectively configured to determine a frame position range in which a target object exists in a neighboring frame of a current frame from the cached point cloud data, and output the frame position range to the assembly module.

[0109] In this embodiment, the frame position may refer to a certain frame, such as the 0th frame, the i-th frame, etc., where i is a positive integer greater than or equal to 0. The setting of the timestamp facilitates the determination of the correspondence between multiple point cloud streams. For example, if the recognition result output by the CNN indicates that there are both a prohibition sign and a close person in the current point cloud frame, it is necessary to set the identifier 1 for the current point cloud frame with the prohibition sign in order to subsequently search for the current point cloud frame, and set the identifier 2 for the current point cloud frame with the close person in order to subsequently search for the current point cloud frame. The timestamp can synchronize identifier 1 and identifier 2 to the current point cloud frame.

[0110] In addition, in order to reduce the chip area occupied by the target frame position determination unit and improve the signal processing speed, it can be achieved by multiplexing the target object position determination subunit. Specifically, multiple parallel timestamp determination subunits can reuse the same target object position determination subunit. For example, the target object position determination subunit may include a table lookup circuit, and when a certain timestamp determination subunit outputs a timestamp, the table lookup circuit selects the channel corresponding to the certain timestamp determination subunit to be turned on to determine the frame position range corresponding to the timestamp.

[0111] It should be noted that the specific value of the timestamp of the first point cloud frame of the point cloud stream (also called the stream timestamp) is not very important. For example, the first frame of the point cloud stream has a stream timestamp of 0 (it can also be the current system time), and the point cloud frame duration is 20ms; the second frame stream timestamp is 20, and the third frame is incremented in sequence. There are errors in the timestamp conversion between different time bases. When converting, it should be ensured that the converted timestamp is not less than the timestamp before the conversion. For another example, if the sampling frequency is 80000Hz, the timestamp unit is 1 / 80000, assuming that 1 second is divided into 80000 time blocks. If 25 frames are sent per second, the transmission of each frame occupies 80000÷25=3200 time blocks. Therefore, if the timestamp increment is defined as the time interval between two adjacent frames, the timestamp increment should be 3200. According to the above relationship, the conversion between timestamp and frame position range can be achieved.

[0112] In some embodiments, in order to reduce the risk of repeatedly storing the same point cloud frame and improve the effective utilization of storage space, the circuit for storing point cloud may further include: a deduplication module, coupled to the plurality of target object position determination subunits respectively, configured to perform deduplication processing on the frame position range to obtain a deduplicated frame position range. For example, please refer to Figure 8 , if the target objects "close person" and "prohibition sign" exist in the current point cloud frame at the same time, the two timestamp determination subunits corresponding to the two target objects both output timestamps, or the two target object position determination subunits corresponding to the two target objects both output frame position ranges, resulting in the frame position range being recorded twice, causing the current point cloud frame to be stored twice, wasting limited storage space. In this embodiment, after determining the correspondence between the target object and the point cloud frame, the frame position that appears multiple times for a point cloud frame is deduplicated to ensure that the point cloud frame with the same frame position is only stored once, effectively reducing the possibility of repeated storage of the same point cloud frame, and helping to improve the effective utilization of storage space.

[0113] In some embodiments, the circuit for storing point cloud may further include: an encryption module. The encryption module is coupled to the assembly module 330 and is configured to encrypt multiple point cloud frames corresponding to the target object to obtain encrypted multiple point cloud frames to be stored. Specifically, the point cloud data may be encrypted by hardware encryption or software encryption.

[0114] For example, hardware encryption can be implemented in the following way. The laser radar main control chip (hereinafter referred to as the main control chip) generates a random number, sends the random number to the encryption chip, and the encryption chip encrypts it and sends it back to the main control chip. The main control chip determines whether the data is the expected random data. If it is, the program of the main control chip continues to execute, otherwise it reports an error or stops output, such as stopping writing point cloud data to the non-volatile memory.

[0115] Taking the logic encryption chip as an example, it can be protected and authenticated by an electrically erasable programmable read-only memory (EEPROM) and peripheral logic circuits through a simple identification (ID) number or a logic authentication password; this method is low-cost and simple to develop, but the protection capability needs to be improved. Taking the encryption chip with a fixed algorithm and a comparison authentication mode as an example, the comparison mode is adopted; the program in the protected CPU is complete, and there are several isolated comparison points between it and the encryption chip. Taking the chip core as an example, the encryption chip has become part of the software product, and the program in the encryption chip cannot be read or copied, making it difficult to crack.

[0116] In a specific embodiment, the encryption module may include: a key generation unit and an encryption unit. The key generation unit is coupled to the recognition result acquisition module, and is configured to generate a secret key based on at least part of the numerical value in the probability of the recognition result. For example, if the confidence that there is a prohibition sign in the current point cloud frame is 81.2628%, then the values ​​from the 3rd to the 5th digits can be taken as the basic values, and then a specific operation is performed on the basic values ​​to generate an encryption key. The secret key obtained in this way is highly random and has a high security level. The encryption unit is coupled to the key generation unit, and is configured to encrypt multiple point cloud frames containing target objects based on the secret key.

[0117] In addition, software encryption can also be performed in the following manner. Before sending information, call the information security module to encrypt the information, and then send the encrypted data. After the encrypted data reaches the recipient, it is decrypted and restored by the corresponding decryption software. The software encryption method has the following advantages: there are already standard security application programming interface (Application Programming Interface, referred to as API) products, easy to implement, and good compatibility.

[0118] In some embodiments, the output module 340 may include a write control unit to write the target point cloud frame into a non-volatile memory, wherein the non-volatile memory includes but is not limited to at least one of the following: an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, etc.

[0119] Specifically, the input end of the write control unit is coupled to the reorganization module or the encryption module, and the output end of the write control unit is coupled to the non-volatile memory, and is configured to write the recognition results and multiple point cloud frames corresponding to the recognition results into the non-volatile memory based on the association relationship.

[0120] For example, in scenarios such as traffic accidents, in addition to using the video data recorded by the dashcam as evidence, in this embodiment, the stored point cloud data with the target object can also be used as evidence. Compared with the image information without distance information, the point cloud data stored in this embodiment also stores accurate distance information, which can better restore the scene of the traffic accident and provide higher quality evidence. In addition, in this embodiment, only high-value point cloud data is saved, and useless point cloud data (such as point cloud frames without target objects) is not retained as much as possible, thereby improving the effective utilization of storage space. Compared with the storage method of overwriting old data with cached new data in related technologies, the risk of the target object's point cloud data being overwritten and unable to be restored is effectively reduced.

[0121] For example, in a backtracking scenario, since the corresponding relationship between the position range of the target object and the point cloud frame has been determined before storing the point cloud data, the user can quickly and efficiently retrieve the target point cloud frame based on the target object, thereby improving the retrieval efficiency and the accuracy of the retrieval results.

[0122] Fig.11 is a schematic diagram of a point cloud frame to be written into a non-volatile memory according to an embodiment of the present application.

[0123] See also Fig.11 In the current point cloud frame, there are two target objects, that is, there are two vehicles very close to the mobile platform, which may interfere with the mobile platform. Therefore, it is necessary to store the point cloud frames of the current point cloud frame and multiple adjacent point cloud frames with at least one target object in the non-volatile memory. This makes it easier to review bad driving habits or retrieve point cloud data as evidence to restore the scene at the time of the incident.

[0124] The circuit for storing point clouds provided in the present application can be applied to multiple fields such as detection, unmanned driving, drones, etc. The circuit for storing point clouds provided in the present application can achieve targeted storage of valid data and improve the convenience of data retrieval.

[0125] Another aspect of the present application also provides a chip.

[0126] Fig.12 It is a block diagram of a chip shown in an embodiment of the present application.

[0127] See also Fig.12 The chip 1200 includes: a distance calculation unit 1210 and the above-mentioned circuit 300 for storing point clouds.

[0128] The distance calculation unit 1210 is configured to calculate the distance between the object and the mobile platform based on the time difference between the emission time point of the laser and the reception time point of the reflected laser, and generate point cloud data.

[0129] The circuit 300 storing the point cloud is configured to output the multiple point cloud frames in which the target object exists in the point cloud data in association with each other.

[0130] Among them, the components of the circuit 300 for storing the point cloud can refer to the relevant descriptions in the above multiple embodiments, and will not be described in detail here.

[0131] Fig.13 It is a block diagram of a chip shown in another embodiment of the present application.

[0132] See also Fig.13 In some embodiments, the chip 1200 may further include: a laser receiving unit.

[0133] Among them, the laser receiving unit is coupled with the distance calculation unit and is configured to receive the laser reflected by the object. When the laser encounters an obstacle, it will be reflected and then received by the laser receiving unit. The laser receiving unit creates a set of point clouds based on the time when each laser beam is emitted and returned. The principle is to measure the propagation distance (Time of Flight, referred to as TOF) between the laser radar and the target object, analyze the reflected energy size, amplitude, frequency and phase of the reflected spectrum on the surface of the target object, and output the point cloud, thereby presenting the accurate three-dimensional structure information of the target object.

[0134] Specifically, the laser receiving unit can be composed of a receiving optical system, a photodetector, an amplifier (preamplifier, main amplifier), etc. The laser is calibrated by an optical lens and received by a photosensitive device (photodiode). After the photodiode receives light, it will generate a photocurrent of corresponding intensity depending on the light intensity. The current is amplified by the amplifier to output an electrical signal.

[0135] In some embodiments, the chip 1200 may further include a laser emitting unit.

[0136] The laser emitting unit is configured to emit laser light. Fig.13 ,The laser emitted by the laser emitting unit is reflected by the target object (such as a passerby) and then received by the laser receiving unit to form point cloud data.

[0137] The laser emitting unit works by emitting layers of laser beams outward. The more layers there are, the higher the accuracy will be, but it may also lead to a larger sensor size. For example, a high-quality lidar can emit more than 200 laser beams per second.

[0138] For the wavelength of the laser, laser transmitters with wavelengths of around 900nm and 1550nm can be used. High power can achieve a longer detection distance, and long wavelength can also improve anti-interference capabilities. Safety is ensured by limiting power and pulse time.

[0139] In some embodiments, a non-volatile memory may be integrated into the chip to improve the security of the point cloud data. In addition, in order to reduce the chip size or reduce the chip cost, an external non-volatile memory may be used to store the point cloud data.

[0140] For example, the chip 1200 may further include a non-volatile memory coupled to a circuit for storing point clouds, and configured to store multiple point cloud frames corresponding to a target object, or configured to store a target object and multiple point cloud frames corresponding to the target object.

[0141] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0142] Another aspect of the present application further provides a radar. The radar may include the chip or board as described above.

[0143] Fig.14 It is a schematic diagram of the structure of a radar shown in an embodiment of the present application.

[0144] See also Fig.14 The radar 1400 may include a chip or board as shown above, and the chip and / or board may include a circuit for storing point clouds as described above. The circuit may be provided on a board 1410, and a plurality of chips, such as a central control chip, may be provided on the board 1410. The board 1410 may be provided in a housing 1420.

[0145] Specifically, the radar 1400 may include a circuit for storing point clouds as described above, or a chip as described above. The radar may be a laser radar, a millimeter wave radar, an ultrasonic radar, etc. The radar may be a scanning radar or a non-scanning radar.

[0146] The key components of LiDAR include control hardware DSP (digital signal processor), laser driver, laser emitting light emitting diode, transmitting optical lens, receiving optical lens, APD (avalanche optical diode), TIA (variable transconductance amplifier) ​​and detector according to the signal chain of signal processing. Among them, except for the transmitting and receiving optical lenses, all are electronic components. With the rapid evolution of semiconductor technology, the performance is gradually improved while the cost is rapidly reduced. Optical components account for the higher cost of LiDAR.

[0147] Another aspect of the present application also provides a method for storing a point cloud.

[0148] Fig.15 It is a flowchart of a method for storing point clouds shown in one embodiment of the present application.

[0149] See also Fig.15 , the method for storing point clouds may include operations S1510 to S1540.

[0150] In operation S1510 , point cloud data is buffered, where the point cloud data includes a plurality of point cloud frames.

[0151] In operation S1520, a recognition result for the multiple point cloud frames is obtained to obtain a recognition result for the target object.

[0152] In operation S1530 , in response to the recognition result indicating that the target object exists, an association relationship between the target object and the multiple point cloud frames is determined.

[0153] In operation S1540, the recognition result and the multi-frame point cloud frames corresponding to the recognition result are output in association based on the association relationship to be written into the non-volatile memory.

[0154] In some embodiments, the above method may further include at least one of the following operations.

[0155] For example, in response to the recognition result indicating that the target object does not exist, writing of the point cloud data to the non-volatile memory is prohibited.

[0156] For example, in response to the recognition result indicating that the target object does not exist, the point cloud data obtained after the current moment overwrites the point cloud data within a preset time period before the current moment.

[0157] In some embodiments, the target objects may also be classified, for example, key target objects (such as target objects that are about to collide or have collided), candidate target objects (objects with a high probability of needing to retain point cloud data), etc. This facilitates the classified storage of point cloud data of different types of target objects, so as to further improve the security of point cloud data of key target objects.

[0158] For example, the point cloud data of the key target object is stored in the first storage area. The point cloud data stored in the first storage area cannot be overwritten and can only be deleted by the user. If the first storage area is full, the point cloud data of the key target object can be stored in the second storage area.

[0159] The point cloud data of the candidate target object can be stored in the second storage area. The point cloud data stored in the second storage area can be overwritten, but is stored in a first-in-first-out manner. The above method further improves the security and reliability of the point cloud data.

[0160] The above method is exemplarily described below in conjunction with a circuit or chip for storing a point cloud.

[0161] Please also see Fig.10 or Fig.13 Before executing the operation of this method, it is necessary to train the neural network (such as CNN) with the training data of the target object (such as the point cloud frame) to obtain the model parameters, such as generating a parameter table of the trained neural network.

[0162] The laser emitting unit emits laser, the laser receiving unit is responsible for receiving the reflected laser, and the distance calculation unit is responsible for converting the time difference between the received laser and the emitted laser into the calculated distance.

[0163] The cache module is responsible for storing the point cloud image obtained by the laser radar scanning, and sending the cached point cloud file to the frame counting circuit, the target object position determination subunit and the assembly module.

[0164] The frame counting circuit decodes the point cloud file into continuous point cloud images, and sends the decoded images to the recognition circuit and the timestamp determination subunit.

[0165] The recognition circuit is responsible for reading the parameter table of the trained neural network, performing point cloud recognition on the point cloud frame, and sending the recognition result to the timestamp determination subunit. For example, the recognition result of the CNN of the recognition circuit can recognize n objects. If the current image recognition result is that object 1 and object 3 exist in the image, the recognition results of object 1 and object 3 in the current image are sent to the timestamp determination subunit corresponding to object 1 and the timestamp determination subunit corresponding to object 3 respectively.

[0166] The timestamp determination subunit is responsible for querying the timestamp information of the point cloud file where the current point cloud frame where the target object appears is located based on the recognition result and the timestamp information of the current point cloud frame sent by the frame number statistics circuit, and then sending the timestamp information to the corresponding target object position determination subunit.

[0167] The target object position determination subunit is responsible for querying the frame position of the nearest decoded random read point in the current point cloud frame where the target object appears based on the timestamp of the current point cloud frame and the point cloud file. The frame position of the decoded random read point found is then sent to the de-overlapping determination frame unit.

[0168] The de-overlapping frame determination unit is responsible for removing the overlapping parts of the frame positions of the decoding random read points determined by the target object position determination subunit where multiple target objects appear, obtaining the frame positions of the decoding random read points that need to be intercepted in the end, and sending them to the assembly module.

[0169] The assembly module reads the frame position of the random read point according to the decoding, and sends its content to the encryption unit. The encryption unit is responsible for encrypting the intercepted new point cloud file according to the key, and the key comes from the key generation unit. Among them, the source information of the key generation unit comes from the decimal part of the probability data of the character judgment result of the recognition circuit. Then, a hash operation is performed on it to obtain the encrypted key. Thus, a highly random and highly secure key is obtained. The write control unit is responsible for writing the encrypted point cloud file to an external non-volatile memory.

[0170] The laser radar main control chip in this embodiment can realize the function of intelligently screening the point cloud frames of specific objects for storage. This embodiment can be based on edge computing of artificial intelligence. After the laser radar acquires the point cloud, the content is screened, and only the point cloud frames where valid objects and specific objects appear are retained and stored. This greatly reduces the storage requirements and also greatly reduces the difficulty of retrieving key information. In addition, the random key generation and encryption mechanism protects the security and reliability of the screening result files.

[0171] Another aspect of the present application further provides a device for storing point clouds. The device for storing point clouds may include multiple modules, and the functions of each module may refer to the functions of each module in the circuit for storing point clouds, which will not be described in detail here.

[0172] Fig.16 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0173] See also Fig.16 The electronic device 1600 may include a memory 1610 and a processor 1620. In addition, the electronic device 1600 may be provided with at least one of a clock generating circuit, a clock storage point cloud circuit, or a radar.

[0174] The processor 1620 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0175] The memory 1610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM may store static data or instructions required by the processor 1620 or other modules of the computer. The permanent storage may be a readable and writable storage device. The permanent storage may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1610 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1610 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0176] The memory 1610 stores executable codes, and when the executable codes are processed by the processor 1620 , the processor 1620 may execute part or all of the above-mentioned methods.

[0177] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0178] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0179] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A circuit for storing a point cloud, characterized in that: include: A cache module is configured to cache point cloud data, wherein the point cloud data includes multiple point cloud frames; A recognition result obtaining module, coupled to the cache module, configured to obtain a plurality of recognition results for a plurality of point cloud frames, wherein the plurality of recognition results include a recognition result indicating the presence of a target object; an assembling module, coupled to the recognition result obtaining module, and configured to determine an association relationship between the target object and the target point cloud frame in response to the recognition result of the existence of the target object; The target point cloud frame is a point cloud frame corresponding to the recognition result of the target object in the multiple point cloud frames; The output module is coupled to the recognition result acquisition module and the assembly module respectively, and is configured to output the recognition result of the target object and the target point cloud frame in association with each other based on the association relationship, so as to write them into a non-volatile memory.

2. The circuit according to claim 1, characterized in that The recognition result obtaining module comprises: a target frame timestamp determination unit, whose input end is coupled to the output end of the cache module and is configured to determine the timestamp of the target point cloud frame; A target frame position determination unit, whose input end is coupled to the output end of the target frame timestamp determination unit, is configured to determine the position of the target point cloud frame in multiple point cloud frames based on the timestamp of the target point cloud frame.

3. The circuit according to claim 2, characterized in that The target frame timestamp determination unit comprises: A frame number counting circuit, whose input end is coupled to the output end of the cache module, is configured to count the frame numbers of multiple frames of the point cloud frames to determine the timestamp of the current point cloud frame, and transmit multiple frames of the point cloud frames; An identification circuit, whose input end is coupled to the output end of the frame number counting circuit, and is configured to identify each frame in a plurality of point cloud frames to obtain an identification result for each target object in a current point cloud frame; The target object appearance frame timestamp determination unit is coupled to the frame number statistics circuit and the recognition circuit respectively, and is configured to obtain the timestamp of the point cloud frame where each target object appears based on the timestamp of the current point cloud frame and the recognition results of each target object in the current point cloud frame.

4. The circuit according to claim 3, characterized in that The target object appearance frame timestamp determination unit comprises: a plurality of parallel timestamp determination subunits, each of which corresponds to one of the target objects.

5. The circuit according to claim 4, characterized in that The target frame position determining unit comprises: A plurality of parallel target object position determination subunits, each target object position determination subunit having a one-to-one corresponding and coupled timestamp determination subunit, and coupled with the cache module, are configured to determine a frame position range in which the target object exists in adjacent frames of the current point cloud frame from the cached point cloud data, and output the frame position range to the assembly module.

6. The circuit according to claim 5, characterized in that Also includes: The deduplication module is coupled to the plurality of target object position determination subunits respectively, and is configured to perform deduplication processing on the frame position range to obtain a deduplicated frame position range.

7. The circuit according to claim 1, characterized in that The target point cloud frame includes multiple frames and also includes: The encryption module is coupled to the assembling module and is configured to encrypt multiple frames of the target point cloud frames to obtain multiple encrypted frames of point cloud frames to be stored.

8. The circuit according to claim 7, characterized in that The encryption module comprises: A key generation unit, coupled to the recognition result obtaining module, configured to generate a secret key based on at least part of the numerical value of the probability of the recognition result; An encryption unit is coupled to the key generation unit and is configured to encrypt multiple frames of the target point cloud frames based on the secret key.

9. A chip, characterized in that: include: a distance calculation unit configured to calculate the distance between the object and the mobile platform based on a time difference between a time point of emission of the laser and a time point of reception of the reflected laser, and generate point cloud data; The circuit for storing point clouds according to any one of claims 1 to 8 is configured to output the target point cloud frame in the point cloud data in an associated manner. 10 . A radar, comprising the circuit for storing point clouds according to any one of claims 1 to 8 or the chip according to claim 9 .

11. A method for storing a point cloud, characterized in that: include: Cache point cloud data, the point cloud data including multiple point cloud frames; Obtaining a plurality of recognition results for a plurality of point cloud frames, wherein the plurality of recognition results include a recognition result indicating the presence of a target object; In response to the recognition result of the existence of the target object, determining an association relationship between the target object and the target point cloud frame; The target point cloud frame is a point cloud frame corresponding to the recognition result of the target object in the multiple point cloud frames; The recognition result of the existing target object and the target point cloud frame are output in association with each other based on the association relationship to be written into a non-volatile memory.

12. The method according to claim 11, characterized in that The plurality of recognition results also include a recognition result in which the target object does not exist, and further include: In response to the recognition result that the target object does not exist, prohibiting the point cloud data from being written into the non-volatile memory; or In response to the recognition result that no target object exists, the point cloud data obtained after the current moment overwrites the point cloud data within a preset time period before the current moment.

13. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to perform the method according to claim 11 or 12.

Images