Device, method and system for laser radar data processing, and laser radar

Abstract

Apparatus, methods, and systems for lidar data processing, as well as lidar and vehicles are provided. The apparatus includes a storage circuit, a prefetch circuit, and a first processing circuit. The storage circuit is configured to store first calibration data of the lidar. The prefetch circuit is configured to: determine first index data; indexing a first subset of the first calibration data based on the first index data; the first subset is transmitted to a first processing circuit. The first processing circuit is configured to perform a first data calibration on the echo data based on the first subset. The device for laser radar data processing can reduce the overhead of storage resources of calibration data by the laser radar, and reduces the circuit cost of the laser radar while meeting the calibration requirements of the laser radar.

Description

Technical Field

[0001] This disclosure relates to the field of optical detection technology, and more particularly to an apparatus, method and system for lidar data processing, as well as lidar and carrier. Background Technology

[0002] Optical detection technology uses light as a medium to detect objects. Compared to ordinary light sources, lasers possess characteristics such as monochromaticity and good directionality, making them widely used for object detection. For example, LiDAR (Light Detection and Ranging) uses lasers to detect objects and has found applications in fields such as autonomous driving, drones, robot recognition, geographic mapping, and environmental monitoring. However, LiDAR still faces the challenge of high cost in its applications. Summary of the Invention

[0003] This disclosure provides an apparatus, method, and system for processing lidar data, as well as a lidar and a vehicle. It can reduce the overhead of internal storage resources in the lidar, lower the cost of the lidar, and facilitate the application of lidar in vehicles.

[0004] In a first aspect, an apparatus for processing lidar data is provided. The apparatus includes a storage circuit, a prefetch circuit, and a first processing circuit. The storage circuit is configured to store first calibration data of the lidar. The prefetch circuit is configured to: determine first index data; index a first subset of the first calibration data based on the first index data; and transmit the first subset to the first processing circuit. The first processing circuit is configured to perform a first data calibration on the echo data based on the first subset.

[0005] The prefetch circuit can use the first index data to index a first subset of the first calibration data and provide the first subset to the first processing circuit, which then performs first data calibration based on the first subset. When the lidar includes multiple first processing circuits, the first processing circuit does not store the complete calibration data of the lidar; instead, it can store partial calibration data (e.g., including the first subset). For the lidar as a whole, this device can meet the parallel processing requirements of the lidar by storing only one copy of the lidar's first calibration data, reducing the storage pressure on the lidar and helping to reduce the area overhead of the storage circuit. The reduction in storage circuit area can lower the cost of the lidar. In addition, the reduction in the storage overhead of the first processing circuit can save circuit area for the layout of logic circuits (e.g., the processing core part in the first processing circuit), which is beneficial for the pipelined layout and circuit design of logic circuits, thereby improving chip performance. Furthermore, due to the introduction of the first index data, the search range of the target calibration data has been narrowed when the prefetch circuit indexes the first subset, so that the amount of data in the first subset stored by the first processing circuit is greatly reduced compared to the complete calibration data, which can reduce the time consumption caused by searching for target data and improve calibration efficiency.

[0006] Optionally, the prefetch circuit is further configured to: determine a first trigger signal; and, based on the first trigger signal and first index data, trigger indexing a first subset or transmit the first subset to the first processing circuit. Thus, the first subset is transmitted to the first processing circuit under the control of the trigger signal.

[0007] Optionally, the first trigger signal is generated based on the signal indicating the start of the lidar scan. In this way, the prefetch circuit can utilize the time interval between the start of the lidar scan and the start of data processing by the first processing circuit to index or transmit data, thereby making full use of the data transmission time and improving the lidar's data processing efficiency.

[0008] Optionally, the storage circuit is also configured to store the second calibration data of the lidar. The prefetch circuit is further configured to: determine second index data; index a second subset of the second calibration data based on the second index data; and transmit the second subset to the first processing circuit. The first processing circuit is further configured to perform a second data calibration on the echo data based on the second subset. The type or level of the first data calibration differs from that of the second data calibration; the first index data and the second index data are different. The lidar calibration data may include multiple sets of calibration data (e.g., including first calibration data and second calibration data), and different sets of calibration data can be used for different types of data calibration or for different levels of data calibration, further reducing the storage overhead of the calibration data.

[0009] Optionally, the first calibration data is used to calibrate the echo data of the detector area set. The second calibration data is used to calibrate the echo data of the detector area group. The detector area group includes the detector area set, and the detector area set includes the detector areas. In this way, detector areas within the same detector area set can reuse the same calibration data, and detector areas within the same detector area group can reuse the same calibration data, further reducing the storage overhead of calibration data.

[0010] Optionally, the detection area set corresponds to the first sub-field of view of the lidar. The detection area group corresponds to the second sub-field of view of the lidar. The first sub-field of view is smaller than the second sub-field of view.

[0011] Optionally, the prefetch circuit is also configured to: determine a second trigger signal; and, based on the second trigger signal and the second index data, trigger a second subset of the index or transmit the second subset to the first processing circuit.

[0012] Optionally, the apparatus for processing lidar data further includes: a processing circuit group comprising multiple second processing circuits configured to determine multiple echo data in parallel; and a first processing circuit configured to perform a first data calibration on the multiple echo data based on a first subset. In this way, by allocating processing resources, processing resources can be utilized more rationally, improving data efficiency while further reducing the storage resource requirements for calibration data. Before calibrating the echo data, multiple second processing circuits can be used to preprocess the echo data in parallel, improving the efficiency of preprocessing the echo data.

[0013] Optionally, the processing circuit group includes 2 to 20 secondary processing circuits. This number setting can meet the requirements for echo data processing time while reducing the circuit area overhead of the processing device.

[0014] Optionally, the number of the first processing circuits is the same as the number of processing circuit groups.

[0015] Optionally, the apparatus for processing LiDAR data also includes a register. This register is configured to store configuration data for the LiDAR. The prefetch circuit is further configured to: determine third index data; index a subset of the configuration data based on the third index data; and transmit the subset of the configuration data to at least one of a first processing circuit or a plurality of second processing circuits. In this way, configuration data for the LiDAR can be prefetched, further reducing the storage overhead of the LiDAR on the configuration data and lowering the cost of the LiDAR. Similarly, the reduction in the storage overhead of the LiDAR on the configuration data can save area in the chip's storage circuitry, allowing the saved circuitry area to be used for logic circuit layout, further improving chip performance.

[0016] Optionally, the first calibration data also includes a third subset. The prefetch circuit is configured to: determine fourth index data; index the third subset based on the fourth index data; and transmit the third subset to the first processing circuit. The first processing circuit is further configured to perform third data calibration on the echo data based on the third subset. This improves the accuracy of determining the effective signal and background noise, further reduces the impact of highly reflective objects on the lidar's detection performance, and enhances the lidar's reliability.

[0017] In a second aspect, a method for processing lidar data is provided, the method comprising: determining first index data; indexing a first subset of first calibration data based on the first index data, wherein the lidar stores the first calibration data; and performing first data calibration based on the first subset.

[0018] Optionally, the method for processing lidar data further includes: determining a first trigger signal; triggering an index of a first subset or transmitting the first subset based on the first trigger signal and first index data.

[0019] Optionally, the method for processing lidar data also includes generating a first trigger signal based on the signal indicating the start of lidar scanning.

[0020] Optionally, the method for processing lidar data further includes: determining second index data; indexing a second subset of the second calibration data based on the second index data, and storing the second calibration data in the lidar; performing second data calibration based on the second subset, wherein the first data calibration and the second data calibration are of different types or levels, and the first index data and the second index data are different.

[0021] Optionally, the first calibration data is used to calibrate the echo data of the detection area set, and the second calibration data is used to calibrate the echo data of the detection area group, which includes the detection area set, and the detection area set includes the detection area.

[0022] Optionally, the method for processing lidar data further includes: determining a second trigger signal; triggering an indexed second subset or transmitting the second subset based on the second trigger signal and the second index data.

[0023] Optionally, a first data calibration is performed based on a first subset, including: determining the echo data; and performing a first data calibration on the echo data based on the first subset.

[0024] Optionally, the method for processing lidar data further includes: determining third index data; indexing a subset of configuration data based on the third index data, storing configuration data for the lidar; and configuring the lidar based on the subset of configuration data.

[0025] Optionally, the method for processing lidar data further includes: determining fourth index data; indexing a third subset of the first calibration data based on the fourth index data; and performing third data calibration based on the third subset.

[0026] Thirdly, a system for processing lidar data is provided. The system includes: a memory comprising multiple storage areas configured to store lidar echo data in blocks; and a device for processing lidar data according to the first aspect or any of the first aspects, configured to perform a first data calibration on the echo data.

[0027] Fourthly, a lidar is provided, comprising: a laser emitting circuit configured to emit a laser; a laser receiving circuit configured to receive an echo of the laser and convert the echo into an electrical signal, the electrical signal being used to generate echo data; a memory configured to store the echo data; and the apparatus of the first aspect or any one of the first aspects configured to perform a first data calibration on the echo data.

[0028] Fifthly, a vehicle is provided, including the lidar described in the fourth aspect above.

[0029] A sixth aspect provides a computer program product including instructions, wherein, when executed by a processor, the second aspect or any of the methods described above is performed.

[0030] A seventh aspect provides a computer-readable storage medium including instructions stored thereon, wherein, when the instructions are executed by a processor, the second aspect or any of the methods of the second aspect are performed. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the following description of the embodiments will be provided as examples. The drawings described below are merely embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain this disclosure and do not constitute a limitation of this disclosure.

[0032] Figure 1 A structural example diagram of a lidar consistent with some embodiments of this disclosure is shown.

[0033] Figure 2 A structural example diagram of a system for lidar data processing, consistent with some embodiments of this disclosure, is shown.

[0034] Figure 3A structural example diagram of an apparatus 220 for lidar data processing, consistent with some embodiments of this disclosure, is shown.

[0035] Figure 4 A structural example diagram of an apparatus for lidar data processing, consistent with some embodiments of this disclosure, is shown.

[0036] Figure 5 An example diagram illustrating the partitioning of calibration data for a lidar consistent with some embodiments of this disclosure is shown.

[0037] Figure 6 An example diagram is shown of the emitting area, the detection area, and the sub-field of view of a lidar consistent with some embodiments of this disclosure.

[0038] Figure 7 A structural example diagram of another apparatus for lidar data processing, consistent with some embodiments of this disclosure, is shown.

[0039] Figure 8 A structural example diagram of another apparatus 800 for lidar data processing, consistent with some embodiments of this disclosure, is shown.

[0040] Figure 9 A flowchart illustrating a method for processing lidar data, consistent with some embodiments of this disclosure, is shown.

[0041] Figure 10 A flowchart illustrating a method for processing lidar data, consistent with some embodiments of this disclosure, is shown. Detailed Implementation

[0042] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the specific implementation methods of this disclosure will be described below with reference to the accompanying drawings. The accompanying drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings or embodiments can be obtained based on these drawings or embodiments without creative effort. Adjustments and improvements made without departing from the concept of this disclosure are all within the protection scope of this disclosure.

[0043] To keep the drawings simple, each figure only schematically shows the parts related to the corresponding embodiment, and they do not represent the actual structure of the product. In addition, for the sake of simplicity and ease of understanding, only some structures or parts are schematically shown, and there may be more or fewer similar structures or parts in reality.

[0044] LiDAR (Light Detection and Ranging) uses laser light as a medium to detect objects and has found applications in a growing number of fields. These include autonomous driving (also known as automated driving or assisted driving, including any level of automated driving, such as L1-L5), industrial manufacturing, drones, robot recognition, geographic mapping, and environmental monitoring. LiDAR emits laser light during detection; when the laser encounters an object, it can be reflected back, and the reflected light is called an echo. LiDAR converts the received echo into an electrical signal and processes it to obtain information about the object, such as its distance, position, speed, or one or more other parameters, or even its three-dimensional structure.

[0045] In applications, LiDAR can be installed on vehicles to provide them with perceptual data reflecting information about objects, such as point cloud data. This allows the vehicle to utilize the perceptual data for analysis, decision-making, or control, among other functions. Vehicles include, but are not limited to, vehicles, manufacturing terminals, ships, aircraft (e.g., flying vehicles or drones), robots (e.g., industrial robots or home robots), or surveying equipment. For example, in intelligent driving scenarios, LiDAR can be installed on vehicles; as the vehicle moves, the LiDAR can detect the surrounding environment, obtain perceptual data (e.g., point cloud data), and provide this data to the vehicle, enabling it to make decisions or implement control measures based on this perceptual data.

[0046] Figure 1 A structural example diagram of a lidar consistent with some embodiments of this disclosure is shown. Figure 1 In this example, a lidar 100 includes a laser emitting circuit 110, a laser receiving circuit 120, an optical system 130, a preprocessing circuit 140, and a control and processing system 150. The laser emitting circuit 110 emits a laser beam, which is then emitted after the optical system 130 adjusts the optical path. When the emitted laser encounters an object 10, it can be reflected by the object 10. At least a portion of the reflected light can be reflected back to the lidar 100. The light reflected back to the lidar 100 can be called an echo. The echo is then directed to the laser receiving circuit 120 after the optical system 130 adjusts the optical path. The laser receiving circuit 120 converts the echo into an electrical signal. The electrical signal can be preprocessed by the preprocessing circuit 140 to obtain echo data, which is then provided to the control and processing system 150. The control and processing system 150 processes the echo data to obtain sensing data, such as point cloud data. The control and processing system 150 can then send the sensing data to a vehicle. The vehicle can use the sensing data to perform one or more functions such as analysis, decision-making, or control.

[0047] and Figure 1Consistent with this example, the laser emitting circuit 110 may include a laser 111 and a driving circuit 112. The laser 111 emits laser light under the drive of the driving circuit 112, and the laser light exits through the optical system 130. The laser may include, for example, a semiconductor laser, a fiber laser, or other types of lasers. Semiconductor lasers may include, for example, an emitting circuit, a vertical cavity surface emitting laser (VCSEL), an edge emitting laser (EEL), a distributed feedback laser (DFB), or similar devices. The above are merely examples, and this disclosure does not limit the type of laser.

[0048] and Figure 1 Consistent, exemplarily, the laser receiving circuit 120 may include a detector 121 and a readout circuit 122. An optical system 130 may focus the echo onto the photosensitive surface of the detector 121. The detector 121 uses the photoelectric effect to convert the echo into an electrical signal. The electrical signal can be read out by the readout circuit 122. In some embodiments, the laser receiving circuit 120 may include multiple detectors 121. For example, multiple detectors 121 may be arranged in a one-dimensional or two-dimensional detector array. In some embodiments, the laser receiving circuit 120 may also include a gating circuit 123. The gating circuit 123 may be used to select some or all of the detectors 121. The selected detector 121 is in a state that can respond to the optical signal and convert the echo into an electrical signal. The detector may include: a photodetector circuit, a PIN photodiode (PINPD), an avalanche photodiode (APD), a single photon avalanche diode (SPAD), a silicon photomultiplier (SiPM), or similar devices. The above are merely examples, and the embodiments disclosed herein do not limit the type of detector.

[0049] and Figure 1Consistent, exemplarily, the optical system 130 may include emitting optical elements and receiving optical elements. The emitting optical elements, located in the emission path of the laser, are used to shape the laser emitted by the laser 111 and adjust the laser's exit path. The receiving optical elements, located in the receiving path of the laser, are used to collect the echo reflected back from the object and converge the echo onto the photosensitive surface of the detector 121. For example, the emitting optical elements may include one or more optical elements such as emitting lenses, mirrors, homogenizers, and beam splitters. For example, the receiving optical elements may include one or more optical elements such as receiving lenses, mirrors, filters, and beam splitters (or beam splitters). The emitting and receiving optical elements may be independent, partially multiplexed, or fully multiplexed. For example, a lidar may include independent emitting optical elements (e.g., independent emitting lenses) and receiving optical elements (e.g., independent receiving lenses). For example, a lidar may include optical elements shared by the emitting and receiving optical paths, such as beam splitters, for separating the emitting and receiving optical paths. For example, a lidar may include a shared lens for shaping the beams in the transmitting and receiving optical paths.

[0050] and Figure 1 Consistent, exemplarily, the preprocessing circuit 140 may include one or more of an amplification circuit, a filtering circuit, or a sampling circuit to perform one or more preprocessing operations such as amplification, filtering, or sampling. The amplification circuit may include an amplifier to amplify the electrical signal converted by the detector 121. The filtering circuit may include a filter to filter out noise or interference. The sampling circuit may include one or more of an analog-to-digital converter (ADC) or a time-to-digital converter (TDC). For example, an ADC can convert an analog electrical signal into a digital signal representing the echo waveform by periodically sampling the output signal of the detector 121, thus obtaining echo data. Similarly, a TDC can measure the arrival time of the echo by time sampling the output signal of the detector 121, thus obtaining echo data. For example, the current signal output by the detector 121 can be converted into a voltage signal and compared with a reference voltage to generate an over-threshold signal. The TDC measures the over-threshold time of the signal based on the received over-threshold signal, thus performing time sampling and obtaining echo data. The echo data may include data representing one or more parameters such as echo time or echo intensity.

[0051] and Figure 1Consistent, exemplarily, the control and processing system 150 can be used to process the echo data to obtain sensing data. The control and processing system 150 can also send a first control signal to the driving circuit 112 to control the driving circuit 112 to drive all or part of the lasers 111 to emit light. In some embodiments, the control and processing system 150 can also send a second control signal to the gating circuit 123 to control the gating circuit 123 to select all or part of the detectors 121 to be in a state that can respond to optical signals. The control and processing system 150 may include one or more processors. Processors may include, but are not limited to: application-specific integrated circuits (ASICs), hardware circuits implemented with programmable logic devices (PLDs), microcontroller units (MCUs), microprocessor units (MPUs), digital signal processors (DSPs), or central processing units (CPUs). Hardware circuits implemented with PLDs may include field-programmable gate arrays (FPGAs), etc. When the control and processing system 150 includes multiple processors, the processors may be of the same or different types. For example, the control and processing system 150 may include an MCU and an FPGA. The control and processing system 150 may also include an MCU, a DSP, and an FPGA. The control and processing system 150 may also include an MCU, an FPGA, and a CPU. The control and processing system 150 may also include a CPU and an FPGA, and so on. When the control and processing system 150 includes multiple processors, these processors may be configured separately, partially integrated, or fully integrated. In some embodiments, the control and processing system 150 may be implemented as a system-on-chip (SOC) or an ASIC.

[0052] and Figure 1Consistently, exemplarily, the lidar 100 (e.g., a mechanical lidar or a semi-solid-state lidar) may also include a scanning system 160. The scanning system 160 may include a scanner and a driving device. The driving device is used to drive the scanner to rotate, enabling the laser to scan one or all of the vertical or horizontal field of view. For example, the laser is emitted through the scanner, and rotation of the scanner can change the laser's emission path. Alternatively, the laser echo can be incident on the scanner and guided to a light receiving path. This disclosure does not limit the type of scanner. The scanner may include, but is not limited to, a rotating mirror, a tilting mirror, a galvanometer, or other components that can direct the laser towards different locations in the environment. For example, the scanning system 160 may include a rotating platform. One or more of a laser emitting circuit, a laser receiving circuit, or an optical system may be mounted on the rotating platform, and as the rotating platform rotates, one or all of the vertical or horizontal field of view is scanned. When the lidar 100 includes the scanning system 160, the control and processing system 150 can also control the movement of the scanning system 160.

[0053] After a lidar system is designed, its echo data can be calibrated to ensure that the measurements accurately reflect the actual information of the object. During lidar detection, echo data can be affected by environmental factors, differences in object reflectivity, and the lidar's own hardware, leading to discrepancies between the echo data and the actual object information. Lidar calibration can reduce the influence of at least one of these factors, minimizing the discrepancy between the echo data and the actual object information, thus improving the accuracy of the echo data and enhancing the lidar's detection performance. Echo data calibration is also known as compensation.

[0054] Figure 2 A structural example diagram of a system for lidar data processing, consistent with some embodiments of this disclosure, is shown. Figure 2Consistent, exemplarily, the system 200 includes a memory 210 and a device 220 for processing lidar data. The memory 210 can store lidar echo data. The device 220 can perform calibration and other processing on the echo data. The system 200 can be part of a lidar preprocessing circuit 140. The system 200 can also be part of a lidar control and processing system 150. The system 200 can also be independent of the preprocessing circuit 140 and the control and processing system 150. The system 200 can also be partially part of the preprocessing circuit 140 and partially part of the control and processing system 150. The control and processing system 150 may include a data processing circuit 151 for processing the echo data to obtain sensing data. The system 200 can provide calibrated echo data to the data processing circuit 151. The data processing circuit 151 obtains sensing data based on more accurate echo data, resulting in sensing data that more accurately reflects object information.

[0055] and Figure 2 Consistent, exemplarily, memory 210 may include multiple storage regions 211-1 to 211-N (N being a positive integer greater than 1). The multiple storage regions can store (e.g., in blocks) the echo data of the LiDAR. The echo data stored in one storage region can be referred to as a data tile. Storing echo data in blocks allows for flexible layout of the storage circuitry of memory 210, reducing the complexity of chip routing. For example, storage regions 211-1 to 211-N can be arranged around device 220. Device 220 can read echo data stored in more than one storage region serially or in parallel. Device 220 can perform parallel processing on the echo data (e.g., parallel processing of echo data stored in one or more storage regions) to improve data processing efficiency. The one or more storage regions may include, for example, some or all of the storage regions 211-1 to 211-N.

[0056] Figure 3 A structural example diagram of an apparatus 220 for lidar data processing, consistent with some embodiments of this disclosure, is shown. Figure 3 Consistent with this, exemplarily, device 220 may include interface circuitry 221 and multiple processing circuits 222-1 to 222-M (where M is a positive integer greater than 1). This disclosure does not limit the values ​​of M and N, and M and N may be equal or unequal. Interface circuitry 221 may be a peripheral interface of device 220 for communication between device 220 and other circuits. For example, device 220 may communicate with memory 210 via the peripheral interface. As another example, device 220 may communicate with data processing circuitry 151 via the peripheral interface. Processing circuits 222-1 to 222-M may operate in parallel to perform parallel calibration of the echo data.

[0057] In some embodiments, processing circuit 222-i includes a processing core and a data storage area, where i∈[1,M]. The data storage area can store complete calibration data for the lidar. The processing core, based on the currently processed echo data, calls the target calibration data corresponding to the echo data from the calibration data. The processing core can use the target calibration data to calibrate the echo data. The target calibration data used by processing circuit 222-i during calibration is related to the currently processed echo data, and the currently processed echo data for processing circuit 222-i is variable. Complete calibration data is stored in the data storage areas of processing circuits 222-1 to 222-M, enabling the processing core to call the calibration data when processing different echo data. While this method of storing calibration data can improve the calibration efficiency of processing circuit 222-i, it incurs significant storage overhead, which occupies a large circuit area, increasing the chip area and resulting in higher circuit costs for the lidar. Furthermore, limited chip area reduces the usable area for internal logic circuits (such as the processing core), hindering pipelined logic layout and circuit design, leading to decreased chip performance. Additionally, the lookup of target calibration data in the processing circuit 222-i incurs time consumption, and calibration efficiency needs further improvement.

[0058] Based on this, the present disclosure provides a solution such as an apparatus, method and system for processing lidar data, which can reduce the storage resource overhead of lidar for calibration data, thereby reducing the storage circuit area overhead of lidar, reducing the cost of lidar processing chip or improving the performance of lidar processing chip, and reducing lidar circuit cost while meeting lidar calibration requirements.

[0059] The following explanation is based on the accompanying drawings.

[0060] Figure 4 A structural example diagram of an apparatus for lidar data processing, consistent with some embodiments of this disclosure, is shown. Figure 4 Consistent, exemplarily, the device 400 may include a storage circuit 410, a prefetch circuit 420, and a first processing circuit 430. The storage circuit 410 may store calibration data D1 of the lidar. The prefetch circuit 420 may: determine index data K1; index a subset dt1 in the calibration data D1 based on the index data K1; and transmit the subset dt1 to the first processing circuit 430. The first processing circuit 430 may perform a first data calibration on the echo data based on the subset dt1. For ease of description, the calibration data D1 may be referred to as the first calibration data, the index data K1 as the first index data, and the subset dt1 as the first subset.

[0061] and Figure 4 Consistent with this, the prefetch circuit 420 can index a subset dt1 of the calibration data D1 based on the index data K1. The prefetch circuit can provide the subset dt1 to the first processing circuit 430. The first processing circuit 430 can perform data calibration based on the subset dt1. The first processing circuit 430 does not need to store all the calibration data of the lidar, but only a portion of the calibration data (e.g., including the subset dt1). When the lidar includes multiple first processing circuits 430, this can significantly reduce the demand for storage resources. For the lidar as a whole, the device 400 only needs to store one set of lidar calibration data D1 to meet the parallel processing requirements of the lidar, reducing the storage pressure on the lidar, which helps to reduce the area overhead of the storage circuit and lower the cost of the lidar. In addition, the reduction in the storage overhead of the first processing circuit 430 can save circuit area for the layout of logic circuits (e.g., the processing core part in the first processing circuit 430), which is beneficial to the pipeline layout and circuit design of logic circuits, thereby improving chip performance. Furthermore, due to the introduction of index data K1, the search range of calibration data has been narrowed when the prefetch circuit 420 indexes the subset dt1, so that the amount of data in the subset dt1 stored by the first processing circuit 430 is greatly reduced compared to the complete calibration data, which can reduce the time overhead caused by searching for calibration data and improve calibration efficiency.

[0062] This disclosure does not limit the content of the index data K1, as long as it can index the subset dt1. For example, the prefetch circuit 420 stores a correspondence that reflects the storage location of the index data and the corresponding subset. The prefetch circuit 420 can determine the storage location of the subset dt1 based on the index data K1 and the correspondence, and then read the subset dt1. In some embodiments, the index data K1 can be sent to the prefetch circuit 420 by the control and processing system, or the prefetch circuit 420 can determine the index data K1 based on the laser's emission timing or the detector's reception timing.

[0063] In some embodiments, the number of index data K1 may include one or more, and the number of subsets dt1 may include one or more. For example, the prefetch circuit 420 may obtain multiple subsets dt1 based on multiple index data K1 at once and provide them to the first processing circuit 430 at once. Alternatively, the prefetch circuit 420 may obtain multiple subsets dt1 based on multiple index data K1 in stages and provide them to the first processing circuit 430 in stages. Yet another example is that the prefetch circuit 420 may obtain multiple subsets dt1 based on one index data K1 at once and provide the multiple subsets dt1 to the first processing circuit 430 either at once or in stages.

[0064] This disclosure does not limit the range of subsets; it can be determined based on the processing capability of the first processing circuit 430 or based on data blocks. In some embodiments of this disclosure, when the prefetch circuit 420 prefetches a subset, the range of the subset is determined based on data blocks. For example, the number of subsets can be the same as the number N of storage areas included in the memory 210 (e.g., the same as the number of data blocks). The echo data obtained by the lidar in one scan can be stored in blocks in N storage areas. By prefetching N subsets, the prefetch circuit 420 can obtain calibration data for the echo data obtained in one scan. The first processing circuit 430 can use the calibration data provided by the prefetch circuit 420 to calibrate the echo data obtained in one scan, reducing the occurrence of insufficient calibration data acquisition range to cope with the calibration of one scan result, and improving the overall calibration efficiency. In addition, calibration data can be prefetched during the scan interval, reducing the occurrence of the first processing circuit 430 waiting for calibration data, further improving calibration efficiency.

[0065] The following is combined with Figure 5 The example shown is described below. Figure 5 An example diagram of prefetching calibration data for a lidar consistent with some embodiments of this disclosure is shown. Figure 5 Consistent, exemplarily, a lidar can include multiple detection zones. One detection zone can correspond to one detector. These multiple detection zones can be arranged as a detection zone array. The lidar's calibration data can be mapped to this detection zone array. A single lidar scan can trigger multiple detection zones in the detection zone array; these multiple detection zones corresponding to a single scan can be called a scan array. The detection zone array can include multiple scan arrays. These multiple scan arrays can correspond to the lidar's field of view or to a laser. For example, the detection zones in the first scan array receive echoes generated by objects in the first field of view, and the detection zones in the second scan array receive echoes generated by objects in the second field of view. As another example, the detection zones in the first scan array receive echoes generated by laser light emitted by a first laser being reflected by an object. The detection zones in the second scan array receive echoes generated by laser light emitted by a second laser being reflected by an object. The multiple scan arrays can be divided according to the lidar's scanning sequence of the spatial field of view, the size of the spatial field of view covered in a single scan, or the lidar's design parameters. For ease of understanding, Figure 5 The scan array shown is uniformly divided, but this representation is for illustrative purposes only. In reality, the detection areas in a scan array may be adjacent or non-adjacent. Different scan arrays can have different division methods. For example, a scan array may include one or more detection areas. The number of detection areas in different scan arrays may be the same or different.

[0066] In some embodiments, the lidar can perform time-division multiplexing of the entire field of view, with one detection corresponding to one scan. For example, the lidar detects the first field of view within a first time window and the second field of view within a second time window. This reduces optical crosstalk. Within the time window of a single detection by the lidar, at least one detection area in the scanning array is activated for echo detection. The detection area in the scanning array simultaneously receives the echo, generates an electrical signal, and transmits it to the preprocessing circuit via a readout circuit. When calibrating the echo data of the scanning array, calibration data corresponding to the scanning array is pre-fetched.

[0067] In some embodiments, with Figure 5 Consistent with this example, a scan array 510 can be divided into multiple subarrays 520. Similar to scan array 510, the division of subarrays 520 is merely for illustrative purposes; the detection areas of different subarrays may be adjacent or non-adjacent. The division methods of different subarrays can be the same or different. For example, a subarray may include one or more detection areas. The number of detection areas in different subarrays can be the same or different.

[0068] In some embodiments, index data may be associated with scan array 510. For example, the storage location of a subset corresponding to scan array 510 can be determined using index data K1. Prefetch circuit 420 can index the subset corresponding to scan array 510 from calibration data based on the index data.

[0069] In some embodiments, index data may be associated with subarray 520. For example, the index data may determine the storage location of a subset corresponding to subarray 520. Prefetch circuit 420 may index the subset corresponding to subarray 520 from calibration data based on the index data. In some embodiments, the number m of subsets may be the same as the number of subarrays 520 included in scan array 510, where m is a positive integer greater than 1. In some embodiments, the prefetch circuit may obtain m subsets at a time and provide the m subsets at once or in installments to a plurality of first processing circuits 430-1 to 430-P, where P is a positive integer greater than 1, representing the number of first processing circuits. Alternatively, the prefetch circuit may obtain m subsets in installments and provide the m subsets at once or in installments to the first processing circuits 430-1 to 430-P. The first processing circuits 430-1 to 430-P may use the m subsets to calibrate the echo data of scan array 510.

[0070] In some embodiments, the number of index data may include multiple data sets. In some embodiments, the number of index data sets is the same as the number of subarrays 520 included in the scan array 510, and one index data set is used to index a subset of one subarray 520. In some embodiments, the number of index data sets may differ from the number of subarrays 520 included in the scan array 510. For example, when the positions of the detection areas in the first and second subarrays included in the scan array 510 can be determined (e.g., when they are arranged adjacently), one index data set can be used to index a subset of the first and second subarrays. That is, one index data set can index all or a subset of the subarrays 520 in the scan array 510.

[0071] In some embodiments, the echo data of a scan array 510 can be stored in blocks across N storage regions. One storage region can correspond to one subarray 520. That is, the subarray 520 can be divided based on the number N of storage regions for storing echo data in blocks (or the number N of echo data blocks). For example, assuming N is 8, the scan array 510 can be divided into 8 subarrays 520.

[0072] In some embodiments of this disclosure, the echo data corresponding to the detection area 530 can be determined based on its own echo data, or it can be determined based on the echo data of multiple neighboring detection areas. For example, the detection area 530 can be determined based on the echo data of four neighboring detection areas, including itself. Doing so can improve the signal-to-noise ratio of the echo data.

[0073] For example, suppose the detection area array includes 640×320 detection areas, and the lidar can scan 3200 detection areas at a time. For instance, scan array 510 includes 40×80 detection areas. Subarray 520 includes 20×20 detection areas. The echo data from the 20×20 detection areas can be grouped into 10×10 detection area echo data by grouping 4 detection areas together. In this way, processing circuits 430-1 to 430-P can store only a subset corresponding to scan array 510, significantly reducing storage resource overhead compared to storing the complete calibration data corresponding to the entire detection area array.

[0074] Consistent with some embodiments of this disclosure, a lidar may include multiple sets of calibration data, which may be used for different types of data calibration or for different levels of data calibration.

[0075] and Figure 4Consistent, exemplarily, in some embodiments, the storage circuit 410 may also store calibration data D2 of the lidar. The prefetch circuit 420 may further: determine index data K2; index a subset dt2 in the calibration data D2 based on the index data K2; and transmit the subset dt2 to the first processing circuit 430. The first processing circuit 430 may also perform a second data calibration on the echo data based on the subset dt2. For ease of description, the calibration data D2 may be referred to as the second calibration data, the index data K2 as the second index data, and the subset dt2 as the second subset.

[0076] Data calibration based on subset dt1 can be referred to as the first data calibration, and data calibration based on subset dt2 can be referred to as the second data calibration. This embodiment does not limit the execution order between the first and second data calibrations; the first data calibration can be performed first, followed by the second data calibration. Alternatively, the second data calibration can be performed first, followed by the first data calibration. Or, the first and second data calibrations can be performed in parallel. This execution order can be set based on the type or level of the data calibration.

[0077] In some embodiments, the first calibration data and the second calibration data can be used for different types of data calibration, i.e., the types of the first data calibration and the second data calibration can be different. In some embodiments, the type of data calibration can include, but is not limited to, one or more of the following: distance calibration, echo intensity calibration (or reflectivity calibration), ambient light calibration, temperature calibration, channel delay calibration, and photon detection efficiency (PDE) calibration. The first data calibration and the second data calibration can be any two of these data calibrations.

[0078] The type of data calibration can be determined based on the calibration object. In some embodiments, the calibration object includes distance, and the type of data calibration can include distance calibration. The calibration parameters can include one or more parameters such as the echo peak value, area, pulse width, leading edge, or slope, and calibration of these parameters can be used to calibrate the distance. In some embodiments, the type of data calibration can include ambient light calibration, which reduces the influence of ambient light on echo data by calibrating echo parameters (e.g., the peak value or area of ​​the echo pulse). In some embodiments, the type of data calibration can include echo intensity calibration, which compensates for echo intensity by calibrating one or more parameters such as the detector's receiving power or the laser's emitting power.

[0079] In some embodiments, the first calibration data and the second calibration data can be used for data calibration at different levels, i.e., the levels of the first data calibration and the second data calibration are different. The types of the first data calibration and the second data calibration can be the same or different. For different levels, the range that a calibration quantity in the calibration data can calibrate is different. For example, the calibration range can include one or more of a detection area, a set of detection areas, or a group of detection areas. A detection area can correspond to one or more detectors, a set of detection areas can include one or more detection areas, and a group of detection areas can include multiple sets of detection areas. When the calibration range includes a detection area, a calibration quantity in the calibration data can be used to calibrate the echo data of one detection area. When the calibration range includes a set of detection areas, a calibration quantity in the calibration data can be used to calibrate the echo data of one set of detection areas, i.e., the echo data of detection areas in the same set of detection areas can be calibrated using the same calibration quantity. When the calibration range includes a group of detection areas, a calibration quantity in the calibration data can be used to calibrate the echo data of one group of detection areas, i.e., the echo data of detection areas in the same group of detection areas can use the same calibration data. Furthermore, the calibration range can include one or more of the following: pixel level, spot level, sector level, or system level. A light spot can correspond to one or more pixels, a region can correspond to one or more light spots, and a system can correspond to the entire field of view of the LiDAR. One pixel can correspond to one or more detection regions. When one pixel corresponds to one detection region, pixel-level calibration can include detection region-level calibration. When one pixel corresponds to multiple detection regions (e.g., four detection regions), pixel-level calibration can include detection region set-level calibration. When the calibration range includes the pixel level, a calibration value in the calibration data can be used to calibrate the echo data of one pixel. When the calibration range includes the light spot level, a calibration value in the calibration data can be used to calibrate the echo data of one light spot; that is, pixels in the same light spot can reuse the same calibration value. When the calibration range includes the region level, a calibration value in the calibration data can be used to calibrate the echo data of a region; that is, pixels in the same region can reuse the same calibration value. When the calibration range includes the system level, a calibration value in the calibration data can be used to calibrate the echo data of a region; that is, pixels in the detection region array can reuse the same calibration value. The same or different levels of calibration can be used for different types of calibration, and one or more levels of calibration can be used for the same type of calibration. For example, detector performance may vary across different detection zones; calibrating photon detection efficiency at the detection zone level can improve calibration accuracy. Furthermore, range or echo intensity calibration can be performed at the spot level, thus meeting the accuracy requirements of lidar range or echo intensity calibration with lower storage resources. Additionally, range calibration can include calibration of different parameters, which can be performed at different levels. For instance, the first parameter might be calibrated at the detection zone level, and the second parameter at the detection zone set level.This approach balances storage resource overhead with calibration accuracy. For example, echo intensity calibration and temperature calibration can employ different levels of calibration. Echo intensity calibration can be performed at the pixel level, while temperature calibration can be performed at the region or system level. This reduces the storage resource requirements for calibration. While the impact of temperature on detection may differ across different detection areas, this difference is relatively small, and under certain circumstances, the effect of temperature on different detection areas can be considered consistent. Different levels of calibration can adapt to different scenarios, achieving a more reasonable balance between storage resource consumption and the accuracy requirements of LiDAR calibration.

[0080] In some embodiments, the first data calibration can be used to calibrate a set of detector regions, and the second data calibration can be used to calibrate a group of detector regions. For example, the first calibration data D1 is used to calibrate the echo data of the set of detector regions, and the second calibration data D2 is used to calibrate the echo data of the group of detector regions. A group of detector regions may include a set of detector regions, and a set of detector regions may include detector regions.

[0081] In some embodiments, the subfield of view corresponding to the detection region set can be called the first subfield of view, and the subfield of view corresponding to the detection region group can be called the second subfield of view. For example, the first subfield of view is smaller than the second subfield of view.

[0082] For example, Figure 6 An example diagram is shown of the emitting area, the detection area, and the sub-field of view of a lidar consistent with some embodiments of this disclosure. Figure 6 The description uses a two-dimensional light-emitting region array and a two-dimensional detector region array as examples. However, in other embodiments of this disclosure, one-dimensional detector region arrays and one-dimensional light-emitting region arrays may also be included. Figure 6 Consistently, exemplarily, the squares in the emitting area array represent emitting areas, and one emitting area can correspond to one or more lasers, or one laser can correspond to multiple emitting areas. The squares in the detector area array represent light spots, which can be understood as the area that the echo reflected by an object can reach in the detector area array when the emitting area emits light and an object exists in the sub-field of view corresponding to that emitting area. One light spot can correspond to one or more pixels, and one pixel can correspond to one or more detector areas. For example, Figure 6 The light spot B2 shown corresponds to k*h detection zones, where k and h are positive integers greater than or equal to 1. Light spots corresponding to different light emission zones can correspond to the same or different numbers of detection zones. For example, the number of detection zones for light spots corresponding to light emission zones A1 and A2 can be the same or different.

[0083] The index data K1 and index data K2 used for the first subset dt1 and the second subset dt2 can be the same or different. For example, the first data calibration and the second data calibration can be used for calibration at the same level, and the index data K1 and index data K2 can be the same. For example, index data K1 and index data K2 can include location information (e.g., a detector area identifier) ​​of the detector area at the same location in the same subarray 520. As another example, the first data calibration and the second data calibration can be used for calibration at different levels, and the index data K1 and index data K2 can be different. For example, index data K1 can include a pixel identifier or a detector area set identifier. Index data K2 can include a spot identifier or a detector area group identifier.

[0084] This disclosure does not limit the type or level of the first data calibration and the second data calibration. For example, the first data calibration and the second data calibration may include any of the following: In the first case, the types of the first data calibration and the second data calibration may be the same, but the levels may be different. For example, the first calibration data and the second calibration data are used for distance calibration; the first calibration data is used for a first parameter of the distance calibration, and the second calibration data is used for a second parameter of the distance calibration; the first parameter uses a detector area level calibration, and the second parameter uses a detector area set level calibration. In the second case, the types of the first data calibration and the second data calibration may be the same, and the levels may be the same. For example, the first calibration data and the second calibration data are used for distance calibration; the first calibration data is used for a first parameter of the distance calibration, and the second calibration data is used for a second parameter of the distance calibration; both the first parameters use pixel-level calibration. In the third case, the types of the first data calibration and the second data calibration may be different, but the levels may be the same. For example, the first calibration data is used for distance calibration, and the second calibration data is used for echo intensity calibration; the first calibration data is used for a first parameter of the distance calibration, and the second calibration data is used for a third parameter of the echo intensity calibration; both the first and third parameters use pixel-level calibration. In the fourth scenario, the types and levels of the first and second data calibrations can differ. For example, the first calibration data is used for distance calibration, and the second calibration data is used for echo intensity calibration. The first calibration data is used for the first parameter of distance calibration, and the second calibration data is used for the third parameter of echo intensity calibration. The first parameter uses pixel-level calibration, and the third parameter uses spot-level calibration.

[0085] and Figure 4 In line with this, the device 400 can perform calibrations using different types or levels of data to match the calibration needs of different scenarios, while taking into account the calibration performance requirements of the laser with reasonable storage resource consumption.

[0086] In some embodiments, the apparatus 400 may further divide the processing resources (e.g., processing cores) into one or more processing resource clusters. These processing resource clusters are used to process echo data in parallel. Echo data processing may include preprocessing and post-processing. Preprocessing may be used to perform one or more processing operations on the echo data (also referred to as raw echo data), such as interpolation, filtering, or peak finding, before echo data calibration. Post-processing may be used to calibrate the preprocessed echo data. Preprocessing and post-processing of the echo data can be performed serially within a processing resource cluster. In some embodiments, multiple second processing circuits may be provided within a processing resource cluster for parallel preprocessing of the echo data.

[0087] For example, Figure 7 A structural example diagram of another apparatus for lidar data processing, consistent with some embodiments of this disclosure, is shown. Figure 7 Consistent, exemplarily, the device 700 includes a storage circuit 710, a prefetch circuit 720, and processing resource clusters 1-S. The processing resource clusters x include first processing circuits 730-x and a group of processing circuits 740, where x ∈ [1, S], and S represents the number of processing resource clusters, where S is a positive integer greater than 1. The group of processing circuits 740 may include multiple second processing circuits 740-1 to 740-Q, where Q represents the number of second processing circuits, where Q is a positive integer greater than 1. The multiple second processing circuits 740-1 to 740-Q can determine multiple echo data (e.g., pre-processed echo data) in parallel. The first processing circuit 730-x can perform data calibration on the multiple echo data based on a subset of calibration data. For example, the subset includes a first subset dt1, and the data calibration includes a first data calibration. Similarly, the subset includes a second subset dt2, and the data calibration includes a second data calibration. Figure 7 The example described uses multiple processing resource clusters. In other embodiments, the apparatus for processing lidar data may include a single processing resource cluster.

[0088] Consistent with some embodiments of this disclosure, the apparatus for processing lidar data can more rationally utilize processing resources by partitioning them, improving data efficiency while further reducing the storage resource requirements for calibration data. Before calibrating the echo data, multiple second processing circuits can be used to perform first processing on the echo data in parallel, improving the efficiency of the first processing of the echo data. Furthermore, a subset can be stored once within a processing resource cluster, reducing not only the amount of calibration data stored by prefetching a subset of calibration data but also the number of copies of the calibration data stored. This balances calibration data processing efficiency with further reduction of storage resource requirements for calibration data, further saving circuit area on the processing chip and thus further reducing the circuit cost of the lidar. It also further increases the usable area of ​​the logic circuit section of the processing chip, thereby further improving chip performance.

[0089] In some embodiments, the number of first processing circuits 730x is the same as the number of processing circuit groups. The above division of processing resource clusters is merely an example, and this disclosure does not limit the number of processing resource clusters or first processing circuits. Furthermore, this disclosure does not limit the number of second processing circuits in processing circuit groups; the number of second processing circuits in different processing circuit groups may be the same or different. In some embodiments of this disclosure, the number Q of second processing circuits can satisfy: 2 ≤ Q ≤ 20. This number setting can reduce the circuit area overhead of the processing device while meeting the requirements for echo data processing time.

[0090] In some embodiments of this disclosure, the apparatus for processing lidar data can further prefetch lidar configuration data into processing resource cluster x, thereby reducing the lidar's storage overhead for configuration data. Figure 7 Consistently, for example, device 700 may also include register 750 for storing configuration data of the lidar. Prefetch circuit 720 may also: determine index data K3 (which may be referred to as third index data for ease of distinction); and index a subset ct in the configuration data based on index data K3. Prefetch circuit 720 may also transmit the subset ct in the configuration data to processing resource cluster x, for example, to a first processing circuit 730-x, or at least one of second processing circuits 740-1 to 740-Q. When subset ct includes configuration data of the first processing circuit 730-x, subset ct is transmitted to the first processing circuit 730-x. When subset ct includes configuration data of the second processing circuit 740-y, subset ct is transmitted to the second processing circuit 740-y, where y∈[1,Q].

[0091] In some embodiments, configuration data can be configured at the region level. The prefetch circuit 820 can select a portion of the configuration data to be transmitted to the processing resource cluster x. The selected portion of the configuration data can correspond to the number N of storage regions (or the number N of echo data blocks) that are stored in blocks. For example, if the number N of echo data blocks is 8, and the register stores configuration data for 64 regions, the prefetch circuit 820 can select configuration data for 8 regions from the 64 regions and transmit it to the processing resource cluster x. In this way, by prefetching the configuration data, at least one processing circuit in the processing resource cluster x can be configured. After configuration, this processing circuit can process the currently stored echo data blocks, balancing data processing efficiency while further reducing the storage resource requirements of the LiDAR.

[0092] In some embodiments, the first processing circuit can calibrate the echo data of the reference detection area. During the detection process of the lidar, when the current laser emits a laser beam, the detection area (hereinafter referred to as the current detection area) corresponding to the same sub-field of view as that laser is activated to be in a state that can respond to light signals, thereby realizing the detection of objects within the sub-field of view (hereinafter referred to as the current sub-field of view) corresponding to that laser. In the case of highly reflective objects in the environment, when the highly reflective objects are not within the detection range of the current sub-field of view, the reflection of stray light by the highly reflective objects may cause the current detection area to receive light signals, thereby generating noise. When the highly reflective objects are within the detection range of the current sub-field of view, the strong echo generated by the highly reflective objects may saturate the detector in the detection area, and the echo data cannot accurately reflect the intensity of the echo. The light reflected by the highly reflective objects may not only enter the current detection area, but may also enter other adjacent detection areas. In addition, the detector also generates electrical signal output in response to ambient light, which will have a certain impact on the echo data of the lidar. Activating other detection areas as reference detection areas and using the echo data obtained from the reference detection areas (hereinafter referred to as reference data) to determine the validity of the echo data obtained from the current detection area can enable the judgment of valid echo data and background noise, reduce the generation of noise points, and suppress the influence of high crosstalk.

[0093] In some embodiments, the first calibration data may include a third subset, and the first processing circuit may further perform third data calibration on the echo data based on the third subset, i.e., it may calibrate the reference data. Improving the accuracy of the reference data can improve the accuracy of the determination of the validity of the echo data obtained from the current detection area. For example, the first calibration data D1 is used to calibrate the echo data of the detection area set, and the first calibration data D1 also includes a third subset. This third subset may include echo data from at least one adjacent detection area of ​​the above one or more detection areas.

[0094] The prefetch circuit 420 or 720 can determine index data K4 (which can be referred to as the fourth index data for ease of description) and index the third subset rt based on the index data K4. The prefetch circuit 420 or 720 can also transmit the third subset rt to the first processing circuit 430 or 730-x.

[0095] In some embodiments, the prefetch circuit can index one or more subsets of the first subset, second subset, third subset, or configuration data based on index data under the control of a trigger signal. Alternatively, the prefetch circuit can prefetch one or more subsets of the first subset, second subset, third subset, or configuration data based on index data, and transmit one or more subsets of the first subset, second subset, third subset, or configuration data to a first processing circuit or at least one second processing circuit under the control of a trigger signal.

[0096] and Figure 4 Consistent, exemplarily, the prefetch circuit 420 can also determine a first trigger signal T1; based on the first trigger signal T1 and the first index data K1, trigger the index subset dt1 or transmit the subset dt1 to the first processing circuit 430. Figure 4 Consistent, exemplarily, the prefetch circuit 720 may further determine a second trigger signal T2; based on the second trigger signal T2 and the second index data K2, trigger or transmit the index subset dt2 to the first processing circuit 730-x. In some embodiments, the prefetch circuit 720 may further determine a third trigger signal T3; based on the third trigger signal T3 and the third index data K3, trigger or transmit the index subset ct to the first processing circuit 730-x, or at least one of the second processing circuits 740-1 to 740-Q. In some embodiments, the prefetch circuit 720 may further determine a fourth trigger signal T4; based on the fourth trigger signal T4 and the fourth index data K4, trigger or transmit the index subset rt to the first processing circuit 730-x.

[0097] In some embodiments, one or more of the first trigger signal T1 to the fourth trigger signal T4 can be generated based on the signal indicating the start of lidar scanning, allowing the prefetch circuit to index or transmit data using the time interval between the start of lidar scanning and the start of data processing by the first or second processing circuit, thereby making full use of the data transmission time and improving the data processing efficiency of the lidar. The start of scanning may include, for example, a trigger signal indicating that the scanning array to be scanned is being scanned, or a trigger signal generated synchronously when scanning of the scanning array begins.

[0098] The prefetch circuit can generate a trigger signal directly or indirectly in response to the start signal of lidar scanning. When the prefetch circuit has multiple prefetch tasks, the embodiments of this disclosure do not limit the execution order of the multiple prefetch tasks. For example, they can be executed sequentially, with a trigger signal generated after one prefetch task is completed, triggering the execution of the next prefetch task. Alternatively, some or all of the multiple prefetch tasks can be executed in parallel.

[0099] Taking a prefetch task that includes prefetching a first subset dt1, a second subset dt2, a subset ct in the configuration data, and a third subset rt as an example, an example of the implementation process of the prefetch task of the prefetch circuit is described with reference to the accompanying drawings.

[0100] Figure 8 A structural example diagram of yet another apparatus 800 for lidar data processing, consistent with some embodiments of this disclosure, is shown. Figure 8 Consistent, exemplarily, the prefetch circuit 820 includes a first sub-circuit 821, a second sub-circuit 822, and a third sub-circuit 823. The first sub-circuit 821, in response to a trigger signal S1, prefetches a subset ct based on index data K3. The trigger signal S1 may include a signal indicating the start of a LiDAR scan, or a signal generated based on the start of a LiDAR scan. The index data K3 may include a sector ID. After completing the prefetch of subset ct, the first sub-circuit 821 may output a trigger signal S2 to the second sub-circuit 822. In response to the trigger signal S2, the second sub-circuit 822 prefetches a subset dt2 based on index data K2. The index data K2 may include a spot identifier or a detection area group identifier. After completing the prefetch of subset dt2, the second sub-circuit 822 may output a trigger signal S3 to the third sub-circuit 823. In response to the trigger signal S3, the third sub-circuit 823 prefetches a subset dt1 based on index data K1. The index data K1 may include a pixel identifier or a detection area group identifier. After completing the prefetch of subset dt1, the third subcircuit 823 can output a trigger signal S4. Responding to the trigger signal S4, the third subcircuit prefetches the third subset rt based on index data K4. Index data K4 may include an identifier of the reference probe area.

[0101] The prefetch order is merely an example, and this disclosure is not limited to it. Other prefetch orders can be used, such as prefetching subset dt1 first, then prefetching subset dt2, with subset rt placed before or after the prefetching of subset dt2. The prefetching of subset ct can be placed in any position. When placed first, the configuration of the processing circuit can be pre-implemented, reducing the probability of untimely configuration of the processing circuit. Furthermore, the prefetching order of subsets dt1, dt2, subset ct in the configuration data, and the third subset rt can be entirely or partially parallel, partially parallel, or partially serial. Moreover, this disclosure does not limit the calibration type or calibration level corresponding to subsets dt1, dt2, subset ct in the configuration data, and the third subset rt.

[0102] This disclosure does not limit the content or form of index data K1 to K4, as long as it can index the corresponding target information. For example, index data K1 to K4 may include the location information of the detection area, channel information, or the identification information of the corresponding indexed object. Index data K1 to K4 may be the same or different.

[0103] This disclosure does not limit the specific content of the calibration data; different calibration data may be used depending on the type of calibration. Calibration data can be obtained through testing and calibration before the lidar leaves the factory, or it can be obtained through software configuration during the use of the lidar.

[0104] and Figure 8 Consistent with this, exemplarily, in some embodiments, the apparatus 800 may further include a transmission circuit 860 for reordering or format conversion of the received calibration data (e.g., one or more of a first subset dt1, a second subset dt2, a subset ct in the configuration data, or a third subset rt, etc.) to meet the requirements of the transmission protocol. The apparatus 800 may further include a receiving circuit 870 for decoding the received subset and transmitting the decoded subset to a corresponding processing circuit, such as one or more of a first processing circuit 830 or at least a second processing circuit 840.

[0105] In some embodiments, the prefetch circuit and the storage circuit may be disposed in the interface circuit of a device for lidar data processing (e.g., device 800). This interface circuit may include a peripheral interface of the device for lidar data processing. The device for lidar data processing can communicate with other circuits (e.g., with memory 210 or with data processing circuit 151) through the interface circuit.

[0106] Based on the same or similar technical concept, embodiments of this disclosure also provide a method for processing lidar data. This method can be executed by a lidar processor (such as a prefetch circuit). The lidar (such as its memory) can store first calibration data of the lidar. Figure 9 A flowchart illustrating a method for processing lidar data, consistent with some embodiments of this disclosure, is shown. Figure 9 Consistent with this, exemplarily, the method includes at least the following steps.

[0107] In step S910, the processor determines first index data. The processor indexes a first subset of the first calibration data based on the first index data. The lidar stores the first calibration data.

[0108] In step S910, the processor performs a first data calibration based on the first subset.

[0109] For descriptions of the first index data, first calibration data, or first subset, please refer to the relevant sections in this disclosure. Figures 1 to 8 Related descriptions.

[0110] In some embodiments, the lidar (such as its memory) may also store second calibration data of the lidar. Figure 10 A flowchart illustrating a method for processing lidar data, consistent with some embodiments of this disclosure, is shown. Figure 10 Consistent, exemplarily, the method for processing lidar data may also include the following steps.

[0111] In step S930, the processor determines the second index data. The processor indexes a second subset of the second calibration data based on the second index data. The lidar stores the second calibration data.

[0112] In step S940, the processor performs a second data calibration based on the second subset.

[0113] In some embodiments, the first data calibration and the second data calibration are of different types or levels. The first index data and the second index data may be the same or different. Descriptions of the second calibration data, second subset, second index data, etc., can be found in the relevant sections of this disclosure. Figures 1 to 8 Related descriptions.

[0114] In some embodiments, the first calibration data can be used to calibrate the echo data of a set of detector regions. The second calibration data can be used to calibrate the echo data of a group of detector regions. A group of detector regions may include multiple sets of detector regions, and each set of detector regions may include at least one detector region. Descriptions of detector regions, sets of detector regions, or groups of detector regions can be found in this disclosure. Figures 1 to 8 Related descriptions.

[0115] In some embodiments, the above method for processing lidar data, performing a first data calibration based on a first subset, may include: determining echo data (e.g., data obtained after preprocessing the original echo data); and performing a first data calibration on the echo data based on the first subset. Descriptions regarding determining echo data and performing a first data calibration on the echo data can be found in the accompanying descriptions. Figure 4 ,or Figure 7 Related descriptions.

[0116] In some embodiments, the method for processing lidar data may further include: determining third index data; indexing a subset of configuration data based on the third index data, storing configuration data for lidar; and configuring lidar based on the subset of configuration data.

[0117] In some embodiments, the method for processing lidar data further includes: determining fourth index data; indexing a third subset of first calibration data based on the fourth index data; and performing third data calibration based on the third subset. Descriptions of data calibration, configuration data, third index data, or fourth index data can be found in this disclosure. Figures 1 to 8 Related descriptions.

[0118] In some embodiments, the method for processing lidar data further includes: determining a first trigger signal; triggering an index of a first subset or transmitting the first subset to a first processing circuit based on the first trigger signal and first index data. In some embodiments, the method for processing lidar data further includes: determining a second trigger signal; triggering an index of a second subset or transmitting the second subset to the first processing circuit based on the second trigger signal and second index data. In some embodiments, the above method for processing lidar data further includes: determining a third trigger signal; triggering a subset of index configuration data, or triggering the transmission of a subset of configuration data to at least one of the first processing circuit or multiple second processing circuits, based on the third trigger signal and third index data. In some embodiments, the method for processing lidar data further includes: determining a fourth trigger signal; triggering an index of a third subset or transmitting the third subset to the first processing circuit based on the fourth trigger signal and fourth index data. Descriptions of the first to fourth trigger signals, etc., can be found in the accompanying descriptions in this disclosure. Figure 4 , Figure 7 , Figure 8 Related descriptions. For example, one or more of the first to fourth trigger signals can be generated based on the signal that initiates a lidar scan.

[0119] This disclosure also provides a computer-readable storage medium including instructions stored thereon, which, when invoked by a processor, execute any of the methods described in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), a magnetic disk storage medium, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but is not limited thereto. The above-mentioned computer-readable storage medium may, for example, include a non-volatile computer storage medium. This disclosure also provides a non-volatile computer storage medium storing instructions that, when invoked by a processor, cause the processor to execute any of the methods described in the above embodiments.

[0120] This disclosure also provides a computer program product including instructions, wherein, when the instructions are executed by a processor, any of the methods of the above embodiments are performed.

[0121] In this disclosure, unless otherwise expressly specified and limited, ordinal numbers, such as "first," "second," etc., are used only to distinguish and describe related objects, and should not be construed as indicating or implying the relative importance or order between related objects. Furthermore, ordinal numbers do not represent the quantity of related objects. For example, "first calibration data" may include one calibration data point or multiple calibration data points.

[0122] "Multiple" includes two or more, and other classifiers are similar.

[0123] The terms "or" and "and / or" in this disclosure are used to describe relationships between related objects, indicating a non-exclusive inclusion. For example, "A and / or B" and "A or B" can both include: "A alone," "B alone," or "A and B," where "A" and "B" can include a single object or multiple objects. Similarly, "A, B and / or C," "A, B or C," and "A, B and C" can both include: "A alone," "B alone," "C alone," "A and B," "A and C," "B and C," or "A, B and C," where "A," "B," and "C" can include a single object or multiple objects. Additionally, the " / " in this disclosure is used to indicate an "or" relationship between related objects. The meanings of "at least one of A or B" and "one or more of A and B" in this disclosure are the same as the meaning of "A or B" above. The meanings of "one or more of A, B, and C" and "at least one of A, B, or C" are the same as the meaning of "A, B, or C" above. The meaning of "one or more of A, B, and C" is the same as the meaning of "A, B, or C" above.

[0124] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail or in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Furthermore, the above embodiments can be freely combined as needed.

Claims

1. An apparatus for processing lidar data, characterized in that, The device includes: A storage circuit is configured to store the first calibration data of the lidar; The prefetch circuit is configured as follows: Determine the first index data; The first subset of the first calibration data is indexed based on the first index data; Transmit the first subset to the first processing circuit; The first processing circuit is configured to perform a first data calibration on the echo data based on the first subset.

2. The apparatus according to claim 1, characterized in that, The prefetch circuit is further configured to: Determine the first trigger signal; Based on the first trigger signal and the first index data, trigger the indexing of the first subset or transmit the first subset to the first processing circuit.

3. The apparatus according to claim 2, characterized in that, The first trigger signal is generated based on the signal indicating the start of the lidar scan.

4. The apparatus according to any one of claims 1-3, characterized in that, The storage circuit is also configured to store the second calibration data of the lidar; The prefetch circuit is further configured to: Determine the second index data; The second subset of the second calibration data is indexed based on the second index data; Transmit the second subset to the first processing circuit; The first processing circuit is further configured to perform a second data calibration on the echo data based on the second subset, wherein the first data calibration and the second data calibration are of different types or levels, and the first index data and the second index data are different.

5. The apparatus according to claim 4, characterized in that, The first calibration data is used to calibrate the echo data of the detection area set, and the second calibration data is used to calibrate the echo data of the detection area group, the detection area group including the detection area set, and the detection area set including the detection area.

6. The apparatus according to claim 5, characterized in that, The detection area set corresponds to the first sub-field of view of the lidar, and the detection area group corresponds to the second sub-field of view of the lidar. The first sub-field of view is smaller than the second sub-field of view.

7. The apparatus according to any one of claims 4-6, characterized in that, The prefetch circuit is further configured to: Determine the second trigger signal; Based on the second trigger signal and the second index data, trigger the indexing of the second subset or transmit the second subset to the first processing circuit.

8. The apparatus according to any one of claims 1-7, characterized in that, The device further includes: The processing circuit group, including multiple second processing circuits, is configured to determine multiple echo data in parallel; The first processing circuit is configured to perform the first data calibration on the plurality of echo data based on the first subset.

9. The apparatus according to claim 8, characterized in that, The processing circuit group includes 2 to 20 second processing circuits.

10. The apparatus according to claim 8 or 9, characterized in that, The number of the first processing circuits is the same as the number of the processing circuit groups.

11. The apparatus according to any one of claims 8-10, characterized in that, Also includes: The register is configured to store the configuration data of the lidar; The prefetch circuit is further configured to: Determine the third index data; A subset of the configuration data is indexed based on the third index data; The subset is transmitted to at least one of the first processing circuit or the plurality of second processing circuits.

12. The apparatus according to any one of claims 1-11, characterized in that, The first calibration data also includes a third subset; The prefetch circuit is configured as follows: Determine the fourth index data; The third subset is indexed based on the fourth index data; The third subset is transmitted to the first processing circuit; The first processing circuit is further configured to perform a third data calibration on the echo data based on the third subset.

13. A method for processing lidar data, characterized in that, The method includes: Determine the first index data; The lidar stores the first calibration data based on a first subset of the first calibration data indexed by the first index data. Perform the first data calibration based on the first subset.

14. The method according to claim 13, characterized in that, Also includes: Determine the first trigger signal; Based on the first trigger signal and the first index data, trigger the indexing of the first subset or transmit the first subset.

15. The method according to claim 14, characterized in that, Also includes: The first trigger signal is generated based on the signal indicating the start of the lidar scan.

16. The method according to any one of claims 13-15, characterized in that, Also includes: Determine the second index data; The lidar stores the second calibration data based on the second subset of the second calibration data indexed by the second index data. A second data calibration is performed based on the second subset, wherein the first data calibration and the second data calibration are of different types or levels, and the first index data and the second index data are different.

17. The method according to claim 16, characterized in that, The first calibration data is used to calibrate the echo data of the detection area set, and the second calibration data is used to calibrate the echo data of the detection area group, wherein the detection area group includes a detection area set, and the detection area set includes detection areas.

18. The method according to claim 16 or 17, characterized in that, Also includes: Determine the second trigger signal; Based on the second trigger signal and the second index data, trigger the indexing of the second subset or transmit the second subset.

19. The method according to any one of claims 13-18, characterized in that, The first data calibration based on the first subset includes: Determine the echo data; The first data calibration is performed on the echo data based on the first subset.

20. The method according to any one of claims 13-19, characterized in that, Also includes: Determine the third index data; The lidar stores the configuration data based on a subset of the configuration data indexed by the third index data. Configure the lidar based on a subset of the configuration data.

21. The method according to any one of claims 13-20, characterized in that, Also includes: Determine the fourth index data; The third subset of the first calibration data is indexed based on the fourth index data; A third data calibration is performed based on the third subset.

22. A system for processing lidar data, characterized in that, The system includes: The memory, comprising multiple storage areas, is configured to store the echo data of the lidar in blocks; The apparatus according to any one of claims 1-12 is configured to perform the first data calibration on the echo data.

23. A lidar, characterized in that, include: A laser emitting circuit, configured to emit a laser; A laser receiving circuit is configured to receive the echo of the laser and convert the echo into an electrical signal, the electrical signal being used to generate echo data; A memory is configured to store the echo data; The apparatus according to any one of claims 1-12 is configured to perform a first data calibration on the echo data.

24. A vehicle, characterized in that, Including the lidar as described in claim 23.

Images