Synthetic imaging method and device, electronic equipment and storage medium

Abstract

The invention discloses a synthetic imaging method and device, electronic equipment and a storage medium, and relates to the technical field of radar imaging. The synthetic imaging method comprises the following steps: detecting current environment information based on a preset number of sensors, and obtaining point cloud data of the current environment information; obtaining the point cloud data of the target point from the point cloud data of the current environment information; and performing space-time alignment based on the point cloud data of the target point to realize imaging. The problems that only instant imaging can be achieved and the use scene is limited in the prior art are solved, real-time large-range imaging is achieved, the scene imaging efficiency is improved, and the working range used by scene imaging is enlarged.

Description

Technical Field

[0001] This invention relates to the field of radar imaging technology, and in particular to a synthetic imaging method, apparatus, electronic device, and storage medium. Background Technology

[0002] Accurate environmental perception is a crucial component of intelligent systems. Currently, commonly used sensors for environmental perception include visual sensors, lidar, millimeter-wave radar, and ultrasonic radar. Millimeter-wave radar, as a commonly used sensor, is widely used in various intelligent systems due to its high detection accuracy, long detection range, strong anti-interference capability, and all-weather operation.

[0003] Radar imaging systems utilize high-performance millimeter-wave radar to clearly observe objects in the environment such as vehicles, curbs, and tunnels, enabling semantically rich observation of the surrounding environment.

[0004] Current millimeter-wave radar imaging systems mostly focus on a single sensor and primarily achieve instantaneous imaging, making it difficult to achieve omnidirectional and large-area imaging, thus limiting their application scenarios. Summary of the Invention

[0005] This invention provides a synthetic imaging method, apparatus, electronic device, and storage medium, which realizes real-time large-area imaging, improves the efficiency of scene imaging, and enhances the scalability of scene imaging.

[0006] In a first aspect, embodiments of the present invention provide a synthetic imaging method, comprising:

[0007] The current environmental information is detected by a preset number of sensors to obtain point cloud data of the current environmental information;

[0008] Obtain the point cloud data of the target point from the point cloud data of the current environmental information;

[0009] Spatiotemporal alignment is performed on point cloud data of target points to achieve imaging.

[0010] Secondly, embodiments of the present invention also provide a synthetic imaging apparatus, comprising:

[0011] The data acquisition module is used to detect the current environmental information based on a preset number of sensors and acquire point cloud data of the current environmental information;

[0012] The feature extraction module is used to obtain the point cloud data of the target point from the point cloud data of the current environment information;

[0013] The imaging module is used for spatiotemporal alignment of point cloud data based on target points to achieve imaging.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0015] At least one processor; and

[0016] A memory that is communicatively connected to at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the synthetic imaging method of any of the embodiments of the present invention.

[0018] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, which are used to cause a processor to execute and implement the synthetic imaging method of any one of the embodiments of the present invention.

[0019] According to the technical solution of the present invention, the current environmental information is detected by a preset number of sensors to obtain point cloud data of the current environmental information, the point cloud data of the target point is obtained from the point cloud data of the current environmental information, and spatiotemporal alignment is performed based on the point cloud data of the target point to achieve imaging. This solves the problem that the prior art can only achieve instantaneous imaging and has limited application scenarios, realizes real-time large-area imaging, improves the efficiency of scene imaging, and improves the scalability of scene imaging.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of a synthetic imaging method provided in an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a synthetic imaging device provided in an embodiment of the present invention;

[0024] Figure 3 A flowchart of another synthetic imaging method provided in an embodiment of the present invention;

[0025] Figure 4 A flowchart of another synthetic imaging method provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of a synthetic imaging device according to an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of the structure of an electronic device for implementing the synthetic imaging method of this invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] In one embodiment, Figure 1 This is a flowchart of a synthetic imaging method provided in an embodiment of the present invention. This embodiment is applicable to radar imaging of all-around and large-scale scenes. The method can be executed by a synthetic imaging device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0031] like Figure 1 As shown, the synthetic imaging method provided in this embodiment may include:

[0032] S110. Based on a preset number of sensors, detect the current environmental information and acquire point cloud data of the current environmental information.

[0033] In this embodiment of the invention, the preset quantity can be understood as the number of sensors pre-set according to the size of the detection environment and the requirements of the detection operation. Current environmental information can be understood as the information of the surrounding environment that needs to be detected at the current moment, including information about objects such as vehicles, curbs, and tunnels in the surrounding environment. Point cloud data refers to representing the surrounding environmental information as a set of vectors in a three-dimensional coordinate system. Environmental information can be recorded in the form of points, and point cloud data can intuitively represent information such as the shape, surface, and texture of objects in the current environmental information.

[0034] Specifically, a fixed number of sensors, set according to actual detection requirements, can be used to detect the current environment, identify various objects in the surrounding environment, and acquire point cloud data of the current environment information returned by the detected objects. Sensors can include visual sensors, lidar, millimeter-wave radar, and ultrasonic radar. The acquired point cloud data can include point cloud data of both stationary and moving objects in the surrounding environment. For example, Figure 2 This is a schematic diagram of a synthetic imaging device provided in an embodiment of the present invention, in which millimeter-wave radar can be placed around the device, such as... Figure 2 It can detect point cloud information of objects such as vehicles, trees and roadblocks in the surrounding environment of the operating vehicle based on 6 millimeter-wave radars.

[0035] S120. Obtain the point cloud data of the target point from the point cloud data of the current environmental information.

[0036] In this embodiment of the invention, the target point can be understood as the object that needs to be imaged in the current environmental information. The selection of the target point can be based on the operational requirements of detection and imaging, and can be obtained by filtering according to the characteristics of each object in the current environmental information.

[0037] Specifically, objects can be filtered from the point cloud data of the current environment information obtained above to obtain the objects that need to be imaged and take them as target points. The point cloud data of the target points can then be processed and imaged.

[0038] S130. Spatiotemporal alignment is performed based on the point cloud data of the target point to achieve imaging.

[0039] In this embodiment of the invention, spatiotemporal alignment refers to merging point cloud data acquired in different time periods together, and can ensure that each set of point cloud data is consistent in both time and space dimensions.

[0040] Specifically, the point cloud data of the target points obtained above can be merged in time and space. The pose information of the target points at different times can be combined to align the pose changes of the target points at different times caused by the movement of the detection platform itself. This can achieve large-scale imaging of the environment around the detection platform. The imaging results can include roads and obstacles in the current environment.

[0041] According to the technical solution of the present invention, the current environmental information is detected by a preset number of sensors to obtain point cloud data of the current environmental information, the point cloud data of the target point is obtained from the point cloud data of the current environmental information, and spatiotemporal alignment is performed based on the point cloud data of the target point to achieve imaging. This solves the problem that the prior art can only achieve instantaneous imaging and has limited application scenarios, realizes real-time large-area imaging, improves the efficiency of scene imaging, and improves the scalability of scene imaging.

[0042] Based on the above embodiments, the synthetic imaging method further includes:

[0043] The imaging results of the current environmental information are segmented and stored in the form of tile data, while the current environmental information is stored as a historical imaging area.

[0044] When passing through a historical imaging area, the point cloud data is updated by combining the current environmental information with the historical imaging area.

[0045] In this embodiment of the invention, tile data refers to a widely used map data format that can divide the entire imaging result into multiple tiles, each of which is an independent image. The segmented imaging result is easy to store and load for subsequent use. The historical imaging area can be understood as the surrounding environment area that has been detected by the sensor and imaged. After the current environmental information at a certain moment is detected and the imaging result is obtained, this area can be used as the historical imaging area.

[0046] Specifically, the imaging results of the current environmental information can be segmented into tiles of a certain size and format using tile data. This tile-based storage reduces storage space and improves the efficiency of subsequent image loading. Simultaneously, the current environmental information that has already been detected and imaged can be stored as historical imaging areas for future updates in response to environmental changes. When the sensor revisits a historical imaging area, it can update the area using the point cloud data of the current environmental information obtained during this detection. This allows for more accurate perception of changes in environmental information and timely updates to the imaging results.

[0047] Based on the above embodiments, the synthetic imaging method further includes:

[0048] Select the master sensor from a preset number of sensors and use the master sensor's timestamp as the base timestamp.

[0049] In this embodiment of the invention, a timestamp refers to a character sequence that can uniquely identify a specific moment in time. A reference timestamp can be understood as a standard timestamp used for time alignment of the target point's point cloud data, allowing the timestamp of the target point's point cloud data to be aligned with this reference timestamp.

[0050] Specifically, a suitable sensor can be selected as the main sensor from a certain number of sensors set according to the imaging operation requirements. The timestamp of the main sensor can be obtained and used as the reference timestamp to provide a timestamp standard when performing time alignment on the point cloud data of the target point in the future.

[0051] Based on the above embodiments, the synthetic imaging method further includes:

[0052] Based on point cloud data with current environmental information, online angle calibration is performed and the calibration results are updated when preset conditions for changes in current environmental information are met.

[0053] In this embodiment of the invention, the preset conditions can be understood as conditions set in advance according to specific changes in the environment that require updating the results. For example, the preset conditions may include obstacles such as clear guardrails on both sides, and the vehicle traveling straight along the guardrail.

[0054] Specifically, based on the obtained point cloud data of the current environment, it can be determined whether the changed information in the current environment meets the preset conditions. When the preset conditions for the change of current environment information are met, the radar extrinsic parameters can be calibrated online, and the calibration results can be updated. For example, the calibrated radar extrinsic parameters can be yaw angle information.

[0055] In one embodiment, Figure 3 This is a flowchart of another synthetic imaging method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment further optimizes and expands the process of obtaining point cloud data of target points from point cloud data of current environmental information and performing spatiotemporal alignment based on point cloud data of target points to achieve imaging.

[0056] like Figure 3 As shown, another synthetic imaging method provided in this embodiment may include:

[0057] S210. Based on a preset number of sensors, detect the current environmental information and acquire point cloud data of the current environmental information.

[0058] S220. Based on a preset feature threshold, filter the final imaging object from the point cloud data of the current environment information to determine the initial information of the target point.

[0059] In this embodiment of the invention, the preset feature threshold can be understood as a critical value for the attribute features of each object in the current environmental information, set in advance according to the imaging operation requirements. When the attribute features of an object meet the critical value, it can be determined that the object needs to be imaged. The initial information of the target point can be understood as the overall point cloud data of the target point obtained by the sensor. The initial information of the target point may include the overall shape information, surface feature information, and spatial feature information of the target point.

[0060] Specifically, objects that meet a preset feature threshold can be selected from the point cloud data of the current environment as the final imaging objects, and these objects can be used as target points. The features of the preset feature threshold can include the object's speed and size, etc. For example, if a moving vehicle in the current environment reaches a preset speed of 80 mph, it can be determined that the moving vehicle needs to be imaged. Once the target point for the final imaging is determined, the initial information of the target point can be determined.

[0061] S230. Extract the point cloud position, velocity, geometry and radar cross section distribution information of the target point from the initial information of the target point.

[0062] In this embodiment of the invention, radar cross section (RCS) distribution information refers to a physical quantity used to describe the echo intensity of a target point.

[0063] Specifically, point cloud features of the target points can be extracted from the initial information obtained above. These features can include the target point's position, velocity, geometry, and RCS distribution. Extracting these point cloud features allows for a more accurate description of the target point's position and shape, facilitating subsequent imaging.

[0064] S240. Use a filtering algorithm to filter the initial information of the target point and filter out noise and other non-target point cloud data.

[0065] In this embodiment of the invention, the filtering algorithm can be understood as an algorithm that preprocesses the initial information of the target point and leaves only the point cloud data of the target point. For example, the filtering algorithm may include median filtering and mean filtering, etc. Non-target point cloud data can be understood as other point cloud data besides the target point cloud data. For example, non-target point cloud data may include various types of noise, low-lying points, and high-altitude points, etc.

[0066] Specifically, filtering algorithms can be used to filter the initial information of target points. This can filter out noise and other non-target point cloud data, highlighting the target point's point cloud data and eliminating the influence of other data on the target point, thus facilitating subsequent processing of the target point's point cloud data. For example, for non-target point cloud data with radar noise, low-reflection-intensity noise can be filtered out using RCS; ghost noise can be filtered out using point cloud shape and density distribution; and for non-target point cloud data of low-lying and high-altitude points, filtering can be performed based on point cloud height.

[0067] S250. The position parameters of the target point are calibrated online based on the point cloud data of the target point.

[0068] In this embodiment of the invention, the position parameter of the target point refers to the position information of the target point in the current environmental information. For example, the position parameter of the target point may include the position of the target point in the sensor coordinate system, the angle of the target point relative to the detection platform, and other information.

[0069] Specifically, the location parameters of the target point can be calibrated online based on the location information in the target point cloud data, and the position and angle of the target point can be accurately detected.

[0070] S260. Combining calibration parameters, the coordinate system of the point cloud data of the target point is converted into the coordinate system of the platform itself for spatial alignment.

[0071] In this embodiment of the invention, the self-platform can be understood as a measurement platform equipped with sensors to measure current environmental information, and the self-platform can be different types of work vehicles.

[0072] Specifically, by combining the target point's location calibration parameters in the current environmental information, the sensor coordinate system where the target point is located can be converted into the platform's own coordinate system, and the target point's position can be converted into the space where the platform is located, thus completing the spatial alignment between the target point and the platform.

[0073] S270. Align the timestamp of the point cloud data of the target point with the base timestamp, and perform motion compensation and time alignment for moving targets in the target point.

[0074] Specifically, the timestamps of the point cloud data of the target points acquired by sensors other than the main sensor can be aligned with the reference timestamp of the main sensor, thus aligning the point cloud data of the target points in time. At the same time, for target points in motion, motion compensation can be performed to compensate for the differences between adjacent frames of the target point caused by motion, making the point cloud data of the target point smoother, and enabling time alignment of target points in motion.

[0075] According to the technical solution of the present invention, the current environmental information is detected by a preset number of sensors to obtain point cloud data of the current environmental information. Based on a preset feature threshold, the final imaging object is filtered from the point cloud data of the current environmental information to determine the initial information of the target point. The point cloud position, velocity, geometry, and RCS distribution information of the target point are extracted from the initial information of the target point. A filtering algorithm is used to filter the initial information of the target point, filtering out noise and other non-target point cloud data. The position parameters of the target point are calibrated online based on the point cloud data of the target point. Combined with the calibration parameters, the coordinate system of the point cloud data of the target point is converted to its own platform coordinate system for spatial alignment. The timestamp of the point cloud data of the target point is aligned with the reference timestamp. Simultaneously, motion compensation and time alignment are performed for moving targets within the target point. This solves the problem that existing technologies can only achieve instantaneous imaging and have limited application scenarios. It achieves real-time large-area imaging, effectively combines historical detection data with updated data, better judges surrounding environmental information, and effectively perceives changes in surrounding environmental information, improving the efficiency and scalability of scene imaging.

[0076] Based on the above embodiments, after performing spatiotemporal alignment based on the point cloud information of the target point, the method further includes:

[0077] By fusing point cloud data of target points at different times into the same spatiotemporal scale in a probabilistic manner, time series data fusion and splicing are performed.

[0078] The point cloud data of the fused target points is subjected to dimensionality reduction processing, and semantic updates are performed on each pixel to generate imaging results.

[0079] In this embodiment of the invention, time series data fusion and stitching can be understood as comprehensively processing point clouds from different times that reflect the same target point using data processing methods to obtain more reliable results that better reflect the essence of the target point. By complementing each other between point cloud data from different times, errors in the processing process are reduced.

[0080] Specifically, after spatiotemporal alignment based on the point cloud information of target points, point cloud data of target points at different times can be probabilistically fused into the same spatiotemporal scale. This time-series data fusion and stitching of target point point cloud data yields more effective overall target point point cloud data, enabling holistic identification of target points. For example, probabilistic fusion can assign different weights to different point cloud data, with higher weights for the most recent data and lower weights for historical data, as well as lower weights for the field of view (Fov) boundary. Dimensionality reduction processing can be performed on the fused target point point cloud data, mapping it to a two-dimensional space, and semantic updates can be performed on each pixel to generate imaging results.

[0081] In one embodiment, Figure 4 This is a flowchart of another synthetic imaging method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment is a preferred embodiment, and the process of synthetic imaging using millimeter-wave radar is specifically described.

[0082] like Figure 4 As shown, another synthetic imaging method provided in this embodiment may include:

[0083] S310. Obtain the point cloud features of the target points and perform preprocessing.

[0084] Specifically, it can acquire point cloud feature information of target points, which may include the target point's point cloud position, velocity, geometry, and RCS distribution information. It can also preprocess the target point point cloud data and filter out point noise and other non-target point cloud information.

[0085] In one embodiment, objects in the current environmental information are filtered based on a preset feature threshold to determine the final imaged object.

[0086] Specifically, objects that meet the preset feature threshold in the point cloud data of the current environment information can be selected as the final imaging objects, and these objects can be used as target points.

[0087] S320. Perform high-dimensional spatiotemporal alignment on the target point cloud data.

[0088] Specifically, by combining the pose information of the target point obtained at different times and from different sensors, the inter-frame pose changes of the target point caused by the motion of the platform itself can be aligned to achieve high-level spatiotemporal alignment.

[0089] In one embodiment, S320 includes:

[0090] S3201. Online calibration is performed using target point cloud data.

[0091] Specifically, the location parameters of the target point can be calibrated online based on the location information in the target point cloud data, and the position and angle of the target point can be accurately detected.

[0092] In one embodiment, the target point location parameters are calibrated online in real time based on the sensed current environmental information and under preset conditions.

[0093] Specifically, it can sense the current environment, obtain current environmental information, and, based on the sensed current environmental information, perform real-time online calibration of target point location parameters when changes in the current environmental information meet preset conditions.

[0094] S3202. Based on the calibration parameters, transfer the point cloud data of each sensor from the sensor coordinate system to its own platform coordinate system for spatial alignment.

[0095] Specifically, by combining the target point's location calibration parameters in the current environmental information, the sensor coordinate system where the target point is located can be converted into the platform's own coordinate system, and the target point's position can be converted into the space where the platform is located, thus completing the spatial alignment between the target point and the platform.

[0096] S3203: Combine the timestamps of each sensor, align the data of each sensor to the reference timestamp, perform motion compensation and time alignment for the moving target point.

[0097] Specifically, the timestamps of the point cloud data of the target points acquired by sensors other than the main sensor can be aligned with the reference timestamp of the main sensor, thus aligning the point cloud data of the target points in time. At the same time, for target points in motion, motion compensation can be performed to compensate for the differences between adjacent frames of the target point caused by motion, making the point cloud data of the target point smoother, and enabling time alignment of target points in motion.

[0098] In one embodiment, a master sensor is selected from a preset number of sensors, and the master sensor timestamp is used as a reference timestamp.

[0099] Specifically, a suitable sensor can be selected as the main sensor from a certain number of sensors set according to the imaging operation requirements, the timestamp of the main sensor can be obtained, and the timestamp of the main sensor can be used as the reference timestamp.

[0100] S330, Perform time series data fusion of target point cloud data.

[0101] Specifically, point cloud data of target points at different times can be fused into the same spatiotemporal scale in a probabilistic manner, and multi-frame data of target point point cloud data can be fused and stitched together to obtain more effective overall target point point cloud data, which can be used to make overall judgments on target points.

[0102] S340. Perform dimensionality reduction processing on the target point cloud data to generate imaging results.

[0103] Specifically, the point cloud data of the fused target points can be dimensionality reduced, mapping the fused target point cloud data to a two-dimensional space, and semantic updates can be performed on each pixel to generate imaging results.

[0104] S350, historical image fusion.

[0105] Specifically, when passing through historically surveyed areas, historical data can be updated based on weights and current environmental information. The latest imaging results are then used to update the corresponding areas in the previously generated imaging results, achieving effective perception of environmental changes while preserving historical information. For example, if there is an obstacle, such as a trash can, in an area, and the obstacle is removed when passing through, the obstacle would remain in the imaging results without real-time updates. However, with dynamic updates, the imaging results will no longer include information about the obstacle.

[0106] S360 stores the imaging results in tile format.

[0107] Specifically, the imaging results of the current environmental information can be segmented into tiles of a certain size and format using tile data, which can be stored in the form of tiles to reduce storage space and improve the efficiency of subsequent loading of imaging results.

[0108] According to the technical solution of the present invention, by acquiring the point cloud features of the target point and performing preprocessing, performing high-dimensional spatiotemporal alignment on the point cloud data of the target point, performing time-series data fusion on the point cloud data of the target point, performing dimensionality reduction processing on the point cloud data of the target point, generating imaging results, fusing historical images, and storing the imaging results in tile data, the problem of the existing technology being able to achieve only instantaneous imaging and having limited application scenarios is solved. Real-time large-area imaging is achieved, and the effective combination of historical detection data and updated data is realized, which better judges the surrounding environmental information and effectively perceives changes in the surrounding environmental information, improves the efficiency of scene imaging, and improves the scalability of scene imaging.

[0109] In one embodiment, Figure 5 This is a schematic diagram of a synthetic imaging device according to an embodiment of the present invention. This embodiment can perform the above-described implementation. This embodiment is applicable to radar imaging of omnidirectional and large-area scenes. The device can be implemented in hardware / software and can be configured in an electronic device.

[0110] like Figure 5 As shown, the synthetic imaging device provided in this embodiment includes: a data acquisition module 401, a feature extraction module 402, and an imaging module 403, wherein:

[0111] The data acquisition module 401 is used to detect the current environmental information based on a preset number of sensors and acquire point cloud data of the current environmental information.

[0112] Feature extraction module 402 is used to obtain point cloud data of target points from point cloud data of current environmental information;

[0113] Imaging module 403 is used for spatiotemporal alignment of point cloud data based on target points to achieve imaging.

[0114] According to the technical solution of the present invention, the current environmental information is detected by a preset number of sensors to obtain point cloud data of the current environmental information, the point cloud data of the target point is obtained from the point cloud data of the current environmental information, and spatiotemporal alignment is performed based on the point cloud data of the target point to achieve imaging. This solves the problem that the prior art can only achieve instantaneous imaging and has limited application scenarios, realizes real-time large-area imaging, improves the efficiency of scene imaging, and improves the scalability of scene imaging.

[0115] Based on the above embodiments, the feature extraction module 402 includes:

[0116] The initial information determination unit is used to filter the final imaging object from the point cloud data of the current environment information based on a preset feature threshold, and to determine the initial information of the target point.

[0117] The feature information extraction unit is used to extract the point cloud position, velocity, geometry and RCS distribution information of the target point from the initial information of the target point.

[0118] The filtering unit is used to filter the initial information of the target point using a filtering algorithm, filtering out noise and other non-target point cloud data.

[0119] Based on the above embodiments, the imaging module 403 includes:

[0120] The calibration unit is used to calibrate the position parameters of the target point online based on the point cloud data of the target point.

[0121] The spatial alignment unit is used to combine calibration parameters to convert the coordinate system of the point cloud data of the target point into its own platform coordinate system for spatial alignment.

[0122] The time alignment unit is used to align the timestamp of the point cloud data of the target point with the reference timestamp, and to perform motion compensation and time alignment for moving targets in the target point.

[0123] Based on the above embodiments, the synthetic imaging device further includes:

[0124] The data fusion module is used to fuse point cloud data of target points at different times into the same spatiotemporal scale in a probabilistic manner, thereby performing time series data fusion and splicing.

[0125] The dimensionality reduction module is used to reduce the dimensionality of the fused target point cloud data and perform semantic updates on each pixel to generate imaging results.

[0126] Based on the above embodiments, the synthetic imaging device further includes:

[0127] The data storage module is used to segment and store the imaging results of the current environment information in the form of tile data, and at the same time, store the current environment information as a historical imaging area.

[0128] The data update module is used to update the historical imaging area by combining the point cloud data with the current environmental information when passing through the historical imaging area.

[0129] Based on the above embodiments, the synthetic imaging device further includes:

[0130] The reference timestamp determination module is used to select a master sensor from a preset number of sensors and use the master sensor's timestamp as the reference timestamp.

[0131] Based on the above embodiments, the synthetic imaging device further includes:

[0132] The calibration result update module is used to perform online angle calibration and update the calibration results based on point cloud data with current environmental information when preset conditions for changes in current environmental information are met.

[0133] The synthetic imaging apparatus provided in this embodiment of the invention can execute any synthetic imaging method provided in this embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Content not described in detail in this embodiment can be referred to the description in any method embodiment of the invention.

[0134] In one embodiment, Figure 6 This is a schematic diagram of the structure of an electronic device for implementing the synthetic imaging method of an embodiment of the present invention. The electronic device 50, which can be used to implement embodiments of the present invention, is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0135] like Figure 6As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52 or a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The memory stores computer programs executable by the at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the ROM 52 or loaded from storage unit 58 into the RAM 53. The RAM 53 can also store various programs and data required for the operation of the electronic device 50. The processor 51, RAM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0136] Multiple components in electronic device 50 are connected to I / O interface 55, including: input unit 56, such as keyboard, mouse, etc.; output unit 57, such as various types of monitors, speakers, etc.; storage unit 58, such as disk, optical disk, etc.; and communication unit 59, such as network card, modem, wireless transceiver, etc. Communication unit 59 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0137] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as synthetic imaging methods.

[0138] In some embodiments, the synthetic imaging method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 50 via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the synthetic imaging method described above may be performed. Alternatively, in other embodiments, processor 51 may be configured to perform the synthetic imaging method by any other suitable means (e.g., by means of firmware).

[0139] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0143] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0144] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0145] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0146] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A synthetic imaging method, characterized in that, The method includes: The current environmental information is detected by a preset number of sensors to obtain point cloud data of the current environmental information; Obtain the point cloud data of the target point from the point cloud data of the current environmental information; Spatiotemporal alignment is performed based on the point cloud data of the target point to achieve imaging.

2. The method according to claim 1, characterized in that, The step of obtaining the point cloud data of the target point from the point cloud data of the current environment information includes: Based on a preset feature threshold, the final imaging object is selected from the point cloud data of the current environmental information to determine the initial information of the target point; Extract the point cloud position, velocity, geometry, and radar cross section distribution information of the target point from the initial information of the target point; A filtering algorithm is used to filter the initial information of the target point, filtering out noise and other non-target point cloud data.

3. The method according to claim 1, characterized in that, The process of performing spatiotemporal alignment on the point cloud data based on the target point to achieve imaging includes: The position parameters of the target point are calibrated online based on the point cloud data of the target point. By combining the calibration parameters, the coordinate system of the point cloud data of the target point is converted into the coordinate system of the platform itself for spatial alignment; The point cloud data timestamps of the target points are aligned with the base timestamps, and motion compensation and time alignment are performed for moving targets in the target points.

4. The method according to claim 3, characterized in that, After performing spatiotemporal alignment based on the point cloud information of the target point, the method further includes: By fusing point cloud data of the target points at different times into the same spatiotemporal scale in a probabilistic manner, time series data fusion and splicing are performed; The point cloud data of the fused target points is subjected to dimensionality reduction processing, and semantic updates are performed on each pixel to generate imaging results.

5. The method according to claim 1, characterized in that, The synthetic imaging method further includes: The imaging results of the current environmental information are segmented and stored in the form of tile data, and the current environmental information is also stored as a historical imaging area. When passing through the historical imaging area, the point cloud data combined with the current environmental information is used to update the historical imaging area.

6. The method according to claim 3, characterized in that, The synthetic imaging method further includes: A master sensor is selected from the preset number of sensors, and the timestamp of the master sensor is used as the reference timestamp.

7. The method according to claim 3, characterized in that, The synthetic imaging method further includes: Based on the point cloud data of the current environmental information, when the preset conditions for changes in the current environmental information are met, online angle calibration is performed and the calibration results are updated.

8. A synthetic imaging device, characterized in that, The device includes: The data acquisition module is used to detect the current environmental information based on a preset number of sensors and acquire point cloud data of the current environmental information; The feature extraction module is used to obtain the point cloud data of the target point from the point cloud data of the current environment information; The imaging module is used to perform spatiotemporal alignment based on the point cloud data of the target point to achieve imaging.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the synthetic imaging method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the synthetic imaging method according to any one of claims 1-7.

Images