Spherical data processing method and device, electronic equipment and medium

Abstract

The invention discloses a spherical data processing method and apparatus, an electronic device and a medium. The method comprises the steps of obtaining original spherical data; rotating the original spherical data for multiple times to obtain multiple rotating spherical data; dividing the original spherical data and the spin spherical data into a plurality of sub-regions; projecting a plurality of sub-regions of the same spherical data to obtain a plane slice set; and performing data fusion on the sampling points on the same physical position among the different plane section sets to obtain a target section set. Therefore, multiple times of rotation, projection and sampling point fusion are carried out on the multiple original spherical data, so that the finally obtained target slice set can be directly applied to a deep learning framework, and the converted data maintain the physical characteristics and spatial correlation of the original spherical data; therefore, geometric distortion and edge errors generated when spherical data are converted into plane data are eliminated.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic device and medium for processing spherical data. Background Art

[0002] With the rapid development of environmental perception technology, vast amounts of spherical data are being generated across various technological fields. For example, in autonomous driving, on-board panoramic cameras and LiDAR devices generate spherical scanning data. In drone aerial photography, 360-degree panoramic cameras and spherical LiDAR can capture spherical imaging data. This spherical data, with its full perspective and high coverage, has significant application value in various fields.

[0003] However, current deep learning techniques are primarily designed for planar data in Euclidean space, making it difficult to directly process data with spherical geometry. To address this technical issue, spherical data can be directly expanded into planar data for use, but this approach introduces significant geometric distortion, resulting in data distortion. Alternatively, spherical data can be processed using the spherical geometric convolution operator, but this method is computationally complex and difficult to implement, significantly impacting data processing efficiency.

[0004] Therefore, how to accurately and efficiently process spherical data while maintaining its geometric characteristics so that the spherical data can be directly used in deep learning frameworks and applied to various technical fields is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, one aspect of the present application provides a method for processing spherical surface data, the method comprising: Get the original spherical data; Rotating the original spherical surface data multiple times to obtain multiple rotated spherical surface data; Dividing the original spherical surface data and the rotated spherical surface data into a plurality of sub-regions respectively; Projecting the multiple sub-regions of the same spherical data respectively to obtain a set of plane slices; The sampling points at the same physical position between the different plane slice sets are data-fused to obtain a target slice set.

[0006] Optionally, fusing data of sampling points at the same physical position between different plane slice sets includes: Splicing the plane slices corresponding to the original spherical data to obtain a first plane image; and splicing the plane slices corresponding to the rotated spherical data to obtain a second plane image; Based on the multiple rotations, reversely rotate the second planar image to obtain a third planar image corresponding to the first planar image; Parallel data mean calculation is performed on sampling points at the same physical position between the first planar map and the third planar map to obtain target sampling points; and the target sampling points are combined to obtain a target planar map to fuse the first planar map and the third planar map.

[0007] Optionally, dividing the original spherical data and the rotated spherical data into a plurality of sub-regions respectively includes: Determining the local feature density of the original spherical surface data; Determining a segmentation resolution according to the local feature density; wherein the local feature density is positively correlated with the segmentation resolution; determining a target number of segmentations of the slice according to the segmentation resolution; Based on the segmentation resolution and the target segmentation quantity, Healpix segmentation is performed on the original spherical surface data and the rotated spherical surface data to obtain the multiple sub-regions.

[0008] Optionally, projecting the multiple sub-regions of the same spherical data separately to obtain a set of plane slices includes: Taking the center of the sub-region as the projection center, and establishing a local coordinate system; The spherical data of the sub-region is mapped to a plane through tangent projection to obtain square plane slices with the same area to form the plane slice set.

[0009] Optionally, rotating the original spherical data multiple times to obtain multiple rotated spherical data includes: Get the specified number of rotations entered by the user; The original spherical surface data is controlled to be uniformly rotated for the specified number of rotations within a specified angle range; and the spherical surface data after each rotation is used as the rotated spherical surface data.

[0010] Optionally, the spherical data processing method further includes: splicing the plane slices in the target slice set to obtain a target plane image; Determining a quality parameter for evaluating the quality of the target planar image; wherein the quality parameter includes at least one of a correlation parameter, a continuity parameter, a geometric variation parameter, and a signal-to-noise ratio parameter; the correlation parameter is used to reflect the correlation between the original spherical data and the target planar image; the continuity parameter is used to reflect the continuity between adjacent slices in the target slice set; and the geometric variation parameter is used to reflect the degree of geometric variation between the original spherical data and the target planar image; Assigning a corresponding weight coefficient to each of the quality parameters; Performing a weighted summation on the quality parameter and the weight coefficient to obtain a target quality value; If the target quality value is greater than a threshold, the planar slices in the target slice set are converted into target format data.

[0011] Optionally, the method for processing spherical data includes: Get the initial spherical data from multiple data sources; Performing standardization processing on the initial spherical surface data; wherein the standardization processing includes unified coordinate system processing, unified resolution processing and value range standardization processing; The initial spherical surface data after the standardization process is filtered; and the filtered initial spherical surface data is used as the original spherical surface data.

[0012] Another aspect of the present application provides a spherical data processing device, the device comprising: An acquisition module is used to obtain original spherical surface data; A rotation module, configured to rotate the original spherical surface data multiple times to obtain multiple rotated spherical surface data; a division module, configured to divide the original spherical surface data and the rotated spherical surface data into a plurality of sub-regions respectively; A projection module, configured to project the multiple sub-regions of the same spherical data respectively to obtain a set of plane slices; The fusion module is used to fuse the data of the sampling points at the same physical position between the different plane slice sets to obtain the target slice set.

[0013] Another aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the spherical data processing method are implemented.

[0014] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the spherical data processing method when the program is executed by a processor.

[0015] The present application provides a method, device, electronic device, and medium for processing spherical data, which have the following beneficial effects: by performing multiple rotations, projections, and sampling point fusion on multiple original spherical data, the resulting target slice set can be directly applied to a deep learning framework. The multiple rotations and fusions can eliminate the geometric distortion and edge errors caused by converting spherical data to planar data. At the same time, the converted data maintains the physical properties and spatial correlation of the original spherical data, thereby ensuring that the deep learning framework can effectively learn spherical feature data, and can be widely applied in various technical fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a method for processing spherical data provided in an embodiment of the present application; Figure 2 A schematic diagram of the rotation of spherical data provided in an embodiment of the present application; Figure 3 A schematic diagram of spherical data segmentation provided in an embodiment of the present application; Figure 4 A flowchart of a method for processing spherical data provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a spherical data processing device provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0017] The accompanying drawings are marked as follows: 50 is an acquisition module, 51 is a rotation module, 52 is a division module, 53 is a projection module, 54 is a fusion module, 60 is a memory, 61 is a processor, 62 is a display screen, 63 is an input and output interface, 64 is a communication interface, 65 is a power supply, 66 is a communication bus, 601 is a computer program, 602 is an operating system, and 603 is data. DETAILED DESCRIPTION

[0018] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0020] Figure 1 A flow chart of a method for processing spherical data provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method includes: S10: Obtain original spherical surface data; In a specific embodiment, original spherical data that need to be processed is obtained. The original spherical data can be spherical scanning data generated by vehicle-mounted panoramic cameras, lidars, and millimeter-wave radars in the field of autonomous driving. It can also be spherical imaging data generated by 360-degree panoramic cameras and spherical lidars in the field of drone aerial photography. It can also be global spherical observation data generated by satellite remote sensing. This application does not limit the data source of the original spherical data.

[0021] It should be noted that the original spherical data can be understood as data sampled at sampling points in a spherical coordinate system, where the sampling points can be pixels in an image or point cloud data, that is, the original spherical data can be composed of data such as pixel points or point clouds, and this application does not limit this.

[0022] S11: rotating the original spherical surface data multiple times to obtain multiple rotated spherical surface data; It is understandable that the deep learning framework can only process planar data in Euclidean space. Therefore, the original spherical data needs to be converted into planar data. In this process, the original spherical data needs to be projected onto a plane. The projection process can actually be understood as the process of unfolding the sphere onto a plane. In this process, the geometric position undergoes changes such as stretching, compression, and rotation, and needs to be re-interpolated, which causes the sampling points, such as image pixel values, to change, that is, it may cause geometric distortion.

[0023] To solve the above technical problem, in an optional embodiment, the original spherical data is rotated to obtain multiple rotated spherical data. Specifically, in this embodiment, it can be understood that the spherical coordinate system includes a polar angle θ and an azimuth angle φ. When rotating the original spherical data, the polar angle θ can be fixed and the azimuth angle φ can be varied, or the azimuth angle φ can be fixed and the polar angle θ can be varied. Figure 2A schematic diagram of the rotation of spherical data provided in the embodiment of the present application is provided below for ease of understanding. Figure 2 Provide explanation.

[0024] For example, Figure 2 As shown, the azimuth angle φ of the spherical data shown in the upper figure is fixed, and the polar angle θ is rotated by 60°, thereby obtaining the rotated spherical data shown in the lower figure.

[0025] It should be noted that the multiple rotations can be performed with the azimuth angle φ unchanged and only the polar angle θ rotated, or with the polar angle θ unchanged and only the azimuth angle φ rotated. Of course, the polar angle θ and the azimuth angle φ can also be rotated the same number of times, which is not limited in this application. In addition, it should be noted that in order to minimize the geometric error caused by the projection, the same angle rotation can be performed each time, that is, the angle is uniformly rotated, and this application does not limit the number of rotations.

[0026] S12: Divide the original spherical surface data and the rotated spherical surface data into a plurality of sub-regions respectively; S13: Projecting multiple sub-regions of the same spherical data respectively to obtain a set of plane slices; To further reduce the geometric distortion caused by projection, in an optional embodiment, the spherical data can be segmented and then each segmented sub-region can be projected separately. Specifically, the original spherical data and all rotated spherical data are segmented separately, thereby dividing each spherical data into multiple sub-regions.

[0027] It should be noted that when segmenting spherical data, the number, shape, and area of ​​subregions obtained from each segmentation of the spherical data are the same. The specific shape of the subregions can be rectangular or other shapes, and this application does not impose any restrictions on this. In addition, this application does not impose any restrictions on the specific segmentation method for segmenting the spherical data or the number of subregions after segmentation.

[0028] After segmentation, each subregion is subjected to projection mapping, resulting in a corresponding planar image for each subregion. Projecting all subregions of the same spherical data creates a set of planar slices. When the original spherical data is rotated five times, the resulting set of planar slices is six. It is worth noting that the projection method can include, but is not limited to, tangent projection, conformal projection, and equidistant projection, and this application does not impose any restrictions on the projection method.

[0029] Figure 3 A schematic diagram of spherical data segmentation provided in an embodiment of the present application is shown in FIG. Figure 3As shown in FIG, in an optional embodiment, the spherical data is divided into 192 sub-regions, wherein the sub-regions are rectangular regions. Further, each sub-region is projected to obtain Figure 3 For each plane slice shown, it should be noted that Figure 3 The mid-plane section is a schematic diagram of a partial slice.

[0030] S14: performing data fusion on the sampling points at the same physical position between different plane slice sets to obtain a target slice set.

[0031] Furthermore, the slices of different plane slice sets are fused, and the target slice set thus obtained can overcome the error caused by projection. Specifically, the sampling points at the same physical position between different plane slice sets are fused, for example, Figure 2 As shown, the spherical data is image data, the sampling points are pixels, the sampling point corresponding to the original spherical data is point A, and the corresponding point of point A on the rotated spherical data is point B. Accordingly, data fusion is to fuse the pixels at point A with the pixels at point B. In an optional embodiment, data fusion can be to calculate the mean of the sampling point data.

[0032] In a specific embodiment, after the slices of multiple plane slice sets are fused, a target slice set with geometric errors eliminated can be finally obtained, and the target slice sets can be spliced ​​to obtain a plane target image of the original spherical data.

[0033] Finally, after obtaining the target slice set, in an optional embodiment, a mapping relationship is constructed between the spherical coordinates of the original spherical data and the plane coordinates of the slices in the target slice set, thereby obtaining a coordinate index array of each sampling point for subsequent data query, restoration and inverse mapping.

[0034] In an optional embodiment, after the original spherical data and the rotated spherical data are projected to obtain planar slices, in order to ensure the accuracy of data fusion, the sampling point data can be normalized and standardized in terms of pixel values, that is, the original pixel values ​​are compressed or translated to an interval that is more suitable for neural network processing.

[0035] In another optional embodiment, label information is generated for the slice samples in the target slice set, where the label information includes but is not limited to rotation parameters, celestial body type, coordinate range, data quality level, etc. In addition, to save storage resources, the generated target slice set can be stored after data compression.

[0036] To address memory limitations in large-scale data processing, an optional embodiment employs a batch processing strategy. Specifically, the total number of subregions is first calculated, and then the processing task is divided into multiple batches, with each batch processing a target number of patches (and subregions) in parallel. After each batch is processed, temporary data in memory is promptly cleared to avoid memory overflow. This ensures that even large-scale Tiantu data exceeding memory capacity can be processed while maintaining stable processing performance.

[0037] In an optional embodiment, the spherical data processing method provided by the present application can process time series data. Specifically, the spherical data at different time points are first time-aligned to ensure the consistency of the time series. For the data at each time point, standard spatial preprocessing is first performed, and then the time features are extracted. Among them, when extracting time features, the data of 2 time points before and after the current time point are considered to form a sliding window of a preset number (for example, 5) time points. Spatial features and time features are combined to generate comprehensive features containing spatiotemporal information, thereby supporting deep learning analysis of time-varying phenomena, which can be applied to scenarios such as astronomical time-varying celestial body observation, dynamic environment perception of autonomous driving, and abnormal behavior detection in security monitoring.

[0038] In a specific embodiment, to ensure data processing efficiency, in an optional embodiment, when the original sky map data (i.e., the original spherical data) is updated, only the changed areas are reprocessed, thereby improving processing efficiency. To facilitate real-time user viewing of the data processing process, in an optional embodiment, data visualization can be provided to display intermediate results during the processing and the final flattened data. In another optional embodiment, data processing can be performed separately for different data layers based on the frequency characteristics or physical quantity type of the sky map.

[0039] It should be noted that the spherical data processing method provided in this application can be processed in real time or at preset intervals. This application does not limit this and can be selected according to actual business needs.

[0040] Thus, the spherical data processing method provided in the embodiments of the present application, through multiple rotations, projections, and sampling point fusion of multiple original spherical data, allows the resulting target slice set to be directly applied to a deep learning framework. The multiple rotations and fusions eliminate the geometric distortion and edge errors caused by converting spherical data to planar data. At the same time, the converted data retains the physical properties and spatial correlation of the original spherical data, ensuring that the deep learning framework can effectively learn spherical feature data, which can be widely applied in various technical fields.

[0041] In an optional embodiment, data fusion is performed on sampling points at the same physical position between different plane slice sets, including: Splicing the plane slices corresponding to the original spherical data to obtain a first plane image; and splicing the plane slices corresponding to the rotated spherical data to obtain a second plane image; Based on the multiple rotations, the second plane image is reversely rotated to obtain a third plane image corresponding to the first plane image; Parallel data mean calculation is performed on sampling points at the same physical position between the first plane map and the third plane map to obtain target sampling points; and the target sampling points are combined to obtain a target plane map to fuse the first plane map and the third plane map.

[0042] Figure 4 A flow chart of a method for processing spherical data provided in an embodiment of the present application is provided. In a specific embodiment, as shown in FIG. Figure 4 As shown, the planes in the plane slice set corresponding to the original spherical data are spliced ​​together to obtain a first plane image, that is, the entire plane image of the original spherical data. At the same time, the planes in the plane slice set corresponding to the rotated spherical data are spliced ​​together to obtain a second plane image.

[0043] It is understandable that the rotated spherical data is obtained by rotating the original spherical data. Therefore, in a specific embodiment, in order to achieve data fusion of subsequent sampling points at the same position, the second plane image is reversely rotated based on the original rotation to obtain a third plane image. Figure 4 As shown, the first plan view corresponds to the third plan view, that is, they are similar images, with only a partial error caused by projection.

[0044] For example, when the original spherical surface data is image data, there may be a certain error in the pixel values ​​of sampling points at the same physical location between the first and third plane images. Therefore, in order to eliminate this error, the first and third plane images need to be fused to obtain a target plane image after the error is eliminated.

[0045] Specifically, the data mean is calculated between the samples at the same physical position on the first plane and the second plane, for example, the pixel mean is calculated, and the pixel mean is used as the pixel value of the target plane at the physical position, thereby obtaining the target plane after pixel fusion.

[0046] In an optional embodiment, in order to improve the efficiency of spherical data processing, the data fusion of sampling points at different positions can be performed through multi-threaded parallel fusion calculation.

[0047] In an optional embodiment, the original spherical surface data and the rotated spherical surface data are divided into a plurality of sub-areas, including: Determine the local feature density of the original spherical data; Determine the segmentation resolution based on the local feature density; the local feature density is positively correlated with the segmentation resolution; Determine the target number of slice segments based on the segmentation resolution; Based on the segmentation resolution and the target segmentation number, Healpix segmentation is performed on the original spherical data and the rotated spherical data to obtain multiple sub-regions.

[0048] In a specific embodiment, the present application does not limit the method of segmenting spherical data. However, in order to strictly ensure that the areas of each sub-region after segmentation are the same and achieve efficient segmentation, in an optional embodiment, the spherical data can be segmented using the Healpix segmentation strategy.

[0049] It is worth noting that in the embodiment of the present application, since the segmentation strategy is the Healpix segmentation strategy, the acquired raw spherical data can be Healpix data in FITS format. In fact, the present application does not limit the format of the input data, which can include but is not limited to FITS format, HDF5 format and NetCDF format.

[0050] It is understood that in Healpix segmentation, the Nside segmentation resolution significantly impacts the number of segmented subregions and subsequent projection accuracy, and the Nside segmentation resolution is closely related to the local feature density of the spherical data. Therefore, in an optional embodiment, the local feature density of the original spherical data is calculated before Healpix segmentation.

[0051] Furthermore, the current segmentation resolution is determined based on the local feature density, wherein the local feature density is positively correlated with the segmentation resolution, that is, when the local feature density is greater, the segmentation resolution is greater. Based on this, in an optional embodiment, when the local feature density is greater than the first density threshold, the spherical data can be segmented with high resolution, for example, the Nside segmentation resolution can be selected as 1024. When the local feature density is greater than the second density threshold and less than or equal to the first density threshold, a medium Nside segmentation resolution can be selected for segmentation, for example, 512 can be selected. When the local feature density is not greater than the second density threshold, a low resolution can be selected for segmentation, for example, 256 can be selected, thereby optimizing computational efficiency while ensuring processing quality.

[0052] It should be noted that in specific embodiments, the density threshold can be further increased based on actual business needs, and this application does not impose any restrictions on this. In addition, it should be noted that in addition to local feature density, Nside segmentation resolution is also closely related to noise level and edge complexity.

[0053] It is worth noting that, in a specific embodiment, when the Nside segmentation resolution is greater, the number of segmented sub-regions increases, and the final projection and data fusion result in a higher precision planar data. Therefore, in a specific embodiment, after obtaining the Nside segmentation resolution, the target number of segmentations for the slice can be determined based on the Nside segmentation resolution, that is, the number of sub-regions can be determined. In an optional embodiment, the target number of segmentations and the Nside segmentation resolution satisfy the following relationship: target number of segmentations = 12 * Nside².

[0054] In an optional embodiment, considering that high noise levels are often accompanied by higher edge complexity, which is related to the imaging device's beamform, when the noise level is lower and the imaging device structure is simpler, the Nside segmentation resolution can be set to a smaller value for spherical data segmentation, thereby reducing the number of subregions to be segmented and improving computational efficiency. When the noise level is higher and the edge complexity is greater, the Nside segmentation resolution can be set to a larger value for spherical data segmentation, resulting in a larger number of subregions, thereby ensuring the accuracy of the planar data.

[0055] In another optional embodiment, the value range of the Nside segmentation resolution is 64 to 2048, preferably 256 to 1024.

[0056] In a specific embodiment, after determining the Nside segmentation resolution and the target segmentation number, the original spherical data and the rotated spherical data are segmented by Healpix segmentation to obtain multiple sub-regions. Figure 3 As shown in the figure, when the Nside segmentation resolution is set to 4, the target number of segmentations = 12*Nside² = 12*16 = 192. Therefore, the spherical data can be divided into 192 sub-regions of equal area.

[0057] In an optional embodiment, the segmentation parameters of Healpix segmentation can be dynamically adjusted based on the characteristics of the spherical data, and by setting different Nside segmentation resolutions, multiple planar data sets of different resolutions can be generated simultaneously. Thus, the same original spherical data can be used to generate multi-resolution planar data sets, which can improve the accuracy of spherical data in various fields and provide accurate data support for deep learning frameworks in different fields. For example, based on multi-scale data, in autonomous driving, long-range and close-range targets can be detected simultaneously, and in security monitoring, panoramic views and local details can be recognized simultaneously.

[0058] In another optional embodiment, after determining the Nside segmentation resolution, the target number of segments is calculated. Simultaneously, a coordinate index array is generated. Specifically, the spherical coordinates (θ, φ) of each sampling point are obtained and converted to a Cartesian coordinate system (x, y, z) to prepare for the subsequent projection transformation.

[0059] Thus, the spherical data processing method provided in the embodiments of this application segments the spherical data using the Healpix segmentation strategy, ensuring that the segmented subregions have the same area and identical geometric features, facilitating subsequent projection and data statistics. Furthermore, the segmentation resolution parameters in Healpix segmentation can be dynamically adjusted, allowing for the simultaneous generation of multi-scale planarized data, improving the accuracy of spherical data processing and providing rich data support for practical applications.

[0060] In an optional embodiment, multiple sub-regions of the same spherical surface data are projected separately to obtain a set of plane slices, including: Take the center of the sub-region as the projection center and establish a local coordinate system; The spherical data of the sub-region is mapped to a plane through tangent projection to obtain square plane slices of the same area to form a plane slice set.

[0061] Based on the above embodiment, each sub-region is projected to convert the spherical data into planar data. Specifically, the center of the sub-region is used as the projection center, and a local coordinate system is established. Based on the local coordinate system, as an optional embodiment, the spherical data of the sub-region is mapped onto a plane through tangent projection, so that a square plane slice with the same area and the same geometric characteristics can be obtained, such as Figure 3 Partial planar slice shown.

[0062] It should be noted that, in a specific embodiment, the sub-region division can be rectangular, and the corresponding generated plane can also be a rectangle. However, in order to facilitate the convolution calculation in the subsequent deep learning framework, in an optional embodiment, the projection plane of the sub-region is a square with the same area.

[0063] Furthermore, it should be noted that, in this embodiment, the size of the projected positive plane slice is closely related to the Nside segmentation resolution. For example, when the Nside segmentation resolution is 512, the output image size after tangent projection is a 256×256 pixel square. Therefore, the Nside segmentation resolution is a key parameter in spherical data segmentation, defining the resolution of the sphere's divisions and determining the number of pixels on the sphere and the size of each pixel.

[0064] In an optional embodiment, the original spherical surface data is rotated multiple times to obtain multiple rotated spherical surface data, including: Get the specified number of rotations entered by the user; Control the original spherical data to rotate uniformly within the specified angle range for the specified number of rotations; and use the spherical data after each rotation as the rotated spherical data.

[0065] It is understandable that, in a specific embodiment, the target slice set finally obtained needs to be fused with the planar slices obtained by projecting the original spherical data and the planar slices of the rotated spherical data, so as to overcome the geometric errors caused by the projection process. Therefore, the amount of rotated spherical data, that is, the setting of the number of rotations and the rotation angle, is crucial to the accuracy of the final target slice.

[0066] In a specific embodiment, the number of rotations can be set by the user according to actual business needs. For high-precision application scenarios, a larger specified number of rotations can be selected. For application scenarios with high requirements for data processing efficiency, the specified number of rotations can be appropriately reduced to improve data processing efficiency.

[0067] After obtaining the specified number of rotations input by the user, in an optional embodiment, to further improve data processing accuracy, the multiple rotations can be evenly distributed within a specified angle range. The specified angle range can be an angle range of 0 to π or 0 to 2π, which is not limited in this application.

[0068] It can be understood that the spherical coordinate system includes polar angle θ and azimuth angle φ. Rotation refers to rotating the polar angle θ or the azimuth angle φ. In an optional embodiment, the specified angle range is 0 to 2π, so the original spherical data can be controlled to rotate uniformly within 0 to 2π for a specified number of rotations, and the spherical data after each rotation is used as a rotated spherical data. In this way, multiple sets of planarized data with different perspectives can be generated to improve data processing accuracy.

[0069] In an optional embodiment, during uniform rotation, the polar angle θ can be fixed, and the azimuth angle φ can be uniformly rotated a first number of times within 0 to 2π. In addition, the azimuth angle φ can be fixed, and the polar angle θ can be uniformly rotated a second number of times within 0 to 2π, wherein the sum of the first number of rotations and the second number of rotations is equal to the specified number of rotations.

[0070] For example, within the range of 0 to 2π, the polar angle θ is fixed and the azimuth angle φ is rotated by 60° each time, which can be rotated 6 times. Similarly, the azimuth angle φ is fixed and the polar angle θ is rotated by 60° each time, which can be rotated 6 times. Thus, the original spherical data is rotated a total of 12 times.

[0071] Therefore, the spherical data processing method provided in the embodiment of the present application solves the geometric errors caused by converting spherical data into plane data through the technology of multiple rotations and averaging of sampling point data, overcomes the distortion problem in the edge area, and improves the accuracy of target detection and recognition. It can be widely used in astronomical observation, autonomous driving, drone aerial photography, intelligent security and satellite remote sensing and other fields, and has good versatility.

[0072] In an optional embodiment, the method for processing spherical data provided by the present application further includes: Splicing the plane slices in the target slice set to obtain the target plane map; Determining a quality parameter for evaluating the quality of the target planar image; wherein the quality parameter includes at least one of a correlation parameter, a continuity parameter, a geometric variation parameter, and a signal-to-noise ratio parameter; the correlation parameter is used to reflect the correlation between the original spherical data and the target planar image; the continuity parameter is used to reflect the continuity between adjacent slices in the target slice set; and the geometric variation parameter is used to reflect the degree of geometric variation between the original spherical data and the target planar image; Assign corresponding weight coefficients to each quality parameter; Perform weighted summation on the quality parameters and weight coefficients to obtain the target quality value; If the target quality value is greater than the threshold, the planar slices in the target slice set are converted into target format data.

[0073] In a specific embodiment, in order to improve the accuracy of spherical data applications in various fields, for example, to improve the accuracy of environmental perception in the field of autonomous driving, based on the above embodiment, as an optional embodiment, the acquired planarized data is quality evaluated so that high-quality data can be directly used while low-quality data is discarded.

[0074] Specifically, the planar slices in the target slice set obtained in the above embodiment are spliced ​​together to obtain a target planar image. Further, based on the target planar image, a quality parameter of at least one of a correlation parameter, a continuity parameter, a geometric variation parameter, and a signal-to-noise ratio parameter is calculated.

[0075] In an optional embodiment, the target plane map and the original spherical data can be used as input data for the target large model, and the quality parameters can be directly evaluated through the target large model. Specifically, the correlation between the currently obtained target plane map and the original spherical data is evaluated, so that the correlation parameters can be obtained. At the same time, it is checked whether the continuity between adjacent plane slices in the target slice set meets the requirements, thereby determining whether the continuity of the entire target plane map meets expectations. In addition, by comparing the original spherical data and the target plane map, the degree of geometric change of the target plane map can also be determined, that is, it can be determined whether the change in the geometric features of the target plane map exceeds the acceptable range after the projection transformation. In addition, the signal-to-noise ratio parameter of the target plane map can also be calculated to measure the quality of the signal.

[0076] Furthermore, corresponding weight coefficients are assigned to different quality parameters, and a weighted sum is performed based on the quality parameters and weight coefficients to obtain a target quality value. When the target quality value is greater than a threshold, it can be determined that the target plan image quality meets expectations, and the plan slices in the target slice set can be converted into target format data for direct use in actual application scenarios.

[0077] Of course, if the target quality value is not greater than the threshold, it is determined that the quality of the target plane map does not meet expectations, the current original spherical data and target slice set are discarded, and a prompt signal is sent to the terminal so that the user can check it in time.

[0078] It should be noted that, in an optional embodiment, to meet the needs of different technical fields, the plane slice format in the output target slice set may include, but is not limited to, NumPy format, TensorFlow format, and PyTorch format. Thus, the output data can support the data interfaces of multiple deep learning frameworks.

[0079] In a specific embodiment, the spherical data processing method adopted by this application can automatically detect the data format and automatically identify the data format based on the file extension or file header information. For different input formats, corresponding parsers are used to load data. At the same time, for different output formats, corresponding data structures and metadata information can be generated according to the requirements of the deep learning framework.

[0080] In an optional embodiment, the method for processing spherical data provided in this application includes: Get the initial spherical data from multiple data sources; Performing standardization processing on the initial spherical surface data; wherein the standardization processing includes unified coordinate system processing, unified resolution processing and value range standardization processing; The normalized initial spherical surface data are filtered, and the filtered initial spherical surface data are used as the original spherical surface data.

[0081] In a specific embodiment, to further improve data processing accuracy, initial spherical surface data is obtained from multiple data sources and standardized. The multi-source initial spherical surface data may include, but is not limited to, autonomous vehicle-mounted panoramic camera data, LiDAR point cloud data, millimeter-wave radar data, drone-mounted 360-degree panoramic aerial imagery, drone-mounted LiDAR data, infrared thermal imaging data, intelligent security panoramic surveillance video, spherical surveillance camera data, 360-degree security surveillance data, satellite remote sensing global observation data, multispectral satellite imagery, and high-resolution Earth observation data.

[0082] It is understandable that in order to facilitate the subsequent parallel processing of multi-source data and thus improve data processing efficiency, the initial spherical data needs to be processed by standardizing the coordinate system, resolution, and numerical range. Among them, coordinate system unification converts different coordinate systems into a unified coordinate system, resolution unification resamples data of different resolutions to the same resolution, and numerical range standardization normalizes different types of data to the same numerical range. Furthermore, to ensure the quality of the original spherical data, the standardized initial spherical data needs to be filtered, where filtering can include but is not limited to filtering out low-quality data such as noisy data, and processing missing values ​​and outliers in the data.

[0083] In an optional embodiment, multi-source data registration is provided based on data normalization to ensure accurate spatial alignment of data from different sources. Furthermore, a weighted fusion strategy is employed, weighting different data sources based on their quality and reliability to generate a unified fused dataset.

[0084] For ease of understanding, the specific application of the spherical data processing method provided in this application in actual application scenarios will be exemplified below.

[0085] For the panoramic camera system on an autonomous vehicle, 360-degree panoramic image data is input. The processing parameters can be set to Nside segmentation resolution of 256, equidistant projection, and an output image size of 256×256 pixels. First, spherical mapping is performed on the panoramic image, converting the image captured by a fisheye or panoramic lens into a standard spherical coordinate system. Healpix segmentation is then performed to divide the panoramic field of view into multiple regions, each corresponding to a different direction and distance around the vehicle. Each region is then planarly projected to generate planar image data suitable for object detection and semantic segmentation.

[0086] The 3D point cloud data generated by the vehicle's LiDAR is mapped to a spherical coordinate system, with the LiDAR at its center. Furthermore, each point is mapped to its corresponding spherical position based on the distance and angle information in the point cloud. An adaptive segmentation strategy is then employed, using a higher resolution in areas with dense objects (such as the road ahead) and a lower resolution in areas with sparse objects (such as the sky). After multi-rotation processing and data fusion, a planarized dataset containing depth information is generated, supporting 3D object detection and path planning.

[0087] For the drone's 360-degree panoramic camera, the input spherical panoramic aerial imagery was set to 512 resolution for Nside segmentation, tangent projection was selected for projection, and the output image size was set to 512×512 pixels. Spherical coordinate mapping was performed on the panoramic aerial imagery to convert the drone's full field of view into a spherical data format. Specifically, adaptive Healpix segmentation was performed, adjusting segmentation parameters based on the density of ground objects. High-resolution segmentation was used in areas with dense buildings and roads, while standard resolution was used in areas of sky and water.

[0088] The drone is equipped with a visible light camera, an infrared camera, and a lidar, and performs multi-sensor data fusion processing. First, the data from different sensors is unified into the same spherical coordinate system, followed by temporal synchronization and spatial registration. Healpix segmentation and planar projection processing are performed on each sensor data. Finally, the multi-source data is fused at the pixel level to generate a comprehensive dataset containing visible light, infrared, and depth information, supporting target recognition and 3D reconstruction in complex environments.

[0089] For panoramic dome cameras used in security surveillance, the Nside segmentation resolution is set to 1024, the projection method to equidistant, and the output image size to 256×256 pixels. The panoramic video stream is received in real time, and each frame is mapped to spherical coordinates. Real-time Healpix segmentation is performed, adaptively adjusting segmentation parameters based on the complexity of the surveillance scene. High-resolution segmentation is used in areas with high human activity, while standard resolution is used in static background areas.

[0090] To meet the real-time requirements of security monitoring, a streaming processing architecture is employed to perform real-time analysis of surveillance video streams. Target detection and behavior analysis are performed concurrently within each segmented area, detecting the presence and movement of targets such as people and vehicles. Historical status information for each area is maintained, and time-series analysis is used to detect abnormal behavior, such as intrusion detection, aggregation detection, and object detection. When an anomaly is detected, the relevant area is automatically marked and an alarm is generated. The relevant flattened data is also saved for subsequent analysis.

[0091] For global observation data from remote sensing satellites, we input satellite remote sensing images covering the entire globe, set the Nside segmentation resolution to 2048, selected the equal-area projection method, and set the output image size to 512×512 pixels. We mapped the global remote sensing data to a standard spherical Earth coordinate system, accounting for the Earth's ellipsoidal shape and projection transformations. We also performed global-scale Healpix segmentation, dividing the Earth's surface into equal-area regions, ensuring that data from different latitudes is equally weighted.

[0092] In a specific embodiment, satellite remote sensing data typically contains multiple spectral bands, and multispectral data is fused. Data from different bands are first radiometrically calibrated and atmospherically corrected, then unified to the same spherical coordinate system. Healpix segmentation and planar projection are performed on each spectral band to generate multi-channel planar data. Finally, channel-level fusion is performed on the multispectral data to generate a dataset containing rich spectral information, supporting applications such as land use classification, vegetation monitoring, and disaster assessment.

[0093] In the above embodiments, the method for processing spherical data is described in detail. The present application also provides a corresponding embodiment of a device for processing spherical data.

[0094] Figure 5 A schematic diagram of the structure of a spherical data processing device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the device includes: An acquisition module 50 is used to acquire original spherical surface data; A rotation module 51 is used to rotate the original spherical surface data multiple times to obtain multiple rotated spherical surface data; a division module 52 for dividing the original spherical surface data and the rotated spherical surface data into a plurality of sub-regions; The projection module 53 is used to project multiple sub-regions of the same spherical data to obtain a set of plane slices; The fusion module 54 is used to fuse the data of the sampling points at the same physical position between different plane slice sets to obtain a target slice set.

[0095] In addition, the spherical data processing device provided in the embodiment of the present application further includes: A splicing module is used to splice the plane slices corresponding to the original spherical data to obtain a first plane image; and to splice the plane slices corresponding to the rotated spherical data to obtain a second plane image; a derotation module, configured to derotate the second planar graph based on the multiple rotations to obtain a third planar graph corresponding to the first planar graph; The target sampling point determination module is used to perform parallel data mean calculation on the sampling points at the same physical position between the first plane map and the third plane map to obtain target sampling points; and combine the target sampling points to obtain a target plane map to fuse the first plane map and the third plane map.

[0096] A local feature density determination module, used to determine the local feature density of the original spherical surface data; A segmentation resolution determination module is used to determine the segmentation resolution based on the local feature density; the local feature density is positively correlated with the segmentation resolution; A target segmentation number determination module is used to determine the target segmentation number of the slice according to the segmentation resolution; The Healpix segmentation module is used to perform Healpix segmentation on the original spherical data and the rotated spherical data based on the segmentation resolution and the target segmentation number to obtain multiple sub-regions.

[0097] The tangent projection module is used to establish a local coordinate system with the center of the sub-region as the projection center; through tangent projection, the spherical data of the sub-region is mapped to a plane to obtain square plane slices of the same area to form a plane slice set.

[0098] The rotation number acquisition module is used to obtain the specified rotation number input by the user; The control module is used to control the original spherical surface data to rotate uniformly for a specified number of rotations within a specified angle range; and use the spherical surface data after each rotation as the rotated spherical surface data.

[0099] The stitching module is also used to stitch the plane slices in the target slice set to obtain the target plane map; a quality parameter determination module, configured to determine quality parameters for evaluating the quality of the target planar image; wherein the quality parameters include at least one of a correlation parameter, a continuity parameter, a geometric variation parameter, and a signal-to-noise ratio parameter; the correlation parameter is configured to reflect the correlation between the original spherical data and the target planar image; the continuity parameter is configured to reflect the continuity between adjacent slices in the target slice set; and the geometric variation parameter is configured to reflect the degree of geometric variation between the original spherical data and the target planar image; A weight coefficient allocation module is used to allocate corresponding weight coefficients to each quality parameter; The weighted summation module is used to perform weighted summation on the quality parameters and weight coefficients to obtain the target quality value; The format conversion module is used to convert the plane slices in the target slice set into target format data if the target quality value is greater than a threshold.

[0100] The acquisition module is also used to obtain initial spherical data from multiple data sources; A standardization module is used to perform standardization processing on the initial spherical surface data; wherein the standardization processing includes unified coordinate system processing, unified resolution processing and value range standardization processing; The filtering module is used to filter the initial spherical surface data after the standardization process; and use the filtered initial spherical surface data as the original spherical surface data.

[0101] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the electronic device includes: a memory 60 for storing computer programs; The processor 61 is configured to implement the steps of the spherical data processing method mentioned in the above embodiment when executing a computer program.

[0102] The electronic device provided in this embodiment may include but is not limited to a laptop computer or a desktop computer.

[0103] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0104] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In this embodiment, the memory 60 is used to store at least the following computer program 601. After being loaded and executed by the processor 61, the computer program can implement the relevant steps of the spherical data processing method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, which may be stored in a temporary or permanent manner. The operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include, but is not limited to, relevant data involved in the spherical data processing method.

[0105] In some embodiments, the electronic device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0106] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure.

[0107] The electronic device provided in an embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the spherical data processing method in the above embodiment.

[0108] It should be noted that although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A method for processing spherical data, characterized in that: The method comprises: Get the original spherical data; Rotating the original spherical surface data multiple times to obtain multiple rotated spherical surface data; Dividing the original spherical surface data and the rotated spherical surface data into a plurality of sub-regions respectively; Projecting the multiple sub-regions of the same spherical data respectively to obtain a set of plane slices; The sampling points at the same physical position between the different plane slice sets are data-fused to obtain a target slice set.

2. The method for processing spherical surface data according to claim 1, wherein: The data fusion of sampling points at the same physical position between different plane section sets includes: Splicing the plane slices corresponding to the original spherical data to obtain a first plane image; and splicing the plane slices corresponding to the rotated spherical data to obtain a second plane image; Based on the multiple rotations, reversely rotate the second planar image to obtain a third planar image corresponding to the first planar image; Parallel data mean calculation is performed on sampling points at the same physical position between the first planar map and the third planar map to obtain target sampling points; and the target sampling points are combined to obtain a target planar map to fuse the first planar map and the third planar map.

3. The method for processing spherical data according to claim 1, wherein: The step of dividing the original spherical surface data and the rotated spherical surface data into a plurality of sub-areas comprises: Determining the local feature density of the original spherical surface data; Determining a segmentation resolution according to the local feature density; wherein the local feature density is positively correlated with the segmentation resolution; determining a target number of segmentations of the slice according to the segmentation resolution; Based on the segmentation resolution and the target segmentation quantity, Healpix segmentation is performed on the original spherical surface data and the rotated spherical surface data to obtain the multiple sub-regions.

4. The method for processing spherical data according to claim 1, wherein: The projecting the multiple sub-regions of the same spherical data to obtain a set of plane slices includes: Taking the center of the sub-region as the projection center, and establishing a local coordinate system; The spherical data of the sub-region is mapped to a plane through tangent projection to obtain square plane slices with the same area to form the plane slice set.

5. The method for processing spherical data according to claim 1, wherein: The step of rotating the original spherical surface data multiple times to obtain a plurality of rotated spherical surface data comprises: Get the specified number of rotations entered by the user; The original spherical surface data is controlled to be uniformly rotated for the specified number of rotations within a specified angle range; and the spherical surface data after each rotation is used as the rotated spherical surface data.

6. The method for processing spherical surface data according to claim 1, wherein: The method further comprises: splicing the plane slices in the target slice set to obtain a target plane image; Determining a quality parameter for evaluating the quality of the target planar image; wherein the quality parameter includes at least one of a correlation parameter, a continuity parameter, a geometric variation parameter, and a signal-to-noise ratio parameter; the correlation parameter is used to reflect the correlation between the original spherical data and the target planar image; the continuity parameter is used to reflect the continuity between adjacent slices in the target slice set; and the geometric variation parameter is used to reflect the degree of geometric variation between the original spherical data and the target planar image; Assigning a corresponding weight coefficient to each of the quality parameters; Performing a weighted summation on the quality parameter and the weight coefficient to obtain a target quality value; If the target quality value is greater than a threshold, the planar slices in the target slice set are converted into target format data.

7. The method for processing spherical data according to claim 6, wherein: The method comprises: Get the initial spherical data from multiple data sources; Performing standardization processing on the initial spherical surface data; wherein the standardization processing includes unified coordinate system processing, unified resolution processing and value range standardization processing; The initial spherical surface data after the standardization process is filtered; and the filtered initial spherical surface data is used as the original spherical surface data.

8. A spherical data processing device, characterized in that: The device comprises: An acquisition module is used to obtain original spherical surface data; A rotation module, configured to rotate the original spherical surface data multiple times to obtain multiple rotated spherical surface data; a division module, configured to divide the original spherical surface data and the rotated spherical surface data into a plurality of sub-regions respectively; A projection module, configured to project the multiple sub-regions of the same spherical data respectively to obtain a set of plane slices; The fusion module is used to fuse the data of the sampling points at the same physical position between the different plane slice sets to obtain the target slice set.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the computer program, the steps of the method for processing spherical data according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the spherical data processing method according to any one of claims 1 to 7 are implemented.

Images