A method and apparatus for processing spherical data, an electronic device, and a medium

Abstract

The application discloses a spherical data processing method and device, electronic equipment and medium, the method comprises the following steps: obtaining original spherical data; rotating the original spherical data multiple times to obtain multiple rotated spherical data; dividing the original spherical data and the rotated spherical data into multiple sub-regions respectively; projecting multiple sub-regions of the same spherical data respectively to obtain a set of plane slices; and fusing the sampling points at the same physical position between different sets of plane slices to obtain a target slice set. Thus, by rotating, projecting and fusing the sampling points of multiple original spherical data multiple times, the target slice set obtained finally can be directly applied to a deep learning framework, and the converted data retains the physical characteristics and spatial correlation of the original spherical data, thereby eliminating the geometric distortion and edge error generated when the spherical data is converted into plane data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, electronic device and medium for processing spherical data. Background Technology

[0002] With the rapid development of environmental perception technology, massive amounts of spherical data have been generated across various technological fields. For example, in the field of autonomous driving, devices such as vehicle-mounted panoramic cameras and LiDAR generate spherical scanning data. In the field of drone aerial photography, 360-degree panoramic cameras and spherical LiDAR can acquire spherical imaging data. Due to its full-view and high-coverage characteristics, this spherical data has significant application value in various fields.

[0003] However, current deep learning techniques are primarily designed for planar data in Euclidean space, making it difficult to directly process data with spherical geometry. To address this issue, spherical data can be directly unfolded into planar data, but this introduces significant geometric distortion, leading to data loss. Alternatively, spherical data can be processed using spherical geometric convolution operators, but this method has high computational complexity and is difficult to implement, severely impacting data processing efficiency.

[0004] Therefore, how to accurately and efficiently process spherical data while preserving its geometric properties, so that it can be directly used in deep learning frameworks and applied to various technical fields, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, one aspect of this application provides a method for processing spherical data, the method comprising:

[0006] Obtain the original spherical data;

[0007] The original spherical data is rotated multiple times to obtain multiple rotated spherical data.

[0008] The original spherical data and the rotated spherical data are each divided into multiple sub-regions;

[0009] The multiple sub-regions of the same spherical data are projected separately to obtain a set of planar slices;

[0010] Data fusion is performed on sampling points at the same physical location among different sets of planar sections to obtain a target slice set.

[0011] Optionally, the step of fusing data from sampling points at the same physical location across different sets of planar sections includes:

[0012] By stitching together the planar slices corresponding to the original spherical data, a first planar image is obtained; and by stitching together the planar slices corresponding to the rotated spherical data, a second planar image is obtained.

[0013] Based on the multiple rotations, the second planar diagram is rotated in reverse to obtain a third planar diagram corresponding to the first planar diagram.

[0014] Parallel data mean calculation is performed on sampling points at the same physical location between the first planar map and the third planar map to obtain target sampling points; and the target sampling points are combined to obtain a target planar map, thereby fusing the first planar map and the third planar map.

[0015] Optionally, dividing the original spherical data and the rotated spherical data into multiple sub-regions includes:

[0016] Determine the local feature density of the original spherical data;

[0017] The segmentation resolution is determined based on the local feature density; the local feature density is positively correlated with the segmentation resolution.

[0018] Based on the segmentation resolution, determine the target number of slices;

[0019] Based on the segmentation resolution and the target segmentation number, Healpix segmentation is performed on the original spherical data and the rotated spherical data respectively to obtain the multiple sub-regions.

[0020] Optionally, the step of projecting the multiple sub-regions of the same spherical data to obtain a set of planar slices includes:

[0021] A local coordinate system is established with the center of the sub-region as the projection center;

[0022] By using tangent projection, the spherical data of the sub-region is mapped onto a plane to obtain square planar slices of the same area, thus forming the set of planar slices.

[0023] Optionally, the step of rotating the original spherical data multiple times to obtain multiple rotated spherical data includes:

[0024] Get the specified number of rotations input by the user;

[0025] The original spherical data is controlled to rotate uniformly within a specified angle range for a specified number of rotations; and the spherical data after each rotation is used as the rotated spherical data.

[0026] Optionally, the method for processing spherical data further includes:

[0027] By stitching together the planar slices in the target slice set, a target planar image is obtained;

[0028] A quality parameter is determined for evaluating the quality of the target planar image; wherein the quality parameter includes at least one of a correlation parameter, a continuity parameter, a geometrical variation parameter, and a signal-to-noise ratio parameter; the correlation parameter reflects the correlation between the original spherical data and the target planar image; the continuity parameter reflects the continuity between adjacent slices in the target slice set; and the geometrical variation parameter reflects the degree of geometrical variation between the original spherical data and the target planar image.

[0029] Assign corresponding weighting coefficients to each of the aforementioned quality parameters;

[0030] The target quality value is obtained by weighted summation of the quality parameters and the weighting coefficients.

[0031] If the target quality value is greater than the threshold, the planar slices in the target slice set are converted into target format data.

[0032] Optionally, the method for processing the spherical data includes:

[0033] Obtain initial spherical data from multiple data sources;

[0034] The initial spherical data is standardized; wherein, the standardization process includes coordinate system unification, resolution unification, and numerical range standardization.

[0035] The initial spherical data after the standardization process is filtered; and the filtered initial spherical data is used as the original spherical data.

[0036] Another aspect of this application provides a spherical data processing apparatus, the apparatus comprising:

[0037] The acquisition module is used to acquire the original spherical surface data;

[0038] A rotation module is used to rotate the original spherical data multiple times to obtain multiple rotated spherical data.

[0039] The partitioning module is used to divide the original spherical data and the rotated spherical data into multiple sub-regions respectively;

[0040] The projection module is used to project the multiple sub-regions of the same spherical data respectively to obtain a set of planar slices;

[0041] The fusion module is used to fuse the data of sampling points at the same physical location between different sets of planar sections to obtain a set of target slices.

[0042] Another aspect of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the method for processing the spherical data.

[0043] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for processing the spherical data.

[0044] The method, apparatus, electronic device, and medium for processing spherical data provided in this application have the following beneficial effects: by performing multiple rotations, projections, and sampling point fusions on multiple original spherical data, the final target slice set can be directly applied to deep learning frameworks. Furthermore, multiple rotations and fusions can eliminate the geometric distortions and edge errors caused by converting spherical data into planar data. Simultaneously, the converted data retains the physical characteristics and spatial correlations of the original spherical data, thereby ensuring that deep learning frameworks can effectively learn spherical feature data and can be widely applied in various technical fields. Attached Figure Description

[0045] Figure 1 A flowchart illustrating a method for processing spherical data provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the rotation of spherical data provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of spherical data segmentation provided in an embodiment of this application;

[0048] Figure 4 A flowchart illustrating a method for processing spherical data provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the structure of a spherical data processing device provided in an embodiment of this application;

[0050] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0051] The reference numerals in the attached diagram are as follows: 50 is the acquisition module, 51 is the rotation module, 52 is the division module, 53 is the projection module, 54 is the fusion module, 60 is the memory, 61 is the processor, 62 is the display screen, 63 is the input / output interface, 64 is the communication interface, 65 is the power supply, 66 is the communication bus, 601 is the computer program, 602 is the operating system, and 603 is the data. Detailed Implementation

[0052] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0053] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0054] Figure 1 This is a flowchart illustrating a method for processing spherical data provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0055] S10: Obtain the original spherical surface data;

[0056] In a specific embodiment, the raw spherical data to be processed is obtained. This raw spherical data may be spherical scanning data generated by vehicle-mounted panoramic cameras, lidar, and millimeter-wave radar in the field of autonomous driving, or spherical imaging data generated by 360-degree panoramic cameras and spherical lidar in the field of drone aerial photography, or global spherical observation data generated by remote sensing satellites in the field of satellite remote sensing. This application does not limit the source of the raw spherical data.

[0057] It should be noted that the original spherical data can be understood as data sampled from sampling points in a spherical coordinate system. The sampling points can be pixels in an image or point cloud data. That is, the original spherical data can be composed of pixel points or point cloud data, etc., and this application does not limit this.

[0058] S11: Rotate the original spherical data multiple times to obtain multiple rotated spherical data;

[0059] Understandably, deep learning frameworks can only process planar data in Euclidean space. Therefore, it is necessary to transform the original spherical data into planar data. In this process, the original spherical data needs to be projected onto a plane. The projection process can be understood as the process of unfolding a sphere onto a plane. During this process, the geometric position undergoes stretching, compression, rotation and other changes, which requires re-interpolation. This results in changes to the sampling points, such as the pixel values ​​of the image, which may lead to geometric distortion.

[0060] To address the aforementioned technical issues, in one optional embodiment, the original spherical data is rotated to obtain multiple rotated spherical data sets. Specifically, in a specific embodiment, it is understood that the spherical coordinate system includes a polar angle θ and an azimuth angle φ. Rotation of the original spherical data includes fixing the polar angle θ and rotating while varying the azimuth angle φ, and fixing the azimuth angle φ and rotating while varying the polar angle θ. Figure 2 This is a schematic diagram of the rotation of spherical data provided in an embodiment of this application. For ease of understanding, it will be described below in conjunction with... Figure 2 Please provide an explanation.

[0061] For example, such as Figure 2 As shown in the figure above, the azimuth angle φ of the spherical data is fixed, while the polar angle θ is rotated by 60°, thus obtaining the rotating spherical data shown in the figure below.

[0062] It should be noted that multiple rotations can be performed with the azimuth angle φ remaining constant and only the polar angle θ being rotated, or vice versa. Of course, the polar angle θ and azimuth angle φ can also be rotated the same number of times; this application does not limit this. Furthermore, it should be noted that in order to minimize the geometric error caused by projection, rotations of the same angle can be performed each time, i.e., uniform angular rotations; this application does not limit the number of rotations.

[0063] S12: Divide the original spherical data and the rotated spherical data into multiple sub-regions respectively;

[0064] S13: Project multiple sub-regions of the same spherical data to obtain a set of planar slices;

[0065] To further reduce geometric distortion caused by projection, in one optional embodiment, the spherical data can be segmented, and then each segmented sub-region can be projected separately. Specifically, the original spherical data and all rotated spherical data are segmented separately, thereby dividing each spherical data into multiple sub-regions.

[0066] It should be noted that when segmenting the spherical data, the number, shape, and area of ​​the sub-regions obtained from each segment are the same. The specific shape of the sub-regions can be rectangular or other shapes, and this application does not limit this. Furthermore, this application does not limit the specific segmentation method of the spherical data or the number of sub-regions after segmentation.

[0067] After segmentation, each sub-region undergoes projection mapping, resulting in a corresponding planar image for each sub-region. Projecting all sub-regions of the same spherical data creates a set of planar slices. When the original spherical data is rotated 5 times, the final set of planar slices comprises 6 sets. It is worth noting that the projection method can be, but is not limited to, tangent projection, conformal projection, and isometric projection; this application does not impose any restrictions on the projection method.

[0068] Figure 3 This is a schematic diagram of spherical data segmentation provided in an embodiment of this application, as shown below. Figure 3 As shown, in one optional embodiment, the spherical data is divided into 192 sub-regions, where each sub-region is a rectangular region. Further, each sub-region is projected to obtain... Figure 3 For each planar slice shown, it should be noted that... Figure 3 The mid-plane section is a schematic diagram of a partial slice.

[0069] S14: Fusion of sampling points at the same physical location between different planar slice sets to obtain the target slice set.

[0070] Furthermore, by fusing slices from different planar slice sets, the resulting target slice set can overcome errors caused by projection. Specifically, data is fused from sampling points at the same physical location across different planar slice sets, for example... Figure 2 As shown, the spherical data is image data, and the sampling points are pixels. The sampling point corresponding to the original spherical data is point A, and the corresponding point of point A on the rotated spherical data is point B. Accordingly, data fusion involves fusing the pixels of point A and the pixels of point B. In an optional embodiment, data fusion may involve calculating the mean of the sampling point data.

[0071] In a specific embodiment, after fusing the slices from multiple planar slice sets, a target slice set with geometric errors eliminated can be obtained. The target slice set can be stitched together to obtain a planar target map of the original spherical data.

[0072] Finally, after obtaining the target slice set, in one optional embodiment, a mapping relationship is constructed between the spherical coordinates of the original spherical data and the planar coordinates of the slices in the target slice set, thereby obtaining the coordinate index array of each sampling point for subsequent data query, restoration and inverse mapping.

[0073] In one optional embodiment, after projecting the original spherical data and the rotated spherical data to obtain planar slices, in order to ensure the accuracy of data fusion, the sampled point data can be normalized and standardized for pixel values, that is, the original pixel values ​​are compressed or shifted to a range more suitable for neural network processing.

[0074] In another optional embodiment, label information is generated for the slice samples in the target slice set. This label information includes, but is not limited to, rotation parameters, celestial body type, coordinate range, and data quality level. Furthermore, to conserve storage resources, the generated target slice set can be compressed before storage.

[0075] To address the memory limitations of large-scale data processing, one optional implementation employs a batch processing strategy. Specifically, the total number of sub-regions is first calculated, and then the processing task is divided into multiple batches, with each batch processing the target number of patches (i.e., sub-regions) in parallel. After each batch is completed, temporary data in memory is promptly cleared to prevent memory overflow, thereby ensuring the ability to process ultra-large-scale sky map data far exceeding memory capacity while maintaining stable processing performance.

[0076] In one optional embodiment, the spherical data processing method provided in this application can process time-series data. Specifically, it first performs time alignment on the spherical data at different time points to ensure the consistency of the time series. For the data at each time point, standard spatial preprocessing is performed first, and then temporal features are extracted. Specifically, during temporal feature extraction, data from two time points before and after the current time point are considered, forming a sliding window of a preset number of time points (e.g., five). The spatial and temporal features are combined to generate comprehensive features containing spatiotemporal information, thereby supporting deep learning analysis of time-varying phenomena. This method can be applied to scenarios such as astronomical observation of time-varying celestial bodies, dynamic environmental perception for autonomous driving, and abnormal behavior detection in security monitoring.

[0077] In specific embodiments, to ensure data processing efficiency, as an optional embodiment, when the original sky map data (i.e., the original spherical data) is updated, only the changed areas are reprocessed, thereby improving processing efficiency. To facilitate real-time viewing of the data processing process, as an optional embodiment, data visualization can be provided to display intermediate results and the final planarized data during processing. In another optional embodiment, data processing at different data layers can be performed based on the frequency characteristics or physical quantity type of the sky map.

[0078] It should be noted that the spherical data processing method provided in this application can be performed in real time or at preset intervals. This application does not limit the processing method and the appropriate method can be selected according to actual business needs.

[0079] Therefore, the spherical data processing method provided in this application, through multiple rotations, projections, and sampling point fusions of multiple original spherical data, allows the final target slice set to be directly applied to deep learning frameworks. Furthermore, the multiple rotations and fusions eliminate the geometric distortions and edge errors caused by converting spherical data into planar data. Simultaneously, the converted data retains the physical properties and spatial correlations of the original spherical data, ensuring that deep learning frameworks can effectively learn spherical feature data and can be widely applied in various technical fields.

[0080] In one optional embodiment, data fusion is performed on sampling points at the same physical location across different sets of planar cross-sections, including:

[0081] By stitching together the planar slices corresponding to the original spherical data, a first planar image is obtained; and by stitching together the planar slices corresponding to the rotated spherical data, a second planar image is obtained.

[0082] Based on multiple rotations, the second planar diagram is rotated inversely to obtain a third planar diagram corresponding to the first planar diagram.

[0083] Parallel data mean calculation is performed on sampling points at the same physical location between the first and third planar maps to obtain target sampling points; and the target sampling points are combined to obtain the target planar map, thus fusing the first and third planar maps.

[0084] Figure 4 This is a flowchart illustrating a method for processing spherical data provided in an embodiment of this application. In a specific embodiment, as shown... Figure 4 As shown, by stitching together the planes in the set of plane sections corresponding to the original spherical data, a first planar image can be obtained, which is the overall planar image of the original spherical data. Simultaneously, by stitching together the planes in the set of plane sections corresponding to the rotated spherical data, a second planar image can be obtained.

[0085] It is understandable that the rotated spherical data is obtained by rotating the original spherical data. Therefore, in a specific embodiment, in order to achieve data fusion of subsequent sampling points at the same location, the second planar image is rotated inversely based on the original rotation to obtain the third planar image. For example... Figure 4 As shown, the first planar view corresponds to the third planar view, that is, they are similar images, with only some errors due to projection.

[0086] For example, when the original spherical data is image data, there may be some error in the pixel values ​​of sampling points at the same physical location between the first and third planar images. Therefore, in order to eliminate this error, the first and third planar images need to be fused to obtain the target planar image after error elimination.

[0087] Specifically, the average value of the data between the samples at the same physical location on the first and second planar images is calculated. For example, the average pixel value is calculated and used as the pixel value of the target planar image at that physical location. Thus, the target planar image after pixel fusion can be obtained.

[0088] In one alternative embodiment, to improve the efficiency of spherical data processing, the fusion of sampling point data at different locations can be performed in parallel using multi-threaded fusion calculations.

[0089] In one optional embodiment, the original spherical data and the rotated spherical data are each divided into multiple sub-regions, including:

[0090] Determine the local feature density of the original spherical data;

[0091] The segmentation resolution is determined based on the local feature density; the local feature density is positively correlated with the segmentation resolution.

[0092] Determine the target number of slices based on the segmentation resolution;

[0093] Based on the segmentation resolution and the number of target segments, Healpix segmentation is performed on the original spherical data and the rotated spherical data respectively to obtain multiple sub-regions.

[0094] In specific embodiments, this application does not limit the method of segmenting spherical data. However, in order to strictly ensure that the area of ​​each sub-region is the same after segmentation and to achieve efficient segmentation, in an optional embodiment, the spherical data can be segmented using the Healpix segmentation strategy.

[0095] It is worth noting that, in this embodiment, since the segmentation strategy is the Healpix segmentation strategy, the obtained raw spherical data can be Healpix data in FITS format. In fact, this application does not limit the format of the input data, and it can include, but is not limited to, FITS format, HDF5 format, and NetCDF format.

[0096] Understandably, in Healpix segmentation, the Nside segmentation resolution has a significant impact on the number of segmented sub-regions and the accuracy of subsequent projection, and the Nside segmentation resolution is closely related to the local feature density of the spherical data. Therefore, in one optional embodiment, the local feature density of the original spherical data is calculated before segmenting the spherical data using Healpix.

[0097] Furthermore, the current segmentation resolution is determined based on the local feature density, where local feature density is positively correlated with segmentation resolution; that is, the higher the local feature density, the higher the segmentation resolution. Based on this, in an optional embodiment, when the local feature density is greater than a first density threshold, the spherical data can be segmented at a high resolution, for example, an Nside segmentation resolution of 1024. When the local feature density is greater than a second density threshold but less than or equal to the first density threshold, a medium Nside segmentation resolution can be selected, for example, 512. When the local feature density is not greater than the second density threshold, a low resolution can be selected for segmentation, for example, 256, thereby optimizing computational efficiency while ensuring processing quality.

[0098] It should be noted that, in specific embodiments, the density threshold can be adjusted by adding more levels of local feature density judgment according to actual business needs, and this application does not limit this. Furthermore, it should be noted that, in addition to local feature density, the Nside segmentation resolution is also closely related to noise level and edge complexity.

[0099] It is worth noting that, in specific embodiments, the higher the Nside segmentation resolution, the more sub-regions are segmented, resulting in higher accuracy of the final planar data after projection and data fusion. Therefore, in specific embodiments, after obtaining the Nside segmentation resolution, the target number of slices can be determined based on the Nside segmentation resolution, i.e., the number of sub-regions can be determined. In an optional embodiment, the target number of slices and the Nside segmentation resolution satisfy the following relationship: target number of slices = 12 * Nside².

[0100] In one alternative embodiment, considering that higher noise levels are typically accompanied by higher edge complexity, which is related to the beamforming of the imaging device, the Nside segmentation resolution can be chosen to segment the spherical data when the noise level is lower and the imaging device structure is simpler. This reduces the number of sub-regions and improves computational efficiency. Conversely, when the noise level is higher and the edge complexity is greater, the Nside segmentation resolution can be chosen to segment the spherical data, resulting in a larger number of sub-regions and thus ensuring the accuracy of the planar data.

[0101] In another alternative embodiment, the Nside segmentation resolution ranges from 64 to 2048, preferably from 256 to 1024.

[0102] In a specific embodiment, after determining the Nside segmentation resolution and the target number of segments, the original spherical data and the rotated spherical data are segmented using Healpix segmentation to obtain multiple sub-regions. For example, as... Figure 3 As shown, when the Nside segmentation resolution is set to 4, the target segmentation number = 12 * Nside² = 12 * 16 = 192. Therefore, the spherical data can be evenly divided into 192 sub-regions of equal area.

[0103] In one optional embodiment, the segmentation parameters of Healpix segmentation can be dynamically adjusted based on the characteristics of the spherical data, and multiple planar datasets with different resolutions can be generated simultaneously by setting different Nside segmentation resolutions. Thus, the same original spherical data can generate multi-resolution planar datasets, improving the accuracy of spherical data usage in various fields and providing precise data support for deep learning frameworks in different domains. For example, based on multi-scale data, both long-distance and short-distance targets can be detected simultaneously in autonomous driving, and both panoramic views and local details can be identified simultaneously in security monitoring.

[0104] In another alternative embodiment, after determining the Nside segmentation resolution, the target segmentation number is calculated. Simultaneously, a coordinate index array is generated; specifically, the spherical coordinates (θ, φ) of each sampling point are obtained and converted to a Cartesian coordinate system (x, y, z) to prepare for subsequent projection transformations.

[0105] Therefore, the spherical data processing method provided in this application segmentes the spherical data using the Healpix segmentation strategy, ensuring that the segmented sub-regions have the same area and identical geometric features, facilitating subsequent projection and data statistics. Furthermore, the segmentation resolution parameter in Healpix segmentation can be dynamically adjusted, enabling the simultaneous generation of multi-scale planarized data, improving the accuracy of spherical data processing, and providing rich data support for practical applications.

[0106] In one optional embodiment, multiple sub-regions of the same spherical data are projected to obtain a set of planar slices, including:

[0107] Establish a local coordinate system with the center of the sub-region as the projection center;

[0108] By using tangent projection, the spherical data of the sub-region is mapped onto a plane to obtain square planar slices of the same area, thus forming a set of planar slices.

[0109] Based on the above embodiments, each sub-region is projected to transform the spherical data into planar data. Specifically, the center of the sub-region is used as the projection center, and a local coordinate system is established. Based on this local coordinate system, as an optional embodiment, tangent projection is used to map the spherical data of the sub-region onto a plane, thereby obtaining square planar patches with the same area and geometric features, such as... Figure 3 The partial planar slice shown.

[0110] It should be noted that, in a specific embodiment, the sub-region can be divided into rectangles, and the corresponding generated plane can also be a rectangle. However, in order to facilitate the convolution calculation in the subsequent deep learning framework, in an optional embodiment, the projection plane of the sub-region is a square with the same area.

[0111] Furthermore, it should be noted that in specific embodiments, the size of the tangential plane after projection is closely related to the Nside segmentation resolution. For example, when the Nside segmentation resolution is 512, the output image after tangential projection is a square of 256×256 pixels. It is evident that the Nside segmentation resolution is a key parameter in spherical data segmentation, defining the resolution of spherical partitioning and determining the number of pixels on the sphere and the size of each pixel.

[0112] In one optional embodiment, the original spherical data is rotated multiple times to obtain multiple rotated spherical data, including:

[0113] Get the specified number of rotations input by the user;

[0114] Control the original spherical data to rotate uniformly within a specified angle range a specified number of times; and use the spherical data after each rotation as the rotated spherical data.

[0115] It is understandable that, in specific embodiments, the final target slice set needs to be fused with planar slices obtained by projecting the original spherical data and planar slices of the rotated spherical data to overcome the geometric errors caused by the projection process. Therefore, the number of rotated spherical data, i.e., the setting of the number of rotations and the rotation angle, is crucial to the accuracy of the final target slice.

[0116] In a specific embodiment, the number of rotations can be set by the user according to actual business needs. For high-precision application scenarios, a larger number of specified rotations can be selected. For application scenarios with high requirements for data processing efficiency, the number of specified rotations can be appropriately reduced to improve data processing efficiency.

[0117] After obtaining the specified number of rotations input by the user, in one optional embodiment, to further improve data processing accuracy, the number of rotations can be evenly distributed within a specified angle range. The specified angle range can be from 0 to π, or from 0 to 2π, and this application does not limit this range.

[0118] It is understood that the spherical coordinate system includes a polar angle θ and an azimuth angle φ. Rotation refers to rotating the polar angle θ or the azimuth angle φ. In one optional embodiment, the specified angle range is 0 to 2π. Therefore, the original spherical data can be controlled to rotate uniformly within the range of 0 to 2π a specified number of times, and the spherical data after each rotation is taken as a rotated spherical data. Thus, multiple sets of planar data from different perspectives can be generated, improving the accuracy of data processing.

[0119] In one alternative embodiment, during uniform rotation, the polar angle θ can be fixed, and the azimuth angle φ can be uniformly rotated a first number of times within 0 to 2π. In addition, the azimuth angle φ can be fixed, and the polar angle θ can be uniformly rotated a second number of times within 0 to 2π, wherein the sum of the first number of rotations and the second number of rotations is equal to the specified number of rotations.

[0120] For example, within the range of 0 to 2π, with the polar angle θ fixed, rotating the azimuth angle φ by 60° each time results in 6 rotations. Similarly, with the azimuth angle φ fixed, rotating the polar angle θ by 60° each time also results in 6 rotations, thus rotating the original spherical data a total of 12 times.

[0121] Therefore, the spherical data processing method provided in this application, through multiple rotations and averaging of sampling point data, solves the geometric errors caused by the conversion of spherical data to planar data, overcomes the distortion problem in the edge region, and improves the accuracy of target detection and recognition. It can be widely used in many fields such as astronomical observation, autonomous driving, drone aerial photography, intelligent security and satellite remote sensing, and has good versatility.

[0122] In an optional embodiment, the spherical data processing method provided in this application further includes:

[0123] By stitching together the planar slices from the target slice set, a target planar image is obtained;

[0124] Determine the quality parameters used to evaluate the quality of the target planar map; wherein the quality parameters include at least one of the following: correlation parameter, continuity parameter, geometric change parameter, and signal-to-noise ratio parameter; the correlation parameter is used to reflect the correlation between the original spherical data and the target planar map; the continuity parameter is used to reflect the continuity between adjacent slices in the target slice set; the geometric change parameter is used to reflect the degree of geometric change between the original spherical data and the target planar map;

[0125] Assign corresponding weighting coefficients to each quality parameter;

[0126] The target quality value is obtained by weighted summation of the quality parameters and weighting coefficients.

[0127] If the target quality value is greater than the threshold, the planar slices in the target slice set will be converted into target format data.

[0128] In a specific embodiment, to improve the accuracy of spherical data applications in various fields, such as improving environmental perception accuracy in the field of autonomous driving, as an optional embodiment based on the above embodiments, the acquired planar data is subjected to quality assessment so that high-quality data can be used directly while low-quality data is discarded.

[0129] Specifically, the planar slices in the target slice set obtained in the above embodiments are stitched together to obtain a target planar map. Further, based on this target planar map, at least one of the following quality parameters is calculated: correlation parameter, continuity parameter, geometric change parameter, and signal-to-noise ratio parameter.

[0130] In one optional embodiment, the target planar image and the original spherical data can be used as input data for a large target model, allowing for direct evaluation of quality parameters through the large target model. Specifically, the correlation between the currently obtained target planar image and the original spherical data is evaluated to obtain correlation parameters. Simultaneously, the continuity between adjacent planar slices in the target slice set is examined to determine if the continuity of the entire target planar image meets expectations. Furthermore, comparing the original spherical data and the target planar image can determine the degree of geometric change in the target planar image; that is, it can be determined whether the changes in geometric features after projection transformation exceed acceptable limits. Additionally, the signal-to-noise ratio (SNR) parameter of the target planar image can be calculated to measure signal quality.

[0131] Furthermore, corresponding weighting coefficients are assigned to different quality parameters, and a weighted sum is performed based on the quality parameters and weighting coefficients to obtain the target quality value. When the target quality value is greater than the threshold, it can be determined that the quality of the target planar map meets the expectations, and the planar slices in the target slice set can be converted into target format data for direct use in subsequent practical application scenarios.

[0132] Of course, if the target quality value is not greater than the threshold, it is determined that the quality of the target planar image does not meet expectations. The current original spherical data and target slice set are then removed, and a prompt signal is sent to the terminal so that the user can check it in a timely manner.

[0133] It should be noted that, in one optional embodiment, to meet the needs of different technical fields, the planar slice format in the output target slice set may include, but is not limited to, NumPy format, TensorFlow format, and PyTorch format. Therefore, the output data can support the data interfaces of multiple deep learning frameworks.

[0134] In specific embodiments, the spherical data processing method employed in this application can automatically detect data formats and automatically identify data formats based on file extensions or file header information. For different input formats, appropriate parsers are used for data loading. Simultaneously, for different output formats, corresponding data structures and metadata information can be generated according to the requirements of deep learning frameworks.

[0135] In one optional embodiment, the spherical data processing method provided in this application includes:

[0136] Obtain initial spherical data from multiple data sources;

[0137] The initial spherical data is standardized; the standardization process includes coordinate system unification, resolution unification, and numerical range standardization.

[0138] The initial spherical data after standardization is filtered, and the filtered initial spherical data is used as the original spherical data.

[0139] In a specific embodiment, to further improve data processing accuracy, initial spherical data from multiple data sources is acquired and standardized. This initial spherical data from multiple sources may include, but is not limited to, data from autonomous driving vehicle-mounted panoramic cameras, LiDAR point cloud data, millimeter-wave radar data, 360-degree panoramic aerial images from drones, LiDAR data from drones, infrared thermal imaging data, intelligent security panoramic surveillance videos, spherical surveillance camera data, 360-degree security surveillance data, as well as satellite remote sensing global observation data, multispectral satellite images, and high-resolution Earth observation data.

[0140] Understandably, to facilitate subsequent parallel processing of multi-source data and thus improve data processing efficiency, the initial spherical data needs to undergo standardization processes such as unifying the coordinate system, resolution, and numerical range. Coordinate system unification converts different coordinate systems into a unified one; resolution unification resamples data from different resolutions to the same resolution; and numerical range standardization normalizes different types of data to the same numerical range. Furthermore, to ensure the quality of the original spherical data, the standardized initial spherical data needs to be filtered. This filtering can include, but is not limited to, removing low-quality data such as noisy data, and handling missing and outlier values.

[0141] In one alternative embodiment, multi-source data registration is provided based on data standardization processing to ensure accurate spatial alignment of data from different sources. Furthermore, a weighted fusion strategy is employed to determine weights based on the quality and reliability of different data sources, generating a unified fused dataset.

[0142] To facilitate understanding, the following examples illustrate the specific applications of the spherical data processing method provided in this application in real-world scenarios.

[0143] For panoramic camera systems installed in autonomous vehicles, input 360-degree panoramic image data. Processing parameters can be set as follows: Nside segmentation resolution of 256, equidistant projection method, and output image size of 256×256 pixels. First, spherical mapping is performed on the panoramic image, converting the image acquired by the fisheye lens or panoramic lens into a standard spherical coordinate system. Then, Healpix segmentation is performed, dividing the panoramic field of view into multiple regions, each corresponding to different directions and distances around the vehicle. Planar projection is then performed on each region to generate planar image data suitable for object detection and semantic segmentation.

[0144] For 3D point cloud data generated by vehicle-mounted LiDAR, the point cloud data is mapped to a spherical coordinate system, establishing a spherical coordinate system with the LiDAR as the center. Further, based on the distance and angle information of the point cloud, each point is mapped to its corresponding spherical position. Then, an adaptive segmentation strategy is employed, using higher resolution in areas with dense targets (e.g., the road ahead) and lower resolution in areas with sparse targets (e.g., the sky). After multi-rotation processing and data fusion, a planarized dataset containing depth information is generated, supporting 3D target detection and path planning.

[0145] For the 360-degree panoramic camera mounted on the drone, the input is a spherical panoramic aerial image. The Nside segmentation resolution is set to 512, the projection method is selected as tangent projection, and the output image size is set to 512×512 pixels. Spherical coordinate mapping is performed on the panoramic aerial image to convert the drone's omnidirectional field of view into a spherical data format. Specifically, adaptive Healpix segmentation is performed, adjusting the segmentation parameters according to the distribution density of ground targets. High-resolution segmentation is used in areas with dense ground buildings and roads, while standard-resolution segmentation is used in the sky and water areas.

[0146] The drone is equipped with a visible light camera, an infrared camera, and a LiDAR sensor to fuse data from multiple sensors. First, the data from different sensors are unified to the same spherical coordinate system, followed by time synchronization and spatial registration. For each type of sensor data, Healpix segmentation and planar projection processing are performed separately. Finally, the multi-source data are fused at the pixel level to generate a comprehensive dataset containing visible light, infrared, and depth information, supporting target recognition and 3D reconstruction in complex environments.

[0147] For panoramic PTZ cameras used in security monitoring, a 360-degree panoramic video stream is input, the Nside segmentation resolution is set to 1024, the projection method is selected as equidistant projection, and the output image size is set to 256×256 pixels. The panoramic video stream is received in real time, and spherical coordinate mapping is performed on each frame. Real-time Healpix segmentation is executed, adaptively adjusting the segmentation parameters according to the complexity of the monitoring scene. High-resolution segmentation is used in areas with frequent personnel activity, while standard-resolution segmentation is used in static background areas.

[0148] To meet the real-time requirements of security monitoring, a streaming processing architecture is adopted to perform real-time analysis of the surveillance video stream. Target detection and behavior analysis are performed in parallel for each segmented area, detecting the presence and movement of targets such as people and vehicles. Historical state information for each area is maintained, and abnormal behaviors, such as intrusion detection, aggregation detection, and object removal, are detected through time-series analysis. When an anomaly is detected, the relevant area is automatically marked and an alarm is generated, while the relevant planarized data is saved for subsequent analysis.

[0149] For global observation data from remote sensing satellites, input global satellite remote sensing images, set the Nside segmentation resolution to 2048, select equal-area projection, and set the output image size to 512×512 pixels. Map the global remote sensing data to a standard Earth spherical coordinate system, considering the Earth's ellipsoidal shape and projection transformation. Perform global-scale Healpix segmentation, dividing the Earth's surface into regions of equal area to ensure data from different latitudes have the same weight.

[0150] In specific embodiments, satellite remote sensing data typically contains multiple spectral bands, and multispectral data fusion processing is performed. First, radiometric calibration and atmospheric correction are applied to the data from different bands, then they are unified to the same spherical coordinate system. Healpix segmentation and planar projection are performed on each spectral band separately to generate multi-channel planarized data. Finally, the multispectral data are fused at the channel level to generate a dataset containing rich spectral information, supporting applications such as land use classification, vegetation monitoring, and disaster assessment.

[0151] In the above embodiments, the method for processing spherical data has been described in detail. This application also provides an embodiment of a device for processing spherical data.

[0152] Figure 5 This is a schematic diagram of the structure of a spherical data processing device provided in an embodiment of this application, as shown below. Figure 5 As shown, the device includes:

[0153] Module 50 is used to acquire the original spherical surface data;

[0154] Rotation module 51 is used to rotate the original spherical data multiple times to obtain multiple rotated spherical data.

[0155] The partitioning module 52 is used to divide the original spherical data and the rotated spherical data into multiple sub-regions respectively;

[0156] Projection module 53 is used to project multiple sub-regions of the same spherical data separately to obtain a set of planar slices;

[0157] The fusion module 54 is used to fuse the data of sampling points at the same physical location between different planar cross-section sets to obtain the target slice set.

[0158] Furthermore, the spherical data processing apparatus provided in this application embodiment also includes:

[0159] The stitching module is used to stitch together the planar slices corresponding to the original spherical data to obtain the first planar image; and to stitch together the planar slices corresponding to the rotated spherical data to obtain the second planar image;

[0160] The anti-rotation module is used to perform anti-rotation on the second planar diagram based on multiple rotations to obtain a third planar diagram corresponding to the first planar diagram.

[0161] The target sampling point determination module is used to perform parallel data mean calculation on sampling points at the same physical location between the first and third planar maps to obtain target sampling points; and to combine the target sampling points to obtain the target planar map, so as to fuse the first and third planar maps.

[0162] The local feature density determination module is used to determine the local feature density of the original spherical data;

[0163] The segmentation resolution determination module is used to determine the segmentation resolution based on the local feature density; the local feature density is positively correlated with the segmentation resolution.

[0164] The target segmentation number determination module is used to determine the target segmentation number of slices based on the segmentation resolution.

[0165] The Healpix segmentation module is used to perform Healpix segmentation on the original spherical data and the rotated spherical data based on the segmentation resolution and the target number of segments, respectively, to obtain multiple sub-regions.

[0166] The tangent projection module is used to establish a local coordinate system with the center of the sub-region as the projection center. Through tangent projection, the spherical data of the sub-region is mapped to a plane to obtain square planar slices of the same area, which form a set of planar slices.

[0167] The rotation count acquisition module is used to acquire the specified number of rotations input by the user;

[0168] The control module is used to control the original spherical data to rotate uniformly within a specified angle range a specified number of times; and to use the spherical data after each rotation as the rotated spherical data.

[0169] The stitching module is also used to stitch together planar slices from the target slice set to obtain the target planar image;

[0170] The quality parameter determination module is used to determine the quality parameters used to evaluate the quality of the target planar map; wherein, the quality parameters include at least one of the following: correlation parameter, continuity parameter, geometric change parameter, and signal-to-noise ratio parameter; the correlation parameter is used to reflect the correlation between the original spherical data and the target planar map; the continuity parameter is used to reflect the continuity between adjacent slices in the target slice set; and the geometric change parameter is used to reflect the degree of geometric change between the original spherical data and the target planar map;

[0171] The weighting coefficient allocation module is used to assign corresponding weighting coefficients to each quality parameter;

[0172] The weighted summation module is used to sum the quality parameters and weighting coefficients to obtain the target quality value.

[0173] The format conversion module is used to convert planar slices in the target slice set into target format data if the target quality value is greater than a threshold.

[0174] The acquisition module is also used to acquire initial spherical data from multiple data sources;

[0175] The standardization module is used to standardize the initial spherical data; the standardization process includes coordinate system unification, resolution unification, and numerical range standardization.

[0176] The filtering module is used to filter the standardized initial spherical data and use the filtered initial spherical data as the original spherical data.

[0177] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the electronic device includes: a memory 60 for storing computer programs;

[0178] The processor 61 is configured to execute a computer program to implement the steps of the spherical data processing method as described in the above embodiments.

[0179] The electronic devices provided in this embodiment may include, but are not limited to, laptops or desktop computers.

[0180] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 61 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 61 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0181] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 60 is used to store at least the following computer program 601, which, after being loaded and executed by the processor 61, is capable of implementing the relevant steps of the spherical data processing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. The operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include, but is not limited to, the relevant data involved in the spherical data processing method.

[0182] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64, a power supply 65, and a communication bus 66.

[0183] Those skilled in the art will understand that Figure 6 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0184] The electronic device provided in this application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the spherical data processing method described in the above embodiments.

[0185] It should be noted that although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Claims

1. A method of processing spherical data, characterized by, The method comprises: acquiring original spherical data; the original spherical data is composed of pixel points or point cloud data; rotating the original spherical data multiple times to obtain multiple rotated spherical data; dividing the original spherical data and the rotated spherical data into multiple sub-regions respectively; projecting the multiple sub-regions of the same spherical data respectively to obtain a set of plane slices; fusing sampling points at the same physical position between different sets of plane slices to obtain a target slice set; the dividing the original spherical data and the rotated spherical data into multiple sub-regions respectively comprises: determining the local feature density of the original spherical data; determining the segmentation resolution according to the local feature density; the local feature density is positively correlated with the segmentation resolution; determining the target segmentation number of slices according to the segmentation resolution; based on the segmentation resolution and the target segmentation number, performing Healpix segmentation on the original spherical data and the rotated spherical data respectively to obtain the multiple sub-regions.

2. The method of processing spherical data according to claim 1, wherein, the fusing sampling points at the same physical position between different sets of plane slices comprises: splicing the plane slices corresponding to the original spherical data to obtain a first plane map; and splicing the plane slices corresponding to the rotated spherical data to obtain a second plane map; based on the multiple rotations, performing anti-rotation on the second plane map to obtain a third plane map corresponding to the first plane map; performing parallel data mean calculation on sampling points at the same physical position between the first plane map and the third plane map to obtain target sampling points; and combining the target sampling points to obtain a target plane map to fuse the first plane map and the third plane map.

3. The method of processing spherical data according to claim 1, wherein, the projecting the multiple sub-regions of the same spherical data respectively to obtain a set of plane slices comprises: taking the center of the sub-region as the projection center and establishing a local coordinate system; mapping the spherical data of the sub-region to a plane by tangent projection to obtain square plane slices with the same area to constitute the set of plane slices.

4. The method of processing spherical data according to claim 1, wherein, the rotating the original spherical data multiple times to obtain multiple rotated spherical data comprises: acquiring a specified number of rotations input by a user; controlling the original spherical data to rotate uniformly within a specified angle range for the specified number of rotations; and taking the spherical data after each rotation as the rotated spherical data.

5. The method of processing spherical data according to claim 1, wherein, The method further comprises: splicing the plane slices in the target slice set to obtain a target plane map; determining a quality parameter for evaluating the quality of the target plane map; wherein the quality parameter comprises at least one of a correlation parameter, a continuity parameter, a geometric change parameter and a signal-to-noise ratio parameter; the correlation parameter is used to reflect the correlation between the original spherical data and the target plane map; the continuity parameter is used to reflect the continuity between adjacent slices in the target slice set; the geometric change parameter is used to reflect the degree of geometric change between the original spherical data and the target plane map; assigning corresponding weight coefficients to each quality parameter; performing weighted summation on the quality parameters and the weight coefficients to obtain a target quality value; if the target quality value is greater than a threshold, converting a planar slice in the target slice set into target format data.

6. The method of processing spherical data according to claim 5, wherein, The method comprises: obtaining initial spherical data of multiple data sources; performing standardization processing on the initial spherical data; wherein the standardization processing comprises uniform coordinate system processing, uniform resolution processing and numerical range standardization processing; performing filtering on the initial spherical data after the standardization processing; and taking the filtered initial spherical data as the original spherical data.

7. A processing device of spherical data, characterized by, The device comprises: an acquisition module configured to acquire original spherical data; the original spherical data is composed of pixel points or point cloud data; a rotation module configured to rotate the original spherical data multiple times to obtain multiple rotated spherical data; a division module configured to divide the original spherical data and the rotated spherical data into multiple sub-regions, respectively; a projection module configured to project the multiple sub-regions of the same spherical data, respectively, to obtain a set of planar slices; a fusion module configured to fuse sampling points at the same physical position between different planar slice sets to obtain a target slice set; a local feature density determination module configured to determine a local feature density of the original spherical data; a segmentation resolution determination module configured to determine a segmentation resolution according to the local feature density; the local feature density is positively correlated with the segmentation resolution; a target segmentation number determination module configured to determine a target segmentation number of slices according to the segmentation resolution; a Healpix segmentation module configured to perform Healpix segmentation on the original spherical data and the rotated spherical data, respectively, based on the segmentation resolution and the target segmentation number, to obtain the multiple sub-regions.

8. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program operable to run on said processor, characterized in that, The processor executes the computer program to implement the steps of the spherical data processing method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the spherical data processing method in any one of claims 1 to 6.

Images