Data redundancy processing method and related equipment

Abstract

The invention discloses a data redundancy processing method and related equipment, and relates to the technical field of data redundancy processing, and the method comprises the steps: obtaining a current frame collected by a sensor, carrying out the preprocessing of the current frame, obtaining a to-be-divided frame, and enabling the current frame to comprise an image frame or a point cloud frame; dividing the frame to be divided into a plurality of target areas; performing difference calculation on the corresponding target areas between the adjacent frames to determine whether the corresponding target areas are redundant areas or not; and removing or compressing the target region corresponding to the redundant region. According to the method and the device, the technical problem that the efficiency and the real-time performance of data processing are reduced due to redundant data between continuous frames in the prior art is solved, so that the efficiency and the real-time performance of data processing are improved.

Description

Technical Field

[0001] This application relates to the field of data technology, and in particular to a data redundancy processing method and related equipment. Background Technology

[0002] Currently, vehicle sensors continuously collect large amounts of image or point cloud data for driving training or analysis. However, during vehicle operation, some areas (such as roads and backgrounds) do not change much between consecutive frames, resulting in a large amount of redundant data. This redundant data not only occupies a lot of storage space and bandwidth resources, but also reduces the efficiency and real-time performance of subsequent data processing. Summary of the Invention

[0003] The main purpose of this application is to provide a data redundancy processing method and related equipment, which aims to solve the technical problem in the current technology of redundant data between consecutive frames, which reduces the efficiency and real-time performance of data processing.

[0004] To achieve the above objectives, this application proposes a data redundancy processing method, which includes: The current frame acquired by the sensor is obtained and preprocessed to obtain the frame to be divided. The current frame includes an image frame or a point cloud frame. The frame to be divided is divided into several target regions; For the corresponding target regions between adjacent frames, perform differential calculation to determine whether the corresponding target regions are redundant regions; The corresponding target regions that are redundant are removed or compressed.

[0005] In one embodiment, the step of dividing the frame to be divided into several target regions includes: The size of each target region is determined based on the resolution of the sensor and the scene in the current frame; The frame to be divided is divided into several target regions based on the size of each target region.

[0006] In one embodiment, if the current frame is an image frame, the step of determining the size of each target region based on the sensor's resolution and the scene in which the current frame is located includes: Determine the image resolution and current vehicle speed for each image frame; The size of each target area is determined based on the resolution and the current vehicle speed; The step of dividing the frame to be divided into several target regions according to the size of each target region includes: The frame to be divided is divided into several target regions based on the size of each target region and the preset region division step size.

[0007] In one embodiment, if the current frame is a point cloud frame, the step of determining the size of each target region based on the sensor's resolution and the scene in which the current frame is located includes: The size of the corresponding fixed voxel is determined based on the sensor's resolution, the corresponding field of view depth range, and the average depth of the current frame. Projecting the point cloud frame onto a fixed voxel of the specified size yields the number of aggregated points and the corresponding first boundary range of the first voxel unit of the fixed voxel. Based on the number of aggregation points, if it is determined that the first voxel unit needs to be further divided, the first voxel unit is further divided until the number of aggregation points of the first voxel unit that needs to be further divided meets the preset requirements, the division is determined to be complete, and the second voxel unit and the corresponding second boundary range are obtained. Based on the first voxel unit and its corresponding first boundary range, and the second voxel unit and its corresponding second boundary range, the size of each target region is determined.

[0008] In one embodiment, the step of performing differential calculation on corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions includes any one of the following: For the corresponding target regions between adjacent frames, a difference calculation is performed to determine whether the corresponding difference value is less than a preset threshold. If it is determined to be less than the preset threshold, the corresponding target region is determined to be a redundant region. The preset threshold is dynamic and is associated with a base threshold and a dynamic adjustment term. The dynamic adjustment term is associated with speed and preset environmental complexity, respectively. For the corresponding target regions between adjacent frames, differential calculation is performed to determine the degree of change of the corresponding target regions, and the corresponding target regions are determined to be redundant regions based on the degree of change.

[0009] In one embodiment, when the current frame is an image frame, the step of performing difference calculation on corresponding target regions between adjacent frames to determine the degree of change of the corresponding target regions includes: The average grayscale difference between corresponding target regions between adjacent frames is determined, and the structural similarity index between corresponding target regions between adjacent frames is determined, wherein the structural similarity index is related to the brightness, brightness variance, and covariance of the corresponding target regions. The degree of change in the corresponding target region is determined based on the average grayscale difference and the structural similarity index.

[0010] In one embodiment, when the current frame is a point cloud frame, the step of performing differential calculation on corresponding target regions between adjacent frames to determine the degree of change of the corresponding target regions includes: ICP registration is performed on the point clouds of the corresponding target regions in adjacent frames, and the average translation distance after registration is calculated. Determine the density change rate of the corresponding target region between adjacent frames; Based on the average translation distance and density change rate, the degree of change in the corresponding target area is determined.

[0011] Furthermore, to achieve the above objectives, this application also proposes a data redundancy processing apparatus, which includes: The acquisition module is used to acquire the current frame collected by the sensor and preprocess the current frame to obtain the frame to be divided. The current frame includes an image frame or a point cloud frame. A segmentation module is used to divide the frame to be segmented into several target regions; The determination module is used to perform differential calculations on corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions. The processing module is used to remove or compress the corresponding target regions that are redundant regions.

[0012] In addition, to achieve the above objectives, this application also proposes a data redundancy processing apparatus, the apparatus comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the data redundancy processing steps as described above.

[0013] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the data redundancy processing steps described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the data redundancy processing steps described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: Compared to related technologies where redundant data between consecutive frames is not processed, which reduces data processing efficiency and real-time performance, this application obtains the current frame acquired by the sensor and preprocesses it to obtain the frame to be divided. The current frame includes an image frame or a point cloud frame. The frame to be divided is then divided into several target regions. Differential calculation is performed on the corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions. The corresponding target regions that are redundant are then removed or compressed. In other words, this application accurately determines the target regions with corresponding redundancy between consecutive frames based on differential calculation, and removes or compresses the corresponding redundant regions, ensuring that subsequent data processing does not suffer from low processing efficiency or reduced real-time performance due to redundant regions. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a first flowchart illustrating the data redundancy processing method of this application. Figure 2 This is a flowchart illustrating the entire process of the data redundancy processing method in this application. Figure 3 This is a schematic diagram of the second process provided in Embodiment 1 of the data redundancy processing method of this application; Figure 4 This is a flowchart illustrating Embodiment 2 of the data redundancy processing method of this application. Figure 5 This is a schematic diagram of the module structure of the data redundancy processing device according to an embodiment of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the data redundancy processing method in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is: to acquire the current frame collected by the sensor and preprocess the current frame to obtain the frame to be divided, wherein the current frame includes an image frame or a point cloud frame; to divide the frame to be divided into several target regions; to perform differential calculation on the corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions; and to remove or compress the corresponding target regions that are redundant regions.

[0023] Compared to related technologies where redundant data between consecutive frames is not processed, which reduces data processing efficiency and real-time performance, this application obtains the current frame acquired by the sensor and preprocesses it to obtain the frame to be divided. The current frame includes an image frame or a point cloud frame. The frame to be divided is then divided into several target regions. Differential calculation is performed on the corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions. The corresponding target regions that are redundant are then removed or compressed. In other words, this application accurately determines the target regions with corresponding redundancy between consecutive frames based on differential calculation, and removes or compresses the corresponding redundant regions, ensuring that subsequent data processing does not suffer from low processing efficiency or reduced real-time performance due to redundant regions.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data redundancy processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or data redundancy processing device capable of performing the above functions. The following description uses a data redundancy processing device as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, embodiments of this application provide a data redundancy processing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data redundancy processing method of this application.

[0026] In this embodiment, the data redundancy processing method includes steps S10 to S40: Step S10: Obtain the current frame collected by the sensor and preprocess the current frame to obtain the frame to be divided. The current frame includes an image frame or a point cloud frame. It should be noted that the execution subject of this embodiment is a data redundancy processing device, which belongs to a data redundancy processing system. Specifically, the data redundancy processing system can be an autonomous driving system. The following will take an autonomous driving system as an example for detailed explanation.

[0027] In this embodiment, a specific application scenario may be: In autonomous driving systems, onboard sensors (such as cameras and lidar) continuously collect a large amount of image or point cloud data. Since some areas (such as roads and backgrounds) do not change much between consecutive frames during vehicle operation, a large amount of redundant data is generated. This redundant data not only occupies a lot of storage space and bandwidth resources, but may also affect the efficiency and real-time performance of subsequent data processing.

[0028] In this embodiment, the current frame collected by the sensor is first obtained. Specifically, it can be video data or point cloud data collected by the sensor. The frame that needs to be processed in the collected video data is taken as the current frame, or the frame that needs to be processed in the collected point cloud data is taken as the current frame. That is, in this embodiment, the current frame includes an image frame or a point cloud frame.

[0029] In this embodiment, the current frame collected by the sensor is acquired, and the current frame is preprocessed to obtain the frame to be divided, such as... Figure 2 As shown, the preprocessing includes operations such as normalization, noise reduction, and downsampling.

[0030] In this embodiment, Gaussian filtering can be used to remove noise. For example, Gaussian filtering (σ=1.0) can be used to remove noise. In addition, in this embodiment, noise can also be removed based on empirical values ​​or based on common image denoising parameters.

[0031] In this embodiment, when downsampling the current frame, the corresponding downsampling rate is dynamically adjusted based on the hardware processing capability, such as dynamically adjusting according to the sensor resolution (for example, a 1080p image can be downsampled to 720p).

[0032] In this embodiment, preprocessing the current frame can make the data more effective, which can improve the efficiency and accuracy of subsequent processing.

[0033] Step S20: Divide the frame to be divided into several target regions; In this embodiment, after determining the frame to be divided, the frame to be divided is divided into several target regions according to the requirements, wherein the target regions are generally local regions.

[0034] The method for dividing the frame to be divided into several target regions can be as follows: Method 1: Divide the frame to be segmented into several target regions; Method 2: Dynamically divide the frame to be divided into several target regions.

[0035] In this embodiment, the target region can specifically be an image patch or a subset of a point cloud.

[0036] Among them, reference Figure 3 The step of dividing the frame to be divided into several target regions includes: Step S21: Determine the size of each target region based on the resolution of the sensor and the scene in the current frame; In this embodiment, when the frame to be divided is dynamically divided into several target regions, the size of each target region can be determined based on the sensor resolution and the scene in which the current frame is located. Specifically, when the target region is an image patch or a subset of a point cloud, the corresponding sensor resolution and the scene in which the current frame is located can be different.

[0037] Specifically, if the current frame is an image frame, the step of determining the size of each target region based on the sensor's resolution and the scene in which the current frame is located includes: Step A1: Determine the image resolution and current vehicle speed of the corresponding image frame; In this embodiment, if the current frame is an image frame, the corresponding image resolution and the current vehicle speed are determined. For example, the resolution of the camera that captures the image frame is determined and used as the image resolution, and the current vehicle speed on the instrument panel is read directly.

[0038] It should be noted that even with the same camera, the resolution of the camera can be different in different scenarios. For example, in a foggy scene, the resolution of the camera can be higher than that in a normal scene.

[0039] Step A2: Determine the size of each target area based on the resolution and the current vehicle speed; Once the resolution and current vehicle speed are determined, the size of each target area can be determined. For example, the image frame division method can be: Let the image resolution be Rw×Rh, and the current vehicle speed be v km / h.

[0040] The size of the target area, i.e., the image block size, is determined to be B = clamp(128-0.8v, 32) pixels. That is, the higher the speed, the larger the block size. If the current vehicle speed varies within the range of 0-120km / h, the image block size will automatically adjust within the range of [32, 128] pixels.

[0041] Step S22: Divide the frame to be divided into several target regions according to the size of each target region.

[0042] After determining the size of each target region, the frame to be divided is divided into several target regions in a manner that ensures that no regions overlap.

[0043] The step of dividing the frame to be divided into several target regions according to the size of each target region includes: Step B1: Divide the frame to be divided into several target regions according to the size of each target region and the preset region division step size.

[0044] In this embodiment, the image is divided into grids according to a specified step size B using a sliding window (there is overlap between different target regions) to obtain several target regions, each target region being labeled as Region(i,j).

[0045] If the current frame is a point cloud frame, the step of determining the size of each target region based on the sensor's resolution and the scene in which the current frame is located includes: Step C1: Determine the size of the corresponding fixed voxel based on the sensor's resolution, the corresponding field of view depth range, and the average depth of the current frame; Step C2: Project the point cloud frame onto a fixed voxel of the specified size to obtain the number of aggregated points and the corresponding first boundary range of the first voxel unit of the fixed voxel. Step C3: Based on the number of aggregation points, if it is determined that the first voxel unit needs to be further divided, the first voxel unit is further divided until the number of aggregation points of the first voxel unit that needs to be further divided meets the preset requirements, the division is determined to be complete, and the second voxel unit and the corresponding second boundary range are obtained. Step C4: Based on the first voxel unit and its corresponding first boundary range, and the second voxel unit and its corresponding second boundary range, determine the size of each target region.

[0046] Specifically, if the current frame is a point cloud frame, the point cloud frame can be divided as follows: First, determine the resolution of the sensor and the corresponding field of view depth range. For example, if the accuracy of the lidar is set to σr=0.02m, the field of view depth range Dmax = 100m. The dimensions of the corresponding fixed voxel are then set as follows:

[0047] Where Dview is the average depth of the current frame. Specifically, for example: if the current scene is a close-up scene (Dview = 5m), then lv = 0.0125m; if the current scene is a distant scene (Dview = 80m), then lv = 0.05m.

[0048] After determining the size of the fixed voxel, the point cloud frame is projected onto the fixed voxel of the specified size to obtain the number of aggregated points and the corresponding first boundary range of the first voxel unit of the fixed voxel: For example, a point cloud frame is projected onto a 3D voxel mesh (first voxel unit). Each voxel mesh (first voxel unit) has an aggregated point count Nv. If Nv > 50, it is considered a high-density area, requiring fine subdivision of the first voxel unit. Specifically, based on the aggregated point count, if it is determined that the first voxel unit needs further subdivision, it is further subdivided until the aggregated point count of the first voxel unit that needs further subdivision meets a preset requirement (e.g., if the aggregated point count Nv of the first voxel unit is 300, then the aggregated point count of the first voxel unit does not meet the requirement and further subdivision is required. This further subdivision may not be done once, but multiple times, until the aggregated point count of the first voxel unit meets the preset requirement, such as being less than 50). Once the subdivision is complete, a second voxel unit and its corresponding second boundary range are obtained. In this embodiment, the first boundary range refers to the size range and region range of the first voxel unit, and the second boundary range refers to the size range and region range of the second voxel unit.

[0049] In this embodiment, the size of each target region is determined based on the first voxel unit and its corresponding first boundary range, and the second voxel unit and its corresponding second boundary range.

[0050] In addition, in this embodiment, a list of voxel indices is determined and output. This results in each point cloud frame forming a defined number of "small regions," each with a fixed index, boundary range, and data set.

[0051] Step S30: Perform differential calculation on the corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions. In this embodiment, after determining the first target region of the current frame, the second target regions corresponding to other frames adjacent to the current frame are also determined. A difference calculation is then performed on the first and second target regions; that is, a difference calculation is performed on the corresponding target regions between adjacent frames, such as... Figure 2As shown, this is to determine whether the corresponding target area is a redundant area.

[0052] Specifically, if the current frame is frame a2, its corresponding target region is a21, and the adjacent frames of frame a2 are frames a1 and a3, the target region corresponding to frame a1 is a11, and the target region corresponding to frame a3 is a31, then a21 and a11 can be differentially calculated to determine whether the corresponding target regions are redundant regions, or a21 and a31 can be differentially calculated to determine whether the corresponding target regions are redundant regions.

[0053] In this embodiment, in order to ensure that the determination of redundant regions is stable rather than random, it can be further determined whether there are consecutive frames. If the corresponding target regions in three consecutive frames all meet the corresponding conditions (all are identified as redundant regions, that is, the difference between a21 and a11 meets the condition, and the difference between a21 and a31 meets the condition), then the corresponding regions a11, a21 and a31 are marked as redundant regions.

[0054] Step S40: Remove or compress the corresponding target regions that are redundant regions.

[0055] In this embodiment, after determining the redundant region, the region identified as redundant can be processed in the following two ways: First, compression processing: Lossy compression is performed on redundant areas to retain key information; Second, discarding: Directly discard data in redundant areas to reduce storage and transmission overhead.

[0056] In this embodiment, after processing the redundant areas, the autonomous driving system outputs the processed data to the on-board storage system or uploads it to the cloud server for subsequent algorithm training, path planning, or decision analysis.

[0057] Compared to related technologies where redundant data between consecutive frames is not processed, which reduces data processing efficiency and real-time performance, this application obtains the current frame acquired by the sensor and preprocesses it to obtain the frame to be divided. The current frame includes an image frame or a point cloud frame. The frame to be divided is then divided into several target regions. Differential calculation is performed on the corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions. The corresponding target regions that are redundant are then removed or compressed. In other words, this application accurately determines the target regions with corresponding redundancy between consecutive frames based on differential calculation, and removes or compresses the corresponding redundant regions, ensuring that subsequent data processing does not suffer from low processing efficiency or reduced real-time performance due to redundant regions.

[0058] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 The step of performing differential calculation on corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions includes any one of the following: Step S31: Perform differential calculation on the corresponding target regions between adjacent frames to determine whether the corresponding differential value is less than a preset threshold. If it is determined to be less than the preset threshold, then determine that the corresponding target region is a redundant region. The preset threshold is dynamic and is associated with a base threshold and a dynamic adjustment term. The dynamic adjustment term is associated with speed and preset environmental complexity, respectively. In this embodiment, a dynamic redundancy threshold, i.e., a preset threshold (or redundancy threshold, such as...), is set based on factors such as vehicle driving status and environmental complexity. Figure 2 As shown in the figure, when the difference value of the corresponding target regions between adjacent frames is lower than the threshold, it is determined to be a redundant region. In this embodiment, the determination of the redundancy threshold is composed of a basic threshold and a dynamic adjustment term: If the current frame is an image frame, the preset threshold T0 is the image SAD (average grayscale difference) = 12. If the current frame is a point cloud frame, the preset threshold is the average translation distance after registration. ; In this embodiment, it should be noted that the dynamic adjustment term increases gradually according to speed and environmental complexity. For example, if the vehicle is at low speed and in a simple environment, the preset threshold decreases to 0.9*T0. If the vehicle is at high speed and in a complex environment, the preset threshold increases to 1.2T0.

[0059] Step S32: Perform differential calculation on the corresponding target regions between adjacent frames to determine the degree of change of the corresponding target regions, and determine the corresponding target regions as redundant regions based on the degree of change.

[0060] In this embodiment, the corresponding target area is also determined to be a redundant area based on the degree of change of the corresponding target area.

[0061] The degree of change can be variable or a fixed value.

[0062] When the current frame is an image frame, the step of performing difference calculation on corresponding target regions between adjacent frames to determine the degree of change of the corresponding target regions includes: Step S321: Determine the average grayscale difference between corresponding target regions between adjacent frames, and determine the structural similarity index between corresponding target regions between adjacent frames, wherein the structural similarity index is related to the brightness, brightness variance, and covariance of the corresponding target regions. Step S322: Determine the degree of change of the corresponding target region based on the average gray level difference and the structural similarity index.

[0063] In this embodiment, the average grayscale difference between corresponding target regions in adjacent frames can first be determined. Specifically, let the target regions corresponding to two adjacent frames be Rt(i,j) and Rt-1(i,j), then the average grayscale difference SAD(i,j) is calculated as follows:

[0064] Where N is the number of pixels in the block, and p is the pixel value of each pixel; in this embodiment, a structural similarity index between corresponding target regions in adjacent frames is also determined, wherein the structural similarity index is related to the brightness, brightness variance, and covariance of the corresponding target region; specifically, the structural similarity index SSIM(i,j) is:

[0065] in , The average brightness of this region in the current frame and the adjacent previous frame; , This represents the luminance variance between the current frame and the adjacent previous frame. This is the covariance between the current frame and the adjacent previous frame; A small constant is added to prevent the denominator from being zero (generally taken as...). (where L is the maximum pixel value, 255). In this embodiment, the degree of change of the corresponding target region is determined based on the average grayscale difference and the structural similarity index. That is, when SAD(i,j) < 12 (accounting for about 5% when the grayscale range is 0~255) and SSIM(i,j) ≥ 0.85, the corresponding target region is determined to be unchanging (a redundant region); otherwise, it is a changing region (not a redundant region). In this embodiment, a further stability check can be performed: that is, if the corresponding target region in 3 consecutive frames meets the above conditions (the average grayscale difference and the structural similarity index meet the above conditions), it is marked as a redundant region.

[0066] When the current frame is a point cloud frame, the step of performing difference calculations on corresponding target regions between adjacent frames to determine the degree of change of the corresponding target regions includes: Step S323: Perform ICP registration on the point clouds in the voxel units of the corresponding target regions between adjacent frames, and calculate the average translation distance after registration; Step S324: Determine the density change rate of the corresponding target region between adjacent frames; Step S325: Based on the average translation distance and density change rate, determine the degree of change of the corresponding target area.

[0067] In this embodiment, ICP registration is performed on the point clouds of the corresponding voxel units of the target region between adjacent frames, and the average translation distance after registration is calculated, that is, for the corresponding voxel units of the target region. Perform ICP registration on the midpoint cloud and calculate the average translation distance after registration. for:

[0068] Where M is the number of points and T is the ICP transformation matrix; In this embodiment, the density change rate of the corresponding target region between adjacent frames is also determined:

[0069] Based on the average translation distance and density change rate, the degree of change in the corresponding target area is determined. Specifically, if... and If the target region is found to be either redundant or refined, it is considered a variable region. In this embodiment, a further stability check can be performed: if the corresponding target region in three consecutive frames meets the above conditions (average translation distance and density change rate meet the above conditions), it is marked as a redundant region.

[0070] In this embodiment, differential calculations are performed on corresponding target regions between adjacent frames to determine whether the corresponding difference value is less than a preset threshold. If it is determined to be less than the preset threshold, the corresponding target region is identified as a redundant region. The preset threshold is dynamic and associated with a base threshold and a dynamic adjustment term, which are respectively associated with speed and a preset environmental complexity. Differential calculations are also performed on corresponding target regions between adjacent frames to determine the degree of change in the corresponding target regions. Based on this degree of change, the corresponding target regions are identified as redundant regions. In this embodiment, accurate differential calculations are performed on corresponding target regions between adjacent frames, laying the foundation for accurately determining whether the corresponding target regions are redundant regions.

[0071] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the data redundancy processing of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0072] This application also provides a data redundancy processing device, please refer to... Figure 5 The data redundancy processing device includes: The acquisition module 10 is used to acquire the current frame collected by the sensor and preprocess the current frame to obtain the frame to be divided. The current frame includes an image frame or a point cloud frame. The segmentation module 20 is used to divide the frame to be segmented into several target regions; The determination module 30 is used to perform differential calculation on the corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions. The processing module 40 is used to remove or compress the corresponding target area that is a redundant area.

[0073] In one embodiment, the data redundancy processing device is used to implement: The size of each target region is determined based on the resolution of the sensor and the scene in the current frame; The frame to be divided is divided into several target regions based on the size of each target region.

[0074] In one embodiment, if the current frame is an image frame, the data redundancy processing device is used to: determine the corresponding image resolution size of the image frame and the current vehicle speed; The size of each target area is determined based on the resolution and the current vehicle speed; The step of dividing the frame to be divided into several target regions according to the size of each target region includes: The frame to be divided is divided into several target regions based on the size of each target region and the preset region division step size.

[0075] In one embodiment, if the current frame is a point cloud frame, the data redundancy processing device is used to implement: The size of the corresponding fixed voxel is determined based on the sensor's resolution, the corresponding field of view depth range, and the average depth of the current frame. Projecting the point cloud frame onto a fixed voxel of the specified size yields the number of aggregated points and the corresponding first boundary range of the first voxel unit of the fixed voxel. Based on the number of aggregation points, if it is determined that the first voxel unit needs to be further divided, the first voxel unit is further divided until the number of aggregation points of the first voxel unit that needs to be further divided meets the preset requirements, the division is determined to be complete, and the second voxel unit and the corresponding second boundary range are obtained. Based on the first voxel unit and its corresponding first boundary range, and the second voxel unit and its corresponding second boundary range, the size of each target region is determined.

[0076] In one embodiment, the data redundancy processing device is used to implement any of the following: For the corresponding target regions between adjacent frames, a difference calculation is performed to determine whether the corresponding difference value is less than a preset threshold. If it is determined to be less than the preset threshold, the corresponding target region is determined to be a redundant region. The preset threshold is dynamic and is associated with a base threshold and a dynamic adjustment term. The dynamic adjustment term is associated with speed and preset environmental complexity, respectively. For the corresponding target regions between adjacent frames, differential calculation is performed to determine the degree of change of the corresponding target regions, and the corresponding target regions are determined to be redundant regions based on the degree of change.

[0077] In one embodiment, the data redundancy processing device is used to implement: The average grayscale difference between corresponding target regions between adjacent frames is determined, and the structural similarity index between corresponding target regions between adjacent frames is determined, wherein the structural similarity index is related to the brightness, brightness variance, and covariance of the corresponding target regions. The degree of change in the corresponding target region is determined based on the average grayscale difference and the structural similarity index.

[0078] In one embodiment, the data redundancy processing device is used to implement: ICP registration is performed on the point clouds of the corresponding target regions in adjacent frames, and the average translation distance after registration is calculated. Determine the density change rate of the corresponding target region between adjacent frames; Based on the average translation distance and density change rate, the degree of change in the corresponding target area is determined.

[0079] The data redundancy processing apparatus provided in this application, employing the data redundancy processing method described in the above embodiments, can solve the technical problems of data redundancy processing apparatuses. Compared with the prior art, the beneficial effects of the data redundancy processing apparatus provided in this application are the same as those of the data redundancy processing method described in the above embodiments, and other technical features in the data redundancy processing apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0080] This application provides a data redundancy processing apparatus, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the data redundancy processing method in Embodiment 1 above.

[0081] The following is for reference. Figure 6 The diagram illustrates a structural schematic of a data redundancy processing device suitable for implementing embodiments of this application. The data redundancy processing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, tablets, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The data redundancy processing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0082] like Figure 6As shown, the data redundancy processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the data redundancy processing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the data redundancy processing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows data redundancy processing devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0083] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0084] The data redundancy processing device provided in this application, employing the data redundancy processing method described in the above embodiments, can solve the technical problems. Compared with the prior art, the beneficial effects of the data redundancy processing device provided in this application are the same as those of the data redundancy processing method described in the above embodiments, and other technical features of this data redundancy processing device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0085] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0086] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0087] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the data redundancy processing method in the above embodiments.

[0088] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0089] The aforementioned computer-readable storage medium may be included in the data redundancy processing device; or it may exist independently and not be assembled into the data redundancy processing device.

[0090] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the data redundancy processing device, the data redundancy processing device: collects access information from different application sources through an access collection program and stores the access information in a message queue, wherein the access collection program is deployed in the runtime environment of the application; performs service call dependency deduction on the access information in the message queue to obtain the call relationship between the different applications, and generates an application topology architecture based on the call relationship.

[0091] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0093] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0094] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described data redundancy processing method, thereby solving the technical problem of data redundancy processing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the data redundancy processing method provided in the above embodiments, and will not be repeated here.

[0095] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data redundancy processing method described above.

[0096] The computer program product provided in this application can solve the technical problem of data redundancy processing. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the data redundancy processing method provided in the above embodiments, and will not be repeated here.

[0097] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A data redundancy processing method, characterized by, The data redundancy processing method comprises: Obtaining a current frame collected by a sensor, and pre-processing the current frame to obtain a frame to be divided, wherein the current frame comprises an image frame or a point cloud frame; Dividing the frame to be divided into a plurality of target regions; Performing difference calculation on the target regions corresponding between adjacent frames to determine whether the corresponding target regions are redundant regions; Performing removal or compression processing on the corresponding target regions which are redundant regions.

2. The data redundancy processing method of claim 1, wherein, The step of dividing the frame to be divided into a plurality of target regions comprises: Determining the size of each target region according to the resolution of the sensor and the scene in which the current frame is located; Dividing the frame to be divided into a plurality of target regions according to the size of each target region.

3. The data redundancy processing method of claim 2, wherein, If the current frame is an image frame, the step of determining the size of each target region according to the resolution of the sensor and the scene in which the current frame is located comprises: Determining the image resolution size corresponding to the image frame and the current vehicle speed; Determining the size of each target region according to the resolution size and the current vehicle speed; The step of dividing the frame to be divided into a plurality of target regions according to the size of each target region comprises: Dividing the frame to be divided into a plurality of target regions according to the size of each target region and a preset region division step length.

4. The data redundancy processing method of claim 2, wherein, If the current frame is a point cloud frame, the step of determining the size of each target region according to the resolution of the sensor and the scene in which the current frame is located comprises: Determining the size of a corresponding fixed voxel according to the resolution of the sensor, the corresponding field of view depth range, and the average depth of the current frame; Projecting the point cloud frame to the fixed voxel of the size to obtain the aggregated point number of a corresponding first voxel unit of the fixed voxel and a corresponding first boundary range; Based on the aggregated point number, if it is determined that the first voxel unit needs to be further divided, the first voxel unit is further divided until the aggregated point number of the first voxel unit that needs to be further divided meets a preset requirement, the division is determined to be completed, and a second voxel unit and a corresponding second boundary range are obtained; Based on the first voxel unit and the corresponding first boundary range, the second voxel unit and the corresponding second boundary range, the size of each target region is determined.

5. The data redundancy processing method of claim 1, wherein, The step of performing difference calculation on the target regions corresponding between adjacent frames to determine whether the corresponding target regions are redundant regions comprises any one of the following: Performing difference calculation on the target regions corresponding between adjacent frames to determine whether the corresponding difference value is less than a preset threshold value, and if it is determined that the corresponding difference value is less than the preset threshold value, it is determined that the corresponding target region is a redundant region, wherein the preset threshold value is dynamic and is associated with a basic threshold value and a dynamic adjustment term, and the dynamic adjustment term is respectively associated with a speed and a preset environmental complexity; Performing difference calculation on the target regions corresponding between adjacent frames to determine the degree of change of the corresponding target regions, and determining that the corresponding target regions are redundant regions according to the degree of change.

6. The data redundancy processing method of claim 5, wherein, When the current frame is an image frame, the step of performing difference calculation on the target regions corresponding between adjacent frames to determine the degree of change of the corresponding target regions comprises: determine an average gray difference between the corresponding target regions between adjacent frames, and determine a structural similarity index between the corresponding target regions between adjacent frames, wherein the structural similarity index is associated with brightness, brightness variance and covariance of the corresponding target regions; determine the degree of change of the corresponding target regions according to the average gray difference and the structural similarity index.

7. The data redundancy processing method of claim 5, wherein, In the case that the current frame is a point cloud frame, the step of performing difference calculation on the corresponding target regions between adjacent frames to determine the degree of change of the corresponding target regions comprises: performing ICP registration on the point cloud in the voxel unit of the corresponding target regions between adjacent frames, and calculating an average translation distance after registration; determining a density change rate of the corresponding target regions between adjacent frames; determining the degree of change of the corresponding target regions based on the average translation distance and the density change rate.

8. A data redundancy processing apparatus characterized by comprising: The data redundancy processing device comprises: an acquisition module configured to acquire a current frame collected by a sensor and to pre-process the current frame to obtain a frame to be divided, wherein the current frame comprises an image frame or a point cloud frame; a division module configured to divide the frame to be divided into a plurality of target regions; a determination module configured to perform difference calculation on the corresponding target regions between adjacent frames to determine whether the corresponding target regions are redundant regions; a processing module configured to remove or compress the corresponding target regions which are the redundant regions.

9. A data redundancy processing device, characterized by comprising: The device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the data redundancy processing method according to any one of claims 1 to 7.

10. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the data redundancy processing method according to any one of claims 1 to 7.

Images