Real-time scene distance field construction method and device and robot

Abstract

The invention relates to the technical field of computer processing, in particular to a real-time scene distance field construction method and device and a robot. The method comprises the following steps: generating a plurality of Gaussian ellipsoid models according to acquired sparse point clouds of a scene environment, and generating an initial truncated symbol distance field according to the plurality of Gaussian ellipsoid models; filtering the initial truncated symbol distance field according to motion consistency information between Gaussian ellipsoid models corresponding to continuous multi-frame sparse point clouds to obtain a static truncated symbol distance field; and calculating the Euclidean symbol distance field of the scene environment according to a layering strategy along the gradient direction of the static truncated symbol distance field. According to the invention, real-time, low-consumption and high-robustness scene distance field construction capability can be provided.

Description

Technical Field

[0001] This application relates to the field of computer processing technology, and in particular to a method, apparatus and robot for constructing a real-time scene distance field. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) systems are used to enable robots to simultaneously localize themselves and build an environment map while moving in unknown target environments. Traditional SLAM mapping methods face three limitations: (1) Voxelized TSDF (Truncated Signed Distance Field) construction leads to excessive memory burden and boundary distortion; (2) Dynamic object interference damages the map. Figure 1 Consistency; (3) ESDF (Euclidean Signed Distance Field) relies on iterative diffusion algorithm, which has low computational efficiency and is difficult to meet the real-time navigation requirements. The existing 3D Gaussian splashing technology has not yet solved the bottleneck of scene geometric distance field generation. Summary of the Invention

[0003] In view of this, embodiments of this application provide a real-time scene distance field construction method, apparatus and robot, which can provide real-time, low-power and robust scene distance field construction capabilities.

[0004] In a first aspect, embodiments of this application provide a method for constructing a real-time scene distance field, including: Multiple Gaussian ellipsoid models are generated based on the sparse point cloud of the collected scene environment, and an initial truncated symbolic distance field is generated based on the multiple Gaussian ellipsoid models. Based on the motion consistency information between the various Gaussian ellipsoid models corresponding to the sparse point cloud in multiple consecutive frames, the initial truncated symbolic distance field is filtered to obtain the static truncated symbolic distance field. Along the gradient direction of the statically truncated symbolic distance field, the Euclidean symbolic distance field of the scene environment is calculated according to a hierarchical strategy.

[0005] In some embodiments, calculating the Euclidean symbolic distance field of the scene environment along the gradient direction of the statically truncated symbolic distance field according to a hierarchical strategy includes: The scene environment space corresponding to the static truncation symbol distance field is divided into multiple hierarchical regions; Along the gradient direction of each of the said hierarchical regions, the Euclidean symbol distance field block of the corresponding hierarchical region is calculated by integration. By fusing all the aforementioned Euclidean symbolic distance field blocks, the Euclidean symbolic distance field is obtained.

[0006] In some embodiments, the step of integrating along the gradient direction of each of the hierarchical regions to calculate the Euclidean symbol distance field block of the corresponding hierarchical region includes: Gradient estimation is performed in parallel for each of the said hierarchical regions to generate the corresponding gradient direction, and integration is performed from each point in each of the said hierarchical regions along the gradient direction to generate the shortest path from each point to the surface of the target object, and the corresponding Euclidean sign distance field block is obtained based on each of the said shortest paths.

[0007] In some embodiments, generating multiple Gaussian ellipsoid models based on the collected sparse point cloud of the scene environment, and generating an initial truncated symbolic distance field based on the multiple Gaussian ellipsoid models, includes: Based on the sparse point cloud of the collected scene environment, construct a Gaussian ellipsoid model for each point in the sparse point cloud; A continuous density field is generated based on each of the Gaussian ellipsoid models; Using the gradient direction of the continuous density field, the probability occupancy value of any point in the scene environment space in the Gaussian ellipsoid model is calculated. The basic truncated symbol distance field is obtained by mapping each of the aforementioned probability occupancy values; The truncation distance of the basic truncation symbol distance field is dynamically adjusted using a truncation distance dynamic adjustment mechanism to obtain the initial truncation symbol distance field; wherein, the truncation distance dynamic adjustment mechanism is determined based on the local curvature characteristics of the object surface.

[0008] In some embodiments, the sparse point cloud includes the three-dimensional coordinates, normal vector, covariance information, and local point cloud density of each point; the parameters of the Gaussian ellipsoid model include position, orientation, and anisotropic covariance matrix. The step of constructing a Gaussian ellipsoid model for each point in the sparse point cloud based on the collected scene environment includes: The position of the corresponding Gaussian ellipsoid model is determined based on the three-dimensional coordinates of each point. The orientation of the corresponding Gaussian ellipsoid model is determined based on the normal vector of each point; The anisotropic covariance matrix of the Gaussian ellipsoid model is obtained by adaptively adjusting the local point cloud density of the sparse point cloud.

[0009] In some embodiments, the step of adaptively adjusting the anisotropic covariance matrix of the Gaussian ellipsoid model based on the local point cloud density of the sparse point cloud includes: The corresponding local point cloud density is calculated by the number of neighboring points or local geometric features of each point in the sparse point cloud. The scale parameter is determined based on the local point cloud density using a preset inverse proportional strategy, and the scale covariance matrix is ​​determined based on the scale parameter. The anisotropic covariance matrix is ​​constructed based on the scale covariance matrix and the rotation matrix calculated based on the corresponding points.

[0010] In some embodiments, generating a continuous density field based on each Gaussian ellipsoid model includes: The probability occupancy value calculation functions of each Gaussian ellipsoid model are weighted and superimposed in the scene environment space to generate a continuous Gaussian density field.

[0011] In some embodiments, the motion consistency information includes dynamic probability weights; The step of filtering the initial truncated symbolic distance field based on the motion consistency information between the various Gaussian ellipsoid models corresponding to multiple consecutive frames of the sparse point cloud to obtain the static truncated symbolic distance field includes: Based on the multiple Gaussian ellipsoid models and the obtained dynamic probability weights and camera pose transformation matrix, the initial truncated symbolic distance field is filtered to obtain the static truncated symbolic distance field.

[0012] In some embodiments, filtering the initial truncated symbolic range field based on the plurality of Gaussian ellipsoid models and the obtained dynamic probability weights and camera pose transformation matrix to obtain the static truncated symbolic range field includes: The theoretical position of each Gaussian ellipsoid model in the current frame is predicted based on the camera pose transformation matrix output by the synchronous localization and mapping system, thus obtaining the predicted position; wherein, the synchronous localization and mapping system is used to acquire sparse point clouds of the scene environment; For each Gaussian ellipsoid model, if the deviation between the actual position and the predicted position of the Gaussian ellipsoid model exceeds a set deviation threshold, the corresponding Gaussian ellipsoid model is marked as a dynamic unit. The dynamic probability weights are calculated based on the dynamic probability weight function, and the transparency of each dynamic unit in the initial truncated symbol distance field is attenuated to obtain the static truncated symbol distance field.

[0013] Secondly, embodiments of this application provide an Euclidean symbolic distance field generation apparatus, comprising: The truncated symbolic distance field generation module is used to generate multiple Gaussian ellipsoid models based on the sparse point cloud of the collected scene environment, and to generate an initial truncated symbolic distance field based on the multiple Gaussian ellipsoid models. The filtering module is used to filter the initial truncated symbolic distance field based on the motion consistency information between the various Gaussian ellipsoid models corresponding to the sparse point cloud in multiple consecutive frames, so as to obtain the static truncated symbolic distance field. The Euclidean symbolic distance field generation module is used to calculate the Euclidean symbolic distance field of the scene environment along the gradient direction of the statically truncated symbolic distance field according to a hierarchical strategy.

[0014] Thirdly, embodiments of this application provide a robot, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement a real-time scene distance field construction method provided in the first aspect of this application.

[0015] The embodiments of this application have the following beneficial effects: This application generates multiple Gaussian ellipsoid models from the collected sparse point cloud of the scene environment, and generates an initial truncated symbolic distance field based on these models. The initial truncated symbolic distance field is then filtered based on the motion consistency information between the Gaussian ellipsoid models corresponding to multiple consecutive frames of sparse point cloud data to obtain a static truncated symbolic distance field. The Euclidean symbolic distance field of the scene environment is calculated along the gradient direction of the static truncated symbolic distance field using a hierarchical strategy. This application utilizes a dynamic filtering mechanism based on the Gaussian ellipsoid models corresponding to multiple consecutive frames of sparse point cloud data and their motion consistency information, effectively eliminating dynamic target interference and avoiding accidental deletion of static features. The hierarchical calculation strategy improves the computation speed, and the differential mapping of the Euclidean symbolic distance field further enhances the generation speed of the Euclidean symbolic distance field. Ultimately, this provides a lightweight scene distance field construction capability with centimeter-level accuracy and millisecond-level response. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a real-time scene distance field construction method according to an embodiment of this application is shown; Figure 2 This paper illustrates a flowchart of an initial truncated symbol distance field generation method in the real-time scene distance field construction method of this application embodiment; Figure 3 This paper illustrates a flowchart of a Gaussian ellipsoid model generation method in a real-time scene distance field construction method according to an embodiment of this application. Figure 4 This paper illustrates a flowchart of a static truncated symbol distance field generation method in the real-time scene distance field construction method of this application embodiment; Figure 5 Another flowchart of the real-time scene distance field construction method according to an embodiment of this application is shown; Figure 6 A schematic diagram of a structure of an Euclidean symbol distance field generation device according to an embodiment of this application is shown.

[0018] Explanation of key component symbols: 610 - Truncated symbolic distance field generation module; 620 - Filtering module; 630 - Euclidean symbolic distance field generation module. Detailed Implementation

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0020] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] In the following text, the terms "comprising," "having," and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof. Furthermore, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0022] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0023] The following explains some of the key technical terms used in this application: Sparse point cloud: Represents a scene with sparse geometric information, without voxelization processing, small data volume, and loose structure.

[0024] The Gaussian ellipsoid model (or simply Gaussian ellipsoid) is a spatial distribution model that represents a three-dimensional Gaussian distribution with directionality (anisotropy).

[0025] Truncated Signed Distance Field (TSDF) is an implicit representation method for 3D scene reconstruction. It assigns a value to each point in 3D space, representing the signed distance of that point from the object's surface. Specifically, the TSDF assigns a value to each point in the scene environment space, representing the signed distance of that point from the target object's surface. A positive value indicates the point is outside the object; a negative value indicates the point is inside the object; and a zero value indicates the point is on the object's surface. The truncation mechanism retains only valid values ​​near the object's surface; values ​​outside this range are truncated or ignored.

[0026] The Euclidean Signed Distance Field (ESDF) is a special type of SDF (Signed Distance Field) whose value represents the Euclidean distance (i.e., the shortest straight-line distance) from a point to the nearest object surface, and is signed. A positive value indicates the point is outside the object; a negative value indicates the point is inside the object; and zero indicates the point is on the object's surface. Unlike the TSDF, the ESDF value is the "true distance," not a truncated local distance.

[0027] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] This application provides an exemplary robot, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program, thereby enabling the robot to perform the functions of various modules in the real-time scene distance field construction method or Euclidean symbolic distance field generation device provided in this application. Furthermore, the robot can utilize the generated Euclidean symbolic distance field to achieve path planning, obstacle avoidance, or 3D reconstruction, etc. This robot is an example of a robot incorporating a SLAM system.

[0029] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0030] Memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). Memory is used to store computer programs, and the processor can execute these programs upon receiving execution instructions.

[0031] The following describes the method for constructing the distance field of this real-time scene using specific examples.

[0032] Figure 1 A flowchart illustrating a real-time scene distance field construction method according to an embodiment of this application is shown. Exemplarily, the real-time scene distance field construction method includes the following steps: S100 generates multiple Gaussian ellipsoid models based on the sparse point cloud of the collected scene environment, and generates an initial truncated symbolic distance field based on the multiple Gaussian ellipsoid models.

[0033] Exemplary, this embodiment generates multiple Gaussian ellipsoid models by utilizing the sparse point cloud of the scene environment, and then uses the associated Gaussian ellipsoid models in combination with the local curvature characteristics of the object surface to generate an initial truncated symbolic distance field.

[0034] S200: Based on the motion consistency information between the various Gaussian ellipsoid models corresponding to multiple consecutive frames of sparse point clouds, the initial truncated symbolic range field is filtered to obtain the static truncated symbolic range field.

[0035] Continuous multi-frame sparse point clouds refer to the Gaussian ellipsoidal models generated from the sparse point clouds of the current frame and the sparse point clouds of the previous adjacent frames, collected from the scene environment. These continuous multi-frame sparse point clouds generate corresponding Gaussian ellipsoidal models for subsequent filtering of the initial truncated symbolic range field to eliminate dynamic interference.

[0036] The Gaussian ellipsoid model corresponding to multiple consecutive frames of sparse point clouds is obtained in the following way: The first step is to generate a corresponding Gaussian ellipsoid model based on the single-frame sparse point cloud collected from the scene environment. Then, the continuous frame sparse point cloud is obtained through the synchronous localization and mapping system. The operation of generating the Gaussian ellipsoid model in the first step is repeated for each frame of sparse point cloud to obtain the Gaussian ellipsoid model corresponding to the continuous frame sparse point cloud.

[0037] In real-world scenarios, TSDF reconstruction often faces the problem of interference from dynamic objects (such as pedestrians, vehicles, and moving furniture). Traditional processing methods typically directly remove dynamic points, which may mistakenly delete static points (such as occluded or temporarily occluded static objects). Furthermore, dynamic removal strategies lack continuity and are susceptible to noise interference. Therefore, this application employs dynamic consistency filtering. The displacement deviation of a Gaussian ellipsoid is used to detect dynamic targets, resulting in dynamic elements. Filtering is then performed based on these dynamic elements, and since the direction is consistent with the surface, the probability of misjudgment is lower.

[0038] By way of example, the embodiments of this application filter out dynamic objects by performing motion consistency verification on the center position of the Gaussian ellipsoid in consecutive frames. This method does not rely on traditional detection methods based on appearance or color changes and has higher geometric consistency and robustness.

[0039] S300 calculates the Euclidean symbolic distance field of the scene environment according to a layered strategy, along the gradient direction of the statically truncated symbolic distance field.

[0040] In this embodiment, the Euclidean symbolic distance field is generated based on the spatial differential properties of the statically truncated symbolic distance field. The spatial differential properties include the gradient direction of the statically truncated symbolic distance field. Furthermore, to improve computational speed, this embodiment employs a hierarchical computation strategy.

[0041] In one implementation, such as Figure 2 As shown, step S100 includes: S110: Based on the collected sparse point cloud of the scene environment, construct a Gaussian ellipsoid model for each point in the sparse point cloud.

[0042] As an example, a sparse point cloud is obtained by scanning the scene environment through a simultaneous localization and mapping system, and initialized as an anisotropic Gaussian ellipsoid (also known as a Gaussian ellipsoid model). The sparse point cloud includes the three-dimensional coordinates of each point, the normal vector (used to estimate the surface orientation), covariance information, and local point cloud density (local density information); anisotropy refers to the different degrees of expansion in different directions, which is specifically controlled by the covariance matrix.

[0043] In this embodiment, each Gaussian ellipsoid model corresponds to a point in the sparse point cloud output by the SLAM system. In other words, each point in the sparse point cloud is initialized as an independent anisotropic Gaussian ellipsoid. For example, if the SLAM system outputs 1000 sparse points, then in this embodiment, 1000 Gaussian ellipsoid models will be initialized.

[0044] The parameters of the Gaussian ellipsoid model include position, orientation, and anisotropic covariance matrix, for example, such as... Figure 3 As shown, in step S110, based on the collected sparse point cloud of the scene environment, a Gaussian ellipsoid model of each point in the sparse point cloud is constructed, including: S111, determine the position of the corresponding Gaussian ellipsoid model based on the three-dimensional coordinates of each point.

[0045] The position of each Gaussian ellipsoid model corresponds to the three-dimensional coordinates of its corresponding point. That is, the center of each Gaussian ellipsoid model is located at the three-dimensional coordinates of its corresponding point. The three-dimensional coordinates of each point represent the center position of its corresponding Gaussian ellipsoid model. In other words, the position is determined by the three-dimensional coordinates of each point in the point cloud.

[0046] S112, determine the orientation of the corresponding Gaussian ellipsoid model based on the normal vector of each point.

[0047] The normal vector of each point is obtained as follows: For each point, a certain number of neighboring points are selected (e.g., using k-nearest neighbors or radius search) to form a local neighborhood set; PCA analysis is performed on the neighboring points to calculate the covariance matrix; eigenvalue decomposition is performed on the covariance matrix to obtain three eigenvalues ​​(usually sorted by size) and their corresponding eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the direction of the normal vector of that point.

[0048] Each Gaussian ellipsoid model has an orientation determined by a rotation matrix calculated from the normal vector of a point. In other words, the normal vector determines the alignment of the principal axis of the Gaussian ellipsoid with the normal vector of the object's surface; each Gaussian ellipsoid model has a unique rotation matrix, i.e., a unique orientation. In other words, this orientation is determined by the rotation matrix calculated from the normal vector of that point.

[0049] S113, the anisotropic covariance matrix of the Gaussian ellipsoid model is obtained by adaptively adjusting the local point cloud density of the sparse point cloud.

[0050] Local point cloud density refers to the number of points in the neighborhood of a given point in the sparse point cloud output by SLAM, and is used to assess the spatial geometric density of the region.

[0051] Each Gaussian ellipsoid model has three scale parameters (along the three principal axes), which constitute the diagonal elements of the scale covariance matrix. Since this application employs anisotropic modeling, the scale size can differ in different directions. The specific scale size is adaptively adjusted by the local point cloud density and expressed through the scale covariance matrix. Understandably, the anisotropic covariance matrix is ​​obtained from the scale covariance matrix. The anisotropic covariance matrix is ​​used to complete the modeling of the Gaussian ellipsoid model and is the final covariance matrix used for modeling, representing a Gaussian distribution with directional and scale characteristics.

[0052] Furthermore, in order to improve the stability and robustness of this application under occlusion, long distance, and dynamic regions, the embodiments of this application do not use a Gaussian distribution with fixed parameters, but adaptively adjust the covariance matrix according to the local point cloud density.

[0053] In one implementation, step S113, obtaining the anisotropic covariance matrix of the Gaussian ellipsoid model based on adaptive adjustment of the local point cloud density of the sparse point cloud, includes: S1131, the corresponding local point cloud density is calculated by the number of neighboring points or local geometric features corresponding to each point in the sparse point cloud.

[0054] As an example, the following method is used to evaluate local point cloud density in order to dynamically adjust the scale of the Gaussian ellipsoid model based on the local point cloud density: For each point in a sparse point cloud, count the number of points in its neighborhood (e.g., within a radius r); or use geometric features such as local curvature and distance between points to estimate the density; the higher the density, the more stable the structure of the region, and the more compact the Gaussian distribution should be (i.e., the smaller the scale).

[0055] S1132, determine the scale parameter based on the local point cloud density using a preset inverse proportional strategy, and determine the scale covariance matrix based on the scale parameter.

[0056] An inverse scaling strategy is employed to map the local point cloud density values ​​to a scaling factor. The initial covariance matrix is ​​set as the identity matrix (representing a spherical distribution); the scaling factor is used to scale the diagonal elements within the identity matrix (by substituting the scaling parameter into the diagonal positions) to control the ellipsoid size.

[0057] In addition, inverse proportion strategies include, but are not limited to, those based on simple linear inverse proportion functions, threshold-controlled inverse proportion functions, and exponential inverse proportion functions.

[0058] In this embodiment, local point cloud density is used to dynamically adjust the scale factor of the Gaussian ellipsoid model. The higher the density, the smaller the scale (an inverse proportional strategy). That is, denser points indicate a more stable structure in the region, and the Gaussian distribution should be more "compact"; denser point clouds result in smaller scale factors (more compact Gaussian distribution); sparser point clouds result in larger scale factors (more diffuse Gaussian distribution). This allows for more accurate modeling of object surfaces, avoiding over-smoothing in sparse areas and distortion in dense areas.

[0059] S1133, construct the anisotropic covariance matrix based on the scale covariance matrix and the rotation matrix calculated based on the corresponding points.

[0060] The orientation of the Gaussian ellipsoid model is adjusted by using a rotation matrix calculated based on points (controlling the ellipsoid orientation to be consistent with the normal) to obtain the anisotropic covariance matrix.

[0061] In essence, the anisotropic covariance matrix is ​​a complete covariance matrix formed by combining the scale covariance matrix with a rotation matrix. This complete covariance matrix describes the distribution characteristics in any direction, reflecting not only scale changes but also directionality (rotation), and is suitable for Gaussian ellipsoid models with arbitrary orientations. Furthermore, it supports the subsequent determination of the consistency between the TSDF gradient direction and the normal, improving reconstruction accuracy.

[0062] In order to achieve orientation alignment of the Gaussian ellipsoid model, the standard Gaussian distribution (spherical or axis-aligned ellipsoid) is rotated to a direction consistent with the normal of the object surface, thereby improving the orientation consistency of geometric modeling.

[0063] This application embodiment also calculates a rotation matrix based on the normal vector of a point, wherein the method for calculating the rotation matrix includes: For each point, construct a rotation matrix aligned with it based on its normal vector, specifically including the following steps: (1) Construct an orthogonal basis perpendicular to the normal vector. Construct an orthogonal coordinate system whose z-axis direction is aligned with the normal vector. (2) Define the target z-axis direction. For example, set the normal vector of the point as the z-axis direction of the orthogonal coordinate system. (3) Construct the x-axis direction. For example, construct the x-axis direction perpendicular to the z-axis through cross product. Choose any non-collinear vector and perform cross product with the z-axis direction vector to obtain an orthogonal vector (i.e., the x-axis vector). (4) Obtain the y-axis direction vector by cross product of the z-axis direction vector and the x-axis direction vector; construct a rotation matrix using the x, y, and z-axis direction vectors as column vectors.

[0064] The rotation matrix is ​​used to rotate the standard Gaussian distribution to the direction that is consistent with the surface normal of the object, so as to construct an anisotropic Gaussian ellipsoid.

[0065] S120 generates a continuous density field based on each Gaussian ellipsoid model.

[0066] The continuous density field reflects the probability that any point in the scene environment space is "occupied". In one implementation, step S120, generating a continuous density field based on each Gaussian ellipsoid model, includes: S121, the probability occupancy value calculation function of each Gaussian ellipsoid model is weighted and superimposed in the scene environment space to generate a continuous Gaussian density field.

[0067] Each Gaussian ellipsoid model corresponds to a multivariate probability occupancy value calculation function (also known as a probability density function).

[0068] The probability occupancy calculation function (also known as the probability density function) of each Gaussian ellipsoid model is weighted and superimposed in the scene environment space to generate a continuous Gaussian density field, or simply a continuous density field. The continuous density field in this application is an improvement on existing applications of Gaussian probability density functions. Its core is to combine adaptively parameterized Gaussian ellipsoid models to achieve accurate spatial occupancy description, rather than the simple superposition of traditional fixed parameters. In other words, this application combines the continuous density field with Gaussian ellipsoid models (with parameters adaptively adjusted by local point cloud features), calculates the spatial point probability occupancy value through the function corresponding to the continuous density field, and directly generates the TSDF using the coincidence of gradient and object normal, without the need for the traditional voxelization step. The function corresponding to the continuous density field (called the continuous density field function) is based on the probability occupancy value of each target point to be calculated in the scene environment space and the Gaussian ellipsoid model number. 、The positions of each Gaussian ellipsoid model, the coordinates of the target point, and the exponential function of the values ​​are used to construct the model. As an example, the function corresponding to the continuous density field is:

[0069] in, x This represents any three-dimensional point in the target environment space for which the probability of "whether it is occupied by the target object" is to be calculated (the coordinates of the target point), such as a specific location in the target environment space (like a point with coordinates (1, 2, 3)).

[0070] p ( x ) represents the probability occupancy value, representing x The higher the value, the greater the probability that the point is part of the object's area.

[0071] i This is used to distinguish different Gaussian ellipsoid models, because multiple models are generated based on multiple points. i Just like the "number" of each model, it can correspond to the model generated at a specific point.

[0072] μ i Indicates the first i The center position of a Gaussian ellipsoid model is exactly the same as the three-dimensional coordinates of its corresponding point, which determines the core position of the Gaussian ellipsoid in space.

[0073] The embodiments of this application use a continuous density field function to calculate the occupancy probability of any point in the modeling space.

[0074] S130 uses the gradient direction of the continuous density field to calculate the probability occupancy value of any point in the scene environment space within the Gaussian ellipsoid model.

[0075] By utilizing the gradient direction of the continuous density field, the probability occupancy value of any point in the scene environment space within the Gaussian ellipsoid model is calculated.

[0076] The probability occupancy value at any point in the scene environment is calculated using a function of the continuous density field. By leveraging the property that the gradient direction of this continuous density field coincides with the normal direction of the object's surface, the sign of the TSDF (Transient Tolerance Function) is directly determined (positive if the gradient points outward, negative if it points outward). In other words, the TSDF sign at any point in space is determined based on the gradient direction of the continuous density field. Specifically, the gradient direction of the continuous density field is calculated, and the normal direction of the target object's surface is determined based on the gradient direction; the sign of the probability occupancy value is then determined based on the normal direction.

[0077] The embodiments of this application are based on continuous density field modeling, which does not require spatial discretization, supports queries of arbitrary precision, has good real-time performance, and high computational efficiency, making it suitable for real-time applications such as online SLAM and robot navigation.

[0078] S140, the basic truncated symbol distance field is obtained by mapping the various probability occupancy values.

[0079] As an example, the process involves: obtaining probability occupancy values; calculating signed distances; truncating the distances; mapping the probability values ​​to TSDF values; and constructing a basic truncated signed distance field. The mapping of the probability occupancy values ​​to obtain the truncated signed distance field (TSDF) includes, but is not limited to, methods such as linear mapping and probability fusion strategies.

[0080] S150, using a dynamic adjustment mechanism for the cutoff distance, dynamically adjusts the cutoff distance of the basic cutoff symbol distance field to obtain the initial cutoff symbol distance field; wherein, the dynamic adjustment mechanism for the cutoff distance is determined based on the local curvature characteristics of the object surface.

[0081] The local curvature characteristics of an object's surface include curvature, etc. For example, the cutoff distance of a truncated signed distance field can be dynamically adjusted based on the curvature in the local curvature characteristics of the object's surface and a preset curvature threshold. For example, the dynamic adjustment mechanism of the cutoff distance includes: maintaining a first cutoff range in flat areas (e.g., the first cutoff range is about 15cm to improve modeling efficiency), and automatically shrinking areas larger than the preset curvature threshold to a second cutoff range (the second cutoff range is 5cm) to preserve a more refined geometric structure.

[0082] In one implementation, the motion consistency information includes dynamic probability weights. Understandably, step S200 includes: S210, based on multiple Gaussian ellipsoid models and the obtained dynamic probability weights and camera pose transformation matrix, the initial truncated symbolic range field is filtered to obtain the static truncated symbolic range field.

[0083] To enhance the reliability of each Gaussian ellipsoid model in subsequent processing such as pose optimization and map building, dynamic probability weights are used in this embodiment to progressively suppress and filter dynamic disturbances. This significantly improves the robustness, accuracy, and intelligence of the SLAM system in complex dynamic environments.

[0084] The camera pose transformation matrix is ​​used to predict the theoretical new position of the static Gaussian ellipsoid in the next frame, serving as a benchmark for determining whether it has undergone abnormal motion (i.e., whether it is dynamic).

[0085] Furthermore, such as Figure 4As shown, to avoid accidentally damaging some static objects with inaccurate motion estimation and to improve robustness, in step S210, the initial truncated symbolic range field is filtered based on multiple Gaussian ellipsoid models and the obtained dynamic probability weights and camera pose transformation matrix to obtain the static truncated symbolic range field, including: S211, predict the theoretical position of each Gaussian ellipsoid model in the current frame based on the camera pose transformation matrix output by the synchronous localization and mapping system, and obtain the predicted position; wherein, the synchronous localization and mapping system is used to collect sparse point clouds of the scene environment.

[0086] The theoretical position of the static Gaussian ellipsoid in the current frame is predicted using the camera pose transformation matrix output by the SLAM system.

[0087] S212. For each Gaussian ellipsoid model, if the deviation between the actual position and the predicted position of the Gaussian ellipsoid model exceeds a set deviation threshold, the corresponding Gaussian ellipsoid model is marked as a dynamic unit (also known as a dynamic Gaussian ellipsoid).

[0088] For example, the deviation threshold can be set to 0.2m. The embodiments of this application obtain dynamic elements based on the motion consistency of the Gaussian ellipsoid model.

[0089] S213, the dynamic probability weight is calculated based on the dynamic probability weight function, and the transparency of each dynamic unit in the initial truncated symbol distance field is reduced to obtain the static truncated symbol distance field.

[0090] Based on the generated initial truncated symbol range field, the marked dynamic units are subjected to transparency attenuation processing using a dynamic probability weighting function to filter out dynamic interference and obtain the static truncated symbol range field.

[0091] This application's embodiments reduce the contribution of individual dynamic Gaussian ellipsoid models through a dynamic probability weighting function, ultimately demonstrating the attenuation effect in the initial truncated symbolic distance field after superposition, resulting in a static truncated symbolic distance field. Each Gaussian ellipsoid model represents a spatial point or surface point, and it is necessary to determine whether it belongs to a dynamic object; if it is mistakenly identified as a dynamic point and directly deleted, it will lead to the loss of static structure. The dynamic weighting function quantifies the uncertainty of the object's dynamics, providing a basis for subsequent transparency attenuation and avoiding misjudgment and deletion. Therefore, the transparency attenuation mechanism adopted in this application is a more gentle and robust processing method.

[0092] In this embodiment, dynamic points are only "faded" when they exhibit inconsistent motion across multiple consecutive frames. In other words, this application does not directly delete dynamic units; instead, it weights and attenuates their contribution to the TSDF construction. For example, during TSDF fusion, a dynamic probability weighting function is used to reduce their spatial probability occupancy (i.e., transparency), meaning the contribution of dynamic units to the probability occupancy is multiplied by their corresponding weights. Therefore, this application uses transparency attenuation instead of "direct deletion," avoiding the accidental deletion of some static objects with inaccurate motion estimation, thus improving the robustness and continuity of TSDF reconstruction.

[0093] The dynamic probability weighting function is constructed based on the logic that "the larger the deviation, the higher the dynamic probability, and the lower the weight should be." An exponential function maps the displacement deviation to a weight value between 0 and 1 to avoid accidental damage to static objects caused by directly deleting dynamic units. For example, the dynamic probability weighting function is:

[0094] in, W i For dynamic probability weights, δ i σ represents the displacement deviation between the actual displacement and the predicted value of the Gaussian ellipsoid, and σ is the deviation adjustment parameter, which is set according to empirical values.

[0095] Furthermore, to ensure the real-time and incremental nature of dynamic object identification, the embodiments of this application also require resetting the flags of dynamic units after each frame update to achieve incremental filtering. For example, after each frame is processed, the dynamic flags of all dynamic units are cleared; dynamic identification is performed only based on the information of the current frame, implementing an incremental filtering mechanism to ensure the real-time and incremental nature of dynamic identification and to ensure the system's rapid response to changes in object state.

[0096] In one implementation, to ensure efficiency, real-time performance, and accuracy by reducing computational resources and enhancing adaptability, such as Figure 5 As shown, in step S300, the Euclidean symbolic distance field of the scene environment is calculated along the gradient direction of the statically truncated symbolic distance field according to a hierarchical strategy, including: S310 divides the scene environment space corresponding to the static truncation symbol distance field into multiple hierarchical regions.

[0097] Hierarchical region division methods include: Centered on the robot corresponding to the synchronous localization and mapping system, the scene environment space is meshed within a set radius area using a set first grid resolution (e.g., 20cm). When the absolute value of the TSDF (Transient Surface Gradient Deficiency) at a grid point is less than a preset ratio of the cutoff distance range, local mesh refinement is triggered, and the mesh is further divided according to a second grid resolution (e.g., 5cm). For example, with the robot's current position as the center, a 3-meter radius sphere is used with a basic grid resolution of 20cm. When the absolute value of the TSDF at a grid point is less than 10% of the cutoff distance range, local mesh refinement to 5cm is triggered, dividing the entire computational domain into 50cm cubes. Thus, this application provides an adaptive mesh refinement mechanism, improving the accuracy of surface gradient estimation.

[0098] S320, along the gradient direction of each level region, integrate to calculate the Euclidean symbol distance field block of the corresponding level region.

[0099] To reduce computational burden and improve computational speed, in step S320, the Euclidean symbolic distance field block of the corresponding level region is calculated by integration along the gradient direction of each level region, including: S321 performs gradient estimation for each layer region in parallel, generates the corresponding gradient direction, and integrates along the gradient direction from each point in each layer region to generate the shortest path from each point to the surface of the target object, and obtains the corresponding Euclidean sign distance field block based on each shortest path.

[0100] In this approach, gradient estimation for each layer can be performed in parallel by using multiple threads simultaneously. For example, GPUs can be used to implement parallel computation.

[0101] In this embodiment, the gradient direction of each point is calculated based on TSDF. For each point, differentiation and integration are performed along the gradient direction to obtain the shortest path direction and distance from that point, thereby generating ESDF. For example, starting from any point, the path is gradually advanced along the gradient direction, accumulating step size until a point with a TSDF value of zero (the surface of the target object) is reached. Finally, the total distance and direction from that point to the target object are obtained. The target object includes, but is not limited to, obstacles, the 3D object to be constructed, etc.

[0102] The TSDF value is calculated during the generation of the Euclidean Signed Distance Field (ESDF) to determine the total distance (also known as Euclidean distance) from any point in space to the surface of the target object. The Euclidean distance is obtained by integrating along the gradient direction of the static TSDF (step-by-step, accumulating step size), and is ultimately used to construct the ESDF, providing a precise distance reference for robot path planning, obstacle avoidance, and other tasks.

[0103] Near isosurfaces with TSDF values ​​close to zero, Euclidean distance is calculated by integrating along the negative gradient direction. The core purpose of this Euclidean distance calculation is to generate the Euclidean Signed Distance Field (ESDF). Because isosurfaces with TSDF values ​​close to zero correspond to the object surface, integrating along the negative gradient direction can accurately find the shortest straight-line distance from each point in space to the object surface. This distance is the Euclidean distance. Integrating these distances yields the ESDF, which can be used in scenarios such as robot path planning and obstacle avoidance.

[0104] The gradient direction estimation methods in the embodiments of this application include, but are not limited to, implementations using the finite difference method or image gradient operators.

[0105] S330, merge all Euclidean symbolic distance field blocks to obtain the Euclidean symbolic distance field.

[0106] Each Euclidean symbolic distance field block is fused to obtain the Euclidean symbolic distance field. In this embodiment, overlapping regions exist between the Euclidean symbolic distance field blocks for gradient propagation. This embodiment achieves continuity and gradient differentiability of the entire ESDF field through boundary fusion and consistency processing. Specifically, the boundary regions of adjacent blocks can be fused to ensure a smooth transition of the gradient field; consistency can be ensured by using weighted averaging, gradient direction consistency detection, etc.

[0107] In one implementation, the regions are first divided into layers, and then the ESDF (Euclidean distance field block) of the corresponding region is calculated in each region using a pre-built formula. Finally, the blocks are fused to obtain the global ESDF.

[0108] As an example, after a scene update, the TSDF field data (ξ) corresponding to the historical scene is first obtained. old ) and weight (ω) old ), the TSDF value (ξ) after dynamic filtering of the current frame curr (Static truncated symbol distance field) and dynamic probability weights W i The current weight of positive correlation (ω) curr Substituting this into the weighted average formula shown below (a pre-built formula), the updated ESDF field (ξ) is calculated. new ):

[0109] Where, ξ new ξ represents the updated ESDF field (i.e., the Euclidean symbolic distance field). curr ω represents the filtered static truncated symbol range field TSDF. curr Represents dynamic probability weights W iThe current weight of positive correlation, ω old ξ represents the historical weight corresponding to the historical scenario. old This represents the TSDF field corresponding to the historical scenario. The core function of this formula is to calculate the updated ESDF by integrating historical and current TSDF data.

[0110] Since high-precision global modeling has a large memory overhead, this application adopts a layered strategy for calculation to avoid the memory and computational overhead caused by high-resolution modeling of the entire area. The embodiments of this application use different mesh resolutions for different areas, that is, high resolution is required for dynamic areas (such as near the robot), while low resolution can be used for areas far away from the robot. The local refinement mechanism can improve the accuracy of key areas. Finally, efficiency is improved by using GPU parallel computing to meet real-time requirements.

[0111] In this embodiment, the TSDF analytical generation technology based on 3D Gaussian density field significantly reduces memory usage and improves geometric accuracy (compared to traditional technology, the boundary error of this application is reduced by more than 70%). The motion consistency dynamic filtering mechanism effectively eliminates 95% of dynamic target interference and avoids the accidental deletion of static features. Combined with ESDF differential integral mapping and hierarchical parallel computing strategy, the distance field generation speed is increased by 8-10 times, and finally, the lightweight scene distance field construction capability with centimeter-level accuracy and millisecond-level response is achieved.

[0112] Figure 6 A schematic diagram of a Euclidean symbolic distance field generation device according to an embodiment of this application is shown. Exemplarily, the Euclidean symbolic distance field generation device includes: a truncated symbolic distance field generation module 610, a filtering module 620, and an Euclidean symbolic distance field generation module 630.

[0113] The truncated symbolic distance field generation module 610 is used to generate multiple Gaussian ellipsoid models based on the sparse point cloud of the collected scene environment, and to generate an initial truncated symbolic distance field based on the multiple Gaussian ellipsoid models. The filtering module 620 is used to filter the initial truncated symbolic range field based on the motion consistency information between the various Gaussian ellipsoid models corresponding to multiple consecutive frames of sparse point clouds, so as to obtain the static truncated symbolic range field. The Euclidean symbolic distance field generation module 630 is used to calculate the Euclidean symbolic distance field of the scene environment according to a layered strategy along the gradient direction of the statically truncated symbolic distance field.

[0114] It is understood that the device in this embodiment corresponds to the real-time scene distance field construction method in the above embodiments, and the options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0115] This application also provides a computer-readable storage medium for storing the computer program used in the robot described above. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0117] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0118] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0119] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for constructing a real-time scene distance field, characterized in that, include: Multiple Gaussian ellipsoid models are generated based on the sparse point cloud of the collected scene environment, and an initial truncated symbolic distance field is generated based on the multiple Gaussian ellipsoid models. Based on the motion consistency information between the various Gaussian ellipsoid models corresponding to the sparse point cloud in multiple consecutive frames, the initial truncated symbolic distance field is filtered to obtain the static truncated symbolic distance field. Along the gradient direction of the statically truncated symbolic distance field, the Euclidean symbolic distance field of the scene environment is calculated according to a hierarchical strategy.

2. The real-time scene distance field construction method according to claim 1, characterized in that, The calculation of the Euclidean symbolic distance field of the scene environment along the gradient direction of the statically truncated symbolic distance field, according to a hierarchical strategy, includes: The scene environment space corresponding to the static truncation symbol distance field is divided into multiple hierarchical regions; Along the gradient direction of each of the said hierarchical regions, the Euclidean symbol distance field block of the corresponding hierarchical region is calculated by integration. By fusing all the aforementioned Euclidean symbolic distance field blocks, the Euclidean symbolic distance field is obtained.

3. The real-time scene distance field construction method according to claim 2, characterized in that, The step of integrating along the gradient direction of each of the said hierarchical regions to calculate the Euclidean symbol distance field block of the corresponding hierarchical region includes: Gradient estimation is performed in parallel for each of the said hierarchical regions to generate the corresponding gradient direction, and integration is performed from each point in each of the said hierarchical regions along the gradient direction to generate the shortest path from each point to the surface of the target object, and the corresponding Euclidean sign distance field block is obtained based on each of the said shortest paths.

4. The real-time scene distance field construction method according to claim 1, characterized in that, The process of generating multiple Gaussian ellipsoid models based on the sparse point cloud of the collected scene environment, and generating an initial truncated symbolic distance field based on the multiple Gaussian ellipsoid models, includes: Based on the sparse point cloud of the collected scene environment, construct a Gaussian ellipsoid model for each point in the sparse point cloud; A continuous density field is generated based on each of the Gaussian ellipsoid models; Using the gradient direction of the continuous density field, the probability occupancy value of any point in the scene environment space in the Gaussian ellipsoid model is calculated. The basic truncated symbol distance field is obtained by mapping each of the aforementioned probability occupancy values; The truncation distance of the basic truncation symbol distance field is dynamically adjusted using a truncation distance dynamic adjustment mechanism to obtain the initial truncation symbol distance field; wherein, the truncation distance dynamic adjustment mechanism is determined based on the local curvature characteristics of the object surface.

5. The real-time scene distance field construction method according to claim 4, characterized in that, The sparse point cloud includes the three-dimensional coordinates, normal vector, covariance information, and local point cloud density of each point; the parameters of the Gaussian ellipsoid model include position, orientation, and anisotropic covariance matrix. The step of constructing a Gaussian ellipsoid model for each point in the sparse point cloud based on the collected scene environment includes: The position of the corresponding Gaussian ellipsoid model is determined based on the three-dimensional coordinates of each point. The orientation of the corresponding Gaussian ellipsoid model is determined based on the normal vector of each point; The anisotropic covariance matrix of the Gaussian ellipsoid model is obtained by adaptively adjusting the local point cloud density of the sparse point cloud.

6. The real-time scene distance field construction method according to claim 5, characterized in that, The step of adaptively adjusting the local point cloud density of the sparse point cloud to obtain the anisotropic covariance matrix of the Gaussian ellipsoid model includes: The corresponding local point cloud density is calculated by the number of neighboring points or local geometric features of each point in the sparse point cloud. The scale parameter is determined based on the local point cloud density using a preset inverse proportional strategy, and the scale covariance matrix is ​​determined based on the scale parameter. The anisotropic covariance matrix is ​​constructed based on the scale covariance matrix and the rotation matrix calculated based on the corresponding points.

7. The real-time scene distance field construction method according to claim 4, characterized in that, The generation of a continuous density field based on each Gaussian ellipsoid model includes: The probability occupancy value calculation functions of each Gaussian ellipsoid model are weighted and superimposed in the scene environment space to generate a continuous Gaussian density field.

8. The real-time scene distance field construction method according to claim 1, characterized in that, The motion consistency information includes dynamic probability weights; The step of filtering the initial truncated symbolic distance field based on the motion consistency information between the various Gaussian ellipsoid models corresponding to multiple consecutive frames of the sparse point cloud to obtain the static truncated symbolic distance field includes: Based on the multiple Gaussian ellipsoid models and the obtained dynamic probability weights and camera pose transformation matrix, the initial truncated symbolic distance field is filtered to obtain the static truncated symbolic distance field.

9. The real-time scene distance field construction method according to claim 8, characterized in that, The step of filtering the initial truncated symbolic range field based on multiple Gaussian ellipsoid models and the obtained dynamic probability weights and camera pose transformation matrix to obtain the static truncated symbolic range field includes: The theoretical position of each Gaussian ellipsoid model in the current frame is predicted based on the camera pose transformation matrix output by the synchronous localization and mapping system, thus obtaining the predicted position; wherein, the synchronous localization and mapping system is used to acquire sparse point clouds of the scene environment; For each Gaussian ellipsoid model, if the deviation between the actual position and the predicted position of the Gaussian ellipsoid model exceeds a set deviation threshold, the corresponding Gaussian ellipsoid model is marked as a dynamic unit. The dynamic probability weights are calculated based on the dynamic probability weight function, and the transparency of each dynamic unit in the initial truncated symbol distance field is attenuated to obtain the static truncated symbol distance field.

10. A device for generating a Euclidean symbolic distance field, characterized in that, include: The truncated symbolic distance field generation module is used to generate multiple Gaussian ellipsoid models based on the sparse point cloud of the collected scene environment, and to generate an initial truncated symbolic distance field based on the multiple Gaussian ellipsoid models. The filtering module is used to filter the initial truncated symbolic distance field based on the motion consistency information between the various Gaussian ellipsoid models corresponding to the sparse point cloud in multiple consecutive frames, so as to obtain the static truncated symbolic distance field. The Euclidean symbolic distance field generation module is used to calculate the Euclidean symbolic distance field of the scene environment along the gradient direction of the statically truncated symbolic distance field according to a hierarchical strategy.

11. A robot, characterized in that, The robot includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the real-time scene distance field construction method according to any one of claims 1-9.

Images