Space environment perception method, device and equipment suitable for multi-agent system

Abstract

The application discloses a space environment perception method, device and equipment suitable for a multi-agent system, relates to the field of space environment perception, and comprises the following steps: inputting a scene image collected by an agent into a space environment perception model to extract multi-granularity visual features in the scene image, mapping the extracted multi-scale features to different visual state spaces, performing semantic segmentation on the scene image by using the mapped scene visual features, and obtaining a space environment perception result; the space environment perception model is a model obtained by training a semantic segmentation model composed of an encoder and a decoder by using historical scene images, a skip connection mode is adopted between the encoder and the decoder, and a multi-scale feature extraction network and a double-path visual state space network are included in the encoder; a plurality of convolution layers with different convolution kernel sizes are connected in parallel in the multi-scale feature extraction network. The application can improve the perception accuracy and the perception efficiency in a complex space environment.

Description

Technical Field

[0001] This application relates to the field of space environment perception, and in particular to space environment perception methods, devices and equipment applicable to multi-agent systems. Background Technology

[0002] With the rapid development of technologies such as communication networks, distributed computing, and reinforcement learning, multi-agent systems, with their excellent coordination, task parallelism, robustness, and environmental adaptability, can achieve efficient task allocation and collaborative execution in complex, dynamic, and uncertain environments through information interaction and dynamic cooperation among multiple agents (such as robots). For example, they can be applied in task allocation scenarios such as extraterrestrial exploration, urban delivery, search and rescue inspection, swarm detection, disaster emergency response, and intelligent transportation.

[0003] However, the complexity of application scenarios poses a significant challenge to the spatial environment perception capabilities of intelligent agents, presenting them with perception problems different from traditional static perception. For example, the position and shape of targets in a scene change dynamically as the agent moves, making it crucial to ensure the effectiveness of spatial environment perception. Furthermore, the high similarity between species and background interference mean that industrial objects, road infrastructure, or debris from disaster areas may be highly similar to the background in texture and color, amplifying perception errors in task allocation strategies and leading to error prioritization or resource waste. Finally, the limited communication bandwidth presents another challenge. Current perception models need to run in real-time on edge devices and share information in compressed / sparse form; maintaining perception accuracy under limited bandwidth and computational budget directly determines the quality of collaborative decision-making.

[0004] For multi-agent environmental perception and task allocation in various complex spatial environments, the current mainstream approach is to extract visual features based on convolutional neural networks (CNNs) and directly input these features into the subsequent decision network. However, convolutional operations essentially only weight local neighborhoods, resulting in significant limitations in capturing long-range dependencies and global semantic information in complex environments. Furthermore, while traditional CNNs can extract local spatial features to some extent, their receptive fields are fixed, and as the number of network layers increases, the dependencies between features gradually become fragmented, failing to effectively model complex dependencies in the global environment. This leads to a lack of high-level abstraction of the global state during task allocation, making it impossible to accurately reflect the diversity and changes in the environment.

[0005] Furthermore, convolutional structures struggle to provide adaptive responses in dynamic environments, often experiencing a decline in perceptual accuracy due to environmental changes, thus affecting the quality and accuracy of task allocation. Additionally, when faced with tasks spanning long periods or large spatial dimensions, convolutional neural networks exhibit slow convergence and are prone to overfitting, limiting the generalization ability of task allocation strategies and hindering the full exploitation of spatiotemporal dependencies in complex environments. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a spatial environment perception method, apparatus, and device suitable for multi-agent systems, which can improve the perception accuracy and efficiency in complex spatial environments. The specific solution is as follows:

[0007] In a first aspect, this application discloses a spatial environment perception method applicable to multi-agent systems, comprising:

[0008] Acquire scene images collected by each agent in the multi-agent system within the target task scenario to obtain multiple initial scene images;

[0009] The initial scene image is input into the trained spatial environment perception model to extract multi-scale features from the multi-granularity visual features in the initial scene image, thereby obtaining multi-granularity scene visual features. The multi-granularity scene visual features are then mapped to different visual state spaces to obtain mapped scene visual features. The initial scene image is then semantically segmented using multiple mapped scene visual features to obtain spatial environment perception results.

[0010] The spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset, wherein the encoder and the decoder adopt a skip connection method.

[0011] The encoder includes a multi-scale feature extraction network and a dual-path visual state space network. The multi-scale feature extraction network includes multiple target convolutional layers and serpentine convolutional layers with different kernel sizes connected in parallel. The dual-path visual state space network includes two paths that connect multiple visual state space blocks based on the Mamba architecture in parallel.

[0012] Optionally, the multi-scale feature extraction network further includes a first residual block;

[0013] Accordingly, the step of performing multi-scale feature extraction on the multi-granularity visual features in the initial scene image to obtain multi-granularity scene visual features includes:

[0014] The initial scene image is input into multiple target convolutional layers and serpentine convolutional layers with different kernel sizes in a multi-scale feature extraction network, so as to extract multi-granularity visual features from the initial scene image using convolutional kernels of different sizes, thereby obtaining multiple initial scene visual features of different scales.

[0015] The initial scene visual features at multiple different scales are weighted and fused to obtain fused visual features. The fused visual features are then input into the first residual block to perform nonlinear representation of the fused scene visual features, thereby obtaining multi-granularity scene visual features.

[0016] Optionally, mapping the multi-granularity scene visual features to different visual state spaces to obtain mapped scene visual features includes:

[0017] The multi-granularity scene visual features are respectively input into two paths of the dual-path visual state space network to map the multi-granularity scene visual features to visual state spaces of different depths, thereby obtaining the first visual state representation vector and the second visual state representation vector.

[0018] The first visual state representation vector and the second visual state representation vector are weighted and fused to obtain the mapped scene visual features.

[0019] Optionally, the dual-path vision state space network may further include a linear layer;

[0020] Accordingly, the step of inputting the multi-granularity scene visual features into two paths of the dual-path visual state space network to map the multi-granularity scene visual features to visual state spaces of different depths, thereby obtaining a first visual state representation vector and a second visual state representation vector, includes:

[0021] The multi-granularity scene visual features are subjected to layer normalization processing to obtain normalized visual features, and the normalized visual features are input into a linear layer to perform channel projection on the normalized visual features to obtain projected visual features.

[0022] The projected visual features are input into two paths of the dual-path visual state space network to map the projected visual features to visual state spaces of different depths, thereby obtaining a first visual state representation vector and a second visual state representation vector.

[0023] Optionally, the decoder includes multiple upsampling layers and multiple second residual blocks;

[0024] Accordingly, the step of semantically segmenting the initial scene image using multiple mapped scene visual features to obtain spatial environment perception results includes:

[0025] The mapped scene visual features are input into the second residual block to perform a nonlinear representation of the mapped scene visual features, thereby obtaining nonlinear visual features;

[0026] The nonlinear visual features are input into the upsampling layer for upsampling to obtain sampled visual features. The sampled visual features are then used to perform semantic segmentation on different environmental regions in the initial scene images collected by each agent, resulting in spatial environment perception results that include environmental semantic label maps and confidence maps.

[0027] Optionally, after obtaining the spatial environment perception result including the environmental semantic label map and the confidence map, the method further includes:

[0028] Connectivity aggregation and boundary fitting are performed on environmental regions with the same semantic labels in the spatial environment perception results to form candidate task units for task allocation.

[0029] Generate corresponding task description information for the candidate task units and obtain the current state information of each agent;

[0030] The task description information and the current state information are input into the task allocation decision model based on reinforcement learning network to generate corresponding execution tasks for each agent and allocate the execution tasks to the corresponding agents.

[0031] Optionally, inputting the initial scene image into the trained spatial environment perception model includes:

[0032] Each of the initial scene images is subjected to denoising and abnormal pixel removal processing to obtain the scene image after removal.

[0033] The scene images after removal are subjected to illumination normalization and contrast enhancement processing to obtain enhanced scene images, and all the enhanced scene images are normalized to obtain normalized scene images.

[0034] The normalized scene image is subjected to distortion correction processing to obtain a preprocessed scene image, and the preprocessed scene image is input into the trained spatial environment perception model.

[0035] Secondly, this application discloses a spatial environment perception device suitable for multi-agent systems, comprising:

[0036] The acquisition module is used to acquire scene images collected by each agent in the multi-agent system in the target task scenario, and obtain multiple initial scene images;

[0037] The spatial environment perception module is used to input the initial scene image into the trained spatial environment perception model to extract multi-scale features from the multi-granularity visual features in the initial scene image, obtain multi-granularity scene visual features, map the multi-granularity scene visual features to different visual state spaces to obtain mapped scene visual features, and use multiple mapped scene visual features to perform semantic segmentation on the initial scene image to obtain spatial environment perception results.

[0038] The spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset, wherein the encoder and the decoder adopt a skip connection method.

[0039] The encoder includes a multi-scale feature extraction network and a dual-path visual state space network. The multi-scale feature extraction network includes multiple target convolutional layers and serpentine convolutional layers with different kernel sizes connected in parallel. The dual-path visual state space network includes two paths that connect multiple visual state space blocks based on the Mamba architecture in parallel.

[0040] Thirdly, this application discloses an electronic device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the aforementioned spatial environment perception method applicable to multi-agent systems.

[0041] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned spatial environment perception method applicable to multi-agent systems.

[0042] As can be seen, this application first acquires scene images collected by each agent in the multi-agent system within the target task scenario, obtaining multiple initial scene images. Then, the initial scene images are input into a trained spatial environment perception model to extract multi-scale features from the multi-granularity visual features in the initial scene images, obtaining multi-granularity scene visual features. These multi-granularity scene visual features are then mapped to different visual state spaces to obtain mapped scene visual features. Finally, the initial scene images are semantically segmented using multiple mapped scene visual features to obtain spatial environment perception results. The spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset. The encoder and decoder use a skip connection method. The encoder includes a multi-scale feature extraction network and a dual-path visual state space network. The multi-scale feature extraction network includes multiple target convolutional layers and serpentine convolutional layers of different kernel sizes connected in parallel. The dual-path visual state space network includes two paths that connect multiple visual state space blocks based on the Mamba architecture in parallel.

[0043] This application pre-creates a semantic segmentation model for multi-agent spatial environment perception. This model consists of an encoder and a decoder connected via skip connections. By using skip connections, low-level detail information can be preserved while enhancing global semantic modeling, thereby achieving high-precision semantic segmentation of complex scenes and subsequent semantic graph construction. The encoder includes a multi-scale feature extraction network (comprising multiple convolutional layers with different kernel sizes, such as serpentine convolutional layers) and a dual-path visual state space network (comprising two parallel paths composed of multiple visual state space blocks based on the Mamba architecture). Therefore, the dual-path visual state space network in this application... The visual state space network adopts a dual-path parallel hybrid structure. This not only obtains local spatial features (i.e., local detailed structure) but also acquires high-level abstractions of the global environment state (i.e., global semantic information). This improves the ability to represent the visual state of local and global features and reduces the overlap of redundant information and complex features, significantly improving the clarity and efficiency of spatial environment perception. In addition, this application also combines multiple parallel convolutional layers with different kernel sizes for multi-scale feature extraction, thereby enhancing the ability to extract global features from complex environments and improving the perception accuracy in complex spatial environments. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 This is a flowchart of a spatial environment perception method applicable to multi-agent systems disclosed in this application;

[0046] Figure 2 This is a schematic diagram of a specific multi-agent spatial environment perception model architecture disclosed in this application;

[0047] Figure 3 This is a schematic diagram of a specific dual-path vision state space network structure disclosed in this application;

[0048] Figure 4 This is a schematic diagram of a specific multi-scale feature extraction network structure disclosed in this application;

[0049] Figure 5 This application discloses a specific method for spatial environment perception applicable to multi-agent systems.

[0050] Figure 6 This is a schematic diagram of the structure of a space environment perception device suitable for multi-agent systems disclosed in this application;

[0051] Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] This application discloses a spatial environment perception method suitable for multi-agent systems. See also... Figure 1 As shown, the method includes:

[0054] Step S11: Obtain scene images collected by each agent in the multi-agent system in the target task scene to obtain multiple initial scene images.

[0055] In this embodiment, firstly, single / multiple scene images (such as monocular images, binocular images, multispectral images, etc.) are acquired by each agent (such as an intelligent robot) in the multi-agent system in the target task scenario through image acquisition devices such as cameras, 3D cameras, and vision sensors to obtain multiple initial scene images.

[0056] The target task scenarios include, but are not limited to, extraterrestrial exploration, urban delivery, search and rescue patrols, swarm detection, disaster emergency response, intelligent transportation, and robot path planning. A multi-agent system is a distributed intelligent system in which multiple intelligent agents with autonomous perception, decision-making, and collaboration capabilities work together through communication and coordination to complete complex tasks.

[0057] It should be noted that while acquiring the scene image, information such as the acquisition timestamp, camera intrinsic and extrinsic parameters, and intelligent body pose corresponding to the initial scene image can also be acquired simultaneously for subsequent image preprocessing and semantic segmentation result alignment operations.

[0058] Step S12: Input the initial scene image into the trained spatial environment perception model to extract multi-scale features from the multi-granularity visual features in the initial scene image, obtain multi-granularity scene visual features, and map the multi-granularity scene visual features to different visual state spaces to obtain mapped scene visual features. Then, use multiple mapped scene visual features to perform semantic segmentation on the initial scene image to obtain spatial environment perception results. The spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset. The encoder and the decoder adopt a skip connection method. The encoder includes a multi-scale feature extraction network and a dual-path visual state space network. The multi-scale feature extraction network includes multiple target convolutional layers and serpentine convolutional layers with different kernel sizes connected in parallel. The dual-path visual state space network includes two paths that connect multiple visual state space blocks based on the Mamba architecture in parallel.

[0059] In this embodiment, after acquiring the scene images collected by each agent in the target task scenario, the initial scene images can be further input into the trained spatial environment perception model in order of the agent's number, level, type, and acquisition time, so as to perceive the physical environment in which each agent is located.

[0060] For details, see Figure 2As shown, the spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset. Furthermore, the encoder (used to enhance the modeling of global semantics while preserving low-level detail information, thereby achieving high-precision semantic segmentation of complex scenes and subsequent semantic graph construction) and the decoder adopt a skip connection approach. This connection approach can effectively bring low-level detail features into the upsampling layer, so that high-level semantic features can obtain more accurate spatial reconstruction and boundary refinement in the decoding stage, thereby avoiding the loss of spatial details of deep features during continuous downsampling.

[0061] Additionally, see Figure 2 As shown, the encoder employs a three-stage progressive downsampling framework, specifically comprising three stages consisting of a multi-scale feature extraction network and a dual-path visual state space network (which can be one or more). In each stage, deep modeling of different receptive fields and spatial information at different levels is achieved through the alternating stacking of the multi-scale feature extraction network and the dual-path visual state space network. This allows for multi-scale feature extraction and progressive feature compression, with the resolution of the resulting feature maps progressively reduced to 1 / 4, 1 / 8, and 1 / 16 of the original image. This results in semantically rich coarse-grained features at low resolution while retaining fine spatial details at high resolution, achieving a balanced representation of both global and local features. By combining the local fine features obtained through the multi-scale feature extraction network with the global temporal and spatial dependency information extracted by the dual-path visual state space network, efficient representation and fusion of multi-scale features can be achieved. This framework significantly enhances the global feature extraction capability of the environment and improves perception accuracy in complex environments, providing robust technical support for subsequent task allocation and decision-making in complex environments.

[0062] The specific perception process includes: first, multi-scale feature extraction is performed on the initial scene image using a multi-scale feature extraction network in the encoder to extract features of local details in each region of the initial scene image and global dependencies between different regions, resulting in multi-granular scene visual features; next, the extracted multi-granular scene visual features are mapped to different visual state spaces using a dual-path visual state space network in the encoder, resulting in mapped scene visual features; finally, multiple mapped scene visual features are used to perform semantic segmentation on different regions in the initial scene image, such as classifying and labeling different regions in the image, extracting finer-grained spatial information, and obtaining the spatial environment perception results of the entire target task scene, so as to make subsequent task allocation decisions, which helps to improve the accuracy of task allocation and decision-making in dynamic environments.

[0063] Among them, multi-granularity visual features refer to image features extracted from different levels or scales of abstraction, covering information from low-level details (such as edges and textures) to high-level semantics (such as objects and scenes). This method can capture the diversity and complexity of images more comprehensively and improve the performance of models in visual understanding tasks.

[0064] Specifically, the multi-scale feature extraction network further includes a first residual block; correspondingly, the multi-scale feature extraction of multi-granularity visual features in the initial scene image to obtain multi-granularity scene visual features may specifically include: inputting the initial scene image into multiple target convolutional layers and serpentine convolutional layers with different kernel sizes in the multi-scale feature extraction network, so as to extract multi-granularity visual features in the initial scene image using convolutional kernels of different sizes, thereby obtaining multiple initial scene visual features at different scales; performing weighted fusion of the multiple initial scene visual features at different scales to obtain fused visual features, and inputting the fused visual features into the first residual block to perform nonlinear representation of the fused scene visual features, thereby obtaining multi-granularity scene visual features. See also Figure 3 As shown, this embodiment proposes a multi-scale feature extraction network comprising residual blocks, linear layers, and three convolutional layers with parallel connections (including a 7×7 convolutional layer, a 5×5 convolutional layer, and a serpentine convolutional layer), to enhance the perception of multi-granular information and irregular target boundaries in complex scenes. Multi-granular information refers to image information that is observable at different levels—pixel-level, region-level, and global-level—and used for semantic segmentation and discrimination, determined by the physical boundaries of the real scene (e.g., occlusion / depth boundaries), material and surface microstructures (e.g., reflection and texture), the regional consistency and component structure of entities (e.g., continuous surfaces and compositional relationships), and scene geometry and contextual layout (e.g., scale, perspective, and co-occurrence relationships).

[0065] Understandably, traditional convolutional layers with fixed convolutional sizes (such as 7×7 or 5×5) perform well in extracting local and global textures, but they are prone to sampling alignment errors, edge blurring, and feature aliasing when dealing with thin, curved, or irregular boundaries. This is especially true in scenarios where there are complex geometric relationships between the target and the background or significant local deformations, making it difficult to accurately fit the target contour. To address these issues, this application introduces a snake-shaped convolutional layer (Dynamic SnakeConvolution, an improved convolutional method in convolutional neural networks, primarily used to enhance the model's ability to recognize thin and weak local structural features as well as complex and variable global morphological features). Through learnable sampling offsets, the snake-shaped convolutional layer enables the convolutional kernel to adaptively bend and extend along the true geometry of the target, using sub-pixel interpolation to finely locate boundary sampling points, thereby reducing redundant responses from irrelevant neighborhoods and maintaining edge continuity. Furthermore, the serpentine convolutional layer complements the 7×7 and 5×5 convolutional layers. The 7×7 convolutional layer has a large receptive field, enabling it to capture the macroscopic spatial distribution features of the environment with fewer layers, making it suitable for perceiving the overall structure of terrain undulations, open areas, and areas with dense obstacles. The serpentine convolutional layer, through path design using asymmetric kernels or dilated convolutions, exhibits greater flexibility and adaptability when handling irregular boundaries and deformed targets. It can identify natural paths, cracks, gullies, and boundary cues in unstructured environments, achieving accurate modeling of complex boundaries and local deformations. This significantly improves the accuracy and robustness of restoring irregular target boundaries without sacrificing global perception capabilities. The 5×5 convolutional layer is used to perceive local terrain details and capture medium-scale texture details.

[0066] In this embodiment, see Figure 3 As shown, the initial scene image can first undergo layer normalization, and then the normalized image is input into a linear layer. This linear transformation maps the data of the input initial scene image to a new feature space. The features G output from the linear layer are then input into the snake-shaped convolutional layer, the 7×7 convolutional layer, and the 5×5 convolutional layer in the multi-scale feature extraction network. This allows for the extraction of multi-granularity visual features from the initial scene image using convolutional kernels of different sizes, resulting in three different scales of initial scene visual features. , and Specifically, it can be expressed as:

[0067] ;

[0068] ;

[0069] ;

[0070] In the formula, This represents the weight parameters of the 7×7 convolutional layer. This represents the weight parameters of a 5×5 convolutional layer. This represents the weight parameters of the serpentine convolutional layer.

[0071] Next, the three initial scene visual features output from the three convolutional branches are concatenated to aggregate features at different scales, resulting in a fused visual feature containing rich information (such as shape, boundary, size, spatial continuity, etc.). :

[0072] .

[0073] Furthermore, regarding the fused visual features Perform layer normalization to obtain normalized features. :

[0074] ;

[0075] The layer normalization process described above ensures a more uniform distribution of each feature, thus avoiding the vanishing or exploding gradient problems and effectively promoting the stability of the training process. Furthermore, layer normalization can accelerate network convergence, thereby improving the robustness of the spatial environment perception model.

[0076] Features after splicing After layer normalization, the features are then processed through residual blocks to refine the normalized features. Perform nonlinear representation to obtain nonlinear representation characteristics. :

[0077] ;

[0078] It should be noted that residual blocks, while ensuring smooth gradients, introduce non-linear expressive power and, through skip connections, help the network effectively avoid the vanishing gradient problem. Residual blocks allow the network to focus on learning the residual between the input and output, rather than the entire mapping, thereby improving training efficiency and effectiveness.

[0079] Finally, the nonlinear representation features Channel compression and feature reprojection are performed using a 1×1 convolutional layer to obtain the projected features. That is, multi-granularity scene visual features:

[0080] ;

[0081] The aforementioned channel compression and feature reprojection operations effectively reduce feature depth, thereby lowering computational overhead and optimizing feature map dimensionality. This makes the extracted multi-scale features more compact during fusion while preserving the effective transmission of key information (such as the spatial structure of obstacles in multi-agent systems). The final output is a feature representation with multi-scale information and compact dimensionality. Furthermore, it allows for efficient abstraction of complex environments, significantly reducing redundancy in state representations while retaining global structure, local interactions, and directional features.

[0082] pass Figure 3 The multi-scale feature extraction network structure in this paper enables the fused features to contain multi-scale information under different receptive fields and have a compact channel representation. This effectively improves the network's ability to perceive and model multi-scale structures and boundary details without significantly increasing computational overhead. This multi-scale feature extraction method not only enhances the network's adaptability to complex scenes, but also provides a more flexible and efficient solution for handling irregular targets.

[0083] In one specific implementation, mapping the multi-granularity scene visual features to different visual state spaces to obtain mapped scene visual features may specifically include: inputting the multi-granularity scene visual features into two paths of a dual-path visual state space network to map the multi-granularity scene visual features to visual state spaces of different depths, obtaining a first visual state representation vector and a second visual state representation vector; and performing weighted fusion of the first visual state representation vector and the second visual state representation vector to obtain the mapped scene visual features. In this embodiment, after multi-scale feature extraction through a multi-scale feature extraction network, the extracted multi-granularity scene visual features can be input into two paths of a dual-path visual state space network to map the multi-granularity scene visual features to visual state spaces of different depths through each path, obtaining corresponding first and second visual state representation vectors, and then performing weighted fusion of the two visual state representation vectors to obtain a mapped scene visual feature.

[0084] Each path includes a Visual StateSpace (VSS) block based on the Mamba architecture. Compared to the traditional approach of directly inputting visual features from the feature extraction network into subsequent task decision networks, which only obtains local spatial features and lacks a high-level abstraction of the global environment state, resulting in insufficient state representation capabilities and problems such as feature redundancy, low state discriminability, and slow reinforcement learning convergence, this solution constructs two parallel hybrid visual statespace paths based on multiple Mamba-based visual statespace blocks. This approach balances local and global representations, simultaneously obtaining both local details and global semantic information.

[0085] Specifically, the dual-path visual state space network also includes a linear layer; correspondingly, the step of inputting the multi-granularity scene visual features into two paths of the dual-path visual state space network to map the multi-granularity scene visual features to visual state spaces of different depths, obtaining a first visual state representation vector and a second visual state representation vector, may specifically include: performing layer normalization processing on the multi-granularity scene visual features to obtain normalized visual features, and inputting the normalized visual features into a linear layer to perform channel projection on the normalized visual features to obtain projected visual features; inputting the projected visual features into two paths of the dual-path visual state space network to map the projected visual features to visual state spaces of different depths, obtaining a first visual state representation vector and a second visual state representation vector. In this embodiment, see... Figure 4 As shown, the dual-path visual state space network specifically includes linear layers, convolutional layers, and two parallel connected visual state space paths. Each path includes a visual state space block based on the Mamba architecture (there can be one or more). Specifically, the extracted multi-granularity scene visual features can first be tensorized and represented. H, W, and C represent the height dimension, width dimension, and number of channels of the tensor, respectively, to obtain the tensorized vector X. Next, layer normalization is performed on the tensorized vector X to obtain the normalized visual features. Specifically, it can be expressed as:

[0086] ;

[0087] Furthermore, the normalized visual features are processed through a linear layer. Perform channel projection to obtain the projected visual features. Specifically, it can be expressed as:

[0088] ;

[0089] In the formula, and For learnable linear projection parameters, For layer normalization.

[0090] Next, the projected visual features The input is fed into a convolutional layer for feature extraction, resulting in extracted features. Specifically, it can be expressed as:

[0091] ;

[0092] Then, the extracted features The data are fed into two parallel paths for processing: the first path is a mapping path consisting of a single visual state space block (VSS), which processes the extracted features obtained after the convolutional layer. Perform a state-space mapping once to obtain the first visual state representation vector. The second path is a deep mapping path consisting of two cascaded visual state space blocks (VSS), which is used for the extracted features. A step-by-step state transition is performed to extract higher-order feature relationships, resulting in the second visual state representation vector. Finally, the outputs of the two paths can be processed by layer normalization and then weighted and fused to form a unified visual state representation vector.

[0093] The update formula for the Visual State Space Block (VSS) is as follows:

[0094] ;

[0095] ;

[0096] in, It is in a hidden state. For learnable parameters, input (Right now () are the features after passing through the convolutional layer.

[0097] pass Figure 4 The dual-path visual state space network structure shown can effectively fuse features from different paths and scales. Furthermore, through residual connections, layer normalization, and linear transformations, feature representation can be further optimized. The visual state space block (VSS) is used for deep analysis of spatial features, achieving efficient representation of long-range dependencies. This reduces redundant information and the aliasing of complex features, significantly improving the clarity and robustness of spatial environment perception. Simultaneously, it enhances the network's ability to perceive and model multi-scale structures, boundary details, and complex morphologies, thereby significantly improving the agent's perception of its surrounding environment without increasing computational complexity.

[0098] Specifically, the decoder includes multiple upsampling layers and multiple second residual blocks; correspondingly, the step of semantically segmenting the initial scene image using multiple mapped scene visual features to obtain spatial environment perception results may specifically include: inputting the mapped scene visual features into the second residual blocks to perform nonlinear representation on the mapped scene visual features to obtain nonlinear visual features; inputting the nonlinear visual features into the upsampling layers for upsampling to obtain sampled visual features; and using the sampled visual features to perform semantic segmentation on different environmental regions in the initial scene image collected by each agent to obtain spatial environment perception results including an environmental semantic label map and a confidence map. In this embodiment, see... Figure 2 As shown, the decoder includes multiple upsampling layers and multiple residual blocks, and it is connected to the outputs of different stages in the encoder via skip connections. Through skip connections, high-precision features from earlier layers in the encoder (e.g., 1 / 4 resolution) can be directly passed to the corresponding decoder stage and input together with the decoder's upsampling results into the feature processing module composed of convolutional layers and residual blocks. This allows low-level detailed features to be input into the upsampling layers, enabling more accurate spatial reconstruction and boundary refinement of high-level semantic features during the decoding stage. In the decoder, the spatial resolution of the feature map is first gradually restored through upsampling operations. Then, the deep semantic features output from the i-th layer of the encoder (e.g., a 1 / 16 resolution feature map) are concatenated with the shallow high-resolution features (e.g., a 1 / 8 resolution feature map) after upsampling at the corresponding stage of the decoder along the channel dimension to obtain a fused feature. This fused feature contains both global semantic information and local details, achieving complementarity between coarse and fine-grained information, which helps improve boundary localization and the integrity of the target structure. Furthermore, the fused features can undergo convolution and normalization to suppress redundant channels and enhance discriminative ability, thereby obtaining a highly fused representation of multi-granular information. Finally, the decoder gradually restores the image to the same spatial resolution as the input image (i.e., the initial scene image) through continuous upsampling, convolution, and residual mapping operations, and outputs the target semantic image (i.e., the spatial environment perception result, which may include an environment semantic label map and the corresponding confidence map). This semantic image not only retains the high-level semantic discriminative ability provided by the deep features of the encoder, but also fully inherits the information of shallow features in terms of boundaries and local details, thus achieving accurate modeling and high-quality restoration of multi-scale structures and boundary details in complex scenes as a whole.

[0099] For spatial environment perception in unstructured scenarios involving extraterrestrial objects, common shortcomings include unstable semantic output, inaccurate boundaries, and the inability to quantify uncertain regions. This often leads to multi-agent task allocation relying on pre-set maps or manually defined task locations. Inaccurate environmental semantic understanding directly distorts task allocation input, such as misclassifying impassable areas as passable and assigning them data collection tasks, overlooking high-risk areas and failing to trigger verification tasks, or causing unreasonable task area division due to unclear semantic boundaries, resulting in duplicate coverage and blind spots. This reduces task efficiency and increases execution risk. This application utilizes a highly robust image semantic segmentation method to obtain the environmental semantic elements required for task allocation, providing a reliable basis for subsequent task generation and multi-agent task allocation.

[0100] Specifically, an environmental semantic label map can be generated based on preset semantic categories, which include, but are not limited to, passable areas, impassable areas, obstacles, steep slopes / loose soil, shadow / dust-occluded areas, and task target areas; the acquired scene images and preset semantic categories are then input into... Figure 2 After semantic segmentation, the spatial environment perception model shown can output pixel-level environmental semantic label maps and confidence maps.

[0101] Furthermore, after obtaining the spatial environment perception result containing the environmental semantic label map and the confidence map, the process may further include: performing connected component aggregation and boundary fitting on environmental regions with the same semantic labels in the spatial environment perception result to form candidate task units for task allocation; generating corresponding task description information for the candidate task units and obtaining the current state information of each agent; inputting the task description information and the current state information into a task allocation decision model based on a reinforcement learning network to generate corresponding execution tasks for each agent and allocate the execution tasks to the corresponding agents. In this embodiment, different regions in the image are classified and labeled by semantic segmentation to obtain spatial environment perception results containing environmental semantic label maps and confidence maps. This allows for the extraction of more fine-grained spatial information, and the resulting high-confidence environmental semantic label map helps improve the accuracy of task allocation and decision-making processes in dynamic environments. For spatial environment perception in unstructured scenarios of extraterrestrial objects, boundary refinement and morphological consistency correction can be performed on pixel-level environmental semantic label maps to obtain refined semantic segmentation maps. At the same time, uncertainty labeling can be applied to low-confidence regions (regions smaller than a preset confidence threshold) in the semantic segmentation map based on the confidence map to obtain a labeled pixel-level category map containing a set of uncertainty regions. The set of uncertainty regions is used to trigger subsequent tasks such as verification, reshooting, or multi-view observation, thereby improving the reliability of task allocation input information.

[0102] Furthermore, semantic regions such as passable areas, impassable areas, obstacles, risky terrain, and uncertain areas are extracted based on pixel-level category maps and confidence maps. Connectivity aggregation and boundary fitting are then performed on each semantic region to form candidate task units that can be used for task allocation. Simultaneously, corresponding task description information is generated for each candidate task unit, such as its location range, scale parameters, risk level, confidence index, and priority index. Additionally, current state information such as the pose, remaining energy, payload status, and communication quality of multiple agents can be acquired to construct the set of state inputs and constraints required for task allocation. This structures the spatial environment perception results into a data representation that can be directly used for task allocation.

[0103] Finally, the current state information and task description information of the multiple agents are input into the task allocation decision model based on reinforcement learning network, so as to generate corresponding execution tasks for each agent. At the same time, under the condition of satisfying constraints such as reachability and security, a mapping relationship and execution order between different execution tasks and agents can be formed. Then, the execution tasks are allocated to the corresponding agents according to the mapping relationship and execution order.

[0104] It should be noted that the task allocation decision model takes the benefits of task completion as positive feedback and comprehensively considers negative costs such as duplicate coverage, resource consumption and risk entry, so that multi-agents can achieve adaptive allocation and dynamic adjustment of candidate task units in unstructured extraterrestrial object scenarios.

[0105] As can be seen, the embodiments of this application pre-create a semantic segmentation model for multi-agent spatial environment perception. This model consists of an encoder and a decoder connected by skip connections. By using skip connections, the model can enhance the modeling of global semantics while preserving low-level detailed information, thereby achieving high-precision semantic segmentation of complex scenes and subsequent semantic graph construction. The encoder includes a multi-scale feature extraction network (including multiple convolutional layers with different kernel sizes, such as serpentine convolutional layers) and a dual-path visual state space network (including two parallel paths connected by multiple visual state space blocks based on the Mamba architecture). The dual-path visual state space network adopts a dual-path parallel hybrid structure. This structure can not only obtain local spatial features (i.e., local detailed structure) but also acquire high-level abstractions of the global environment state (i.e., global semantic information). This improves the ability to represent the visual state of local and global features and reduces the overlap of redundant information and complex features, significantly improving the clarity and efficiency of spatial environment perception. In addition, by combining multiple parallel convolutional layers with different kernel sizes for multi-scale feature extraction, the ability to extract global features of complex environments can be enhanced, and the perception accuracy in complex environments can be improved.

[0106] This application discloses a specific spatial environment perception method applicable to multi-agent systems. (See also...) Figure 5 As shown, the method includes:

[0107] Step S21: Obtain scene images collected by each agent in the multi-agent system in the target task scene to obtain multiple initial scene images.

[0108] Step S22: Denoise and abnormal pixel removal are performed on each of the initial scene images to obtain the scene images after removal.

[0109] In this embodiment, after obtaining the scene images of each intelligent agent, corresponding preprocessing operations can be performed on them. First, noise reduction and abnormal pixel removal can be performed on each scene image to obtain the scene image after removal.

[0110] Step S23: Perform illumination normalization and contrast enhancement processing on the removed scene image to obtain an enhanced scene image, and perform normalization processing on all the enhanced scene images to obtain a normalized scene image.

[0111] Next, the scene images after removal are subjected to illumination normalization and contrast enhancement processing to reduce the brightness deviation caused by shadows, strong reflections and dust obstruction, resulting in enhanced scene images. Then, the scale of all enhanced scene images is unified (i.e., normalization processing is performed) to obtain normalized scene images.

[0112] Step S24: Perform distortion correction processing on the normalized scene image to obtain a preprocessed scene image, and input the preprocessed scene image into the trained spatial environment perception model to extract multi-scale features from the multi-granularity visual features in the preprocessed scene image to obtain multi-granularity scene visual features. Map the multi-granularity scene visual features to different visual state spaces to obtain mapped scene visual features, and use multiple mapped scene visual features to perform semantic segmentation on the initial scene image to obtain spatial environment perception results.

[0113] In this embodiment, distortion correction processing can be performed on the normalized scene image, and resolution alignment and field of view cropping can be performed on the corrected image from multiple sources (multi-agents) to obtain a preprocessed scene image. Then, the preprocessed scene image is input into the trained spatial environment perception model, so as to perform spatial environment perception through semantic segmentation and obtain spatial environment perception results.

[0114] For a more detailed description of the process of step S21, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0115] As can be seen, the embodiments of this application perform preprocessing operations on scene images, such as denoising and abnormal pixel removal, illumination normalization and contrast enhancement, normalization, and distortion correction, which makes the input data of the spatial environment perception model more accurate, thereby further improving the accuracy of spatial environment perception.

[0116] Accordingly, this application also discloses a spatial environment perception device suitable for multi-agent systems, see [link to relevant documentation]. Figure 6 As shown, the device includes:

[0117] The acquisition module 11 is used to acquire scene images collected by each agent in the multi-agent system in the target task scene, and obtain multiple initial scene images;

[0118] The spatial environment perception module 12 is used to input the initial scene image into the trained spatial environment perception model, to extract multi-scale features from the multi-granularity visual features in the initial scene image, to obtain multi-granularity scene visual features, and to map the multi-granularity scene visual features to different visual state spaces to obtain mapped scene visual features, and to use multiple mapped scene visual features to perform semantic segmentation on the initial scene image to obtain spatial environment perception results.

[0119] The spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset, wherein the encoder and the decoder adopt a skip connection method.

[0120] The encoder includes a multi-scale feature extraction network and a dual-path visual state space network. The multi-scale feature extraction network includes multiple target convolutional layers and serpentine convolutional layers with different kernel sizes connected in parallel. The dual-path visual state space network includes two paths that connect multiple visual state space blocks based on the Mamba architecture in parallel.

[0121] The specific workflow of each of the above modules can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.

[0122] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0123] Figure 7This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the spatial environment perception method for multi-agent systems disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0124] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0125] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0126] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the spatial environment perception method for a multi-agent system disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0127] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned spatial environment perception method suitable for multi-agent systems. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0128] Furthermore, embodiments of this application also disclose a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the spatial environment perception method for multi-agent systems disclosed above.

[0129] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0130] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0131] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0132] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0133] The above provides a detailed description of the spatial environment perception method, apparatus, and device for multi-agent systems provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A spatial environment perception method suitable for multi-agent systems, characterized in that, include: Acquire scene images collected by each agent in the multi-agent system within the target task scenario to obtain multiple initial scene images; The initial scene image is input into the trained spatial environment perception model to extract multi-scale features from the multi-granularity visual features in the initial scene image, thereby obtaining multi-granularity scene visual features. The multi-granularity scene visual features are then mapped to different visual state spaces to obtain mapped scene visual features. The initial scene image is then semantically segmented using multiple mapped scene visual features to obtain spatial environment perception results. The spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset, wherein the encoder and the decoder adopt a skip connection method. The encoder includes a multi-scale feature extraction network and a dual-path visual state space network. The multi-scale feature extraction network includes multiple target convolutional layers and serpentine convolutional layers with different kernel sizes connected in parallel. The dual-path visual state space network includes two paths that connect multiple visual state space blocks based on the Mamba architecture in parallel. The multi-scale feature extraction network also includes a first residual block; Accordingly, the step of extracting multi-granularity visual features from the initial scene image at multiple scales to obtain multi-granularity scene visual features includes: inputting the initial scene image into multiple target convolutional layers and serpentine convolutional layers with different kernel sizes in a multi-scale feature extraction network, so as to extract multi-granularity visual features from the initial scene image using convolutional kernels of different sizes, thereby obtaining multiple initial scene visual features at different scales; performing weighted fusion on the multiple initial scene visual features at different scales to obtain fused visual features, and inputting the fused visual features into a first residual block to perform nonlinear representation on the fused scene visual features, thereby obtaining multi-granularity scene visual features.

2. The spatial environment perception method for multi-agent systems according to claim 1, characterized in that, The step of mapping the multi-granularity scene visual features to different visual state spaces to obtain the mapped scene visual features includes: The multi-granularity scene visual features are respectively input into two paths of the dual-path visual state space network to map the multi-granularity scene visual features to visual state spaces of different depths, thereby obtaining the first visual state representation vector and the second visual state representation vector. The first visual state representation vector and the second visual state representation vector are weighted and fused to obtain the mapped scene visual features.

3. The spatial environment perception method for multi-agent systems according to claim 2, characterized in that, The dual-path vision state space network also includes a linear layer; Accordingly, the step of inputting the multi-granularity scene visual features into two paths of the dual-path visual state space network to map the multi-granularity scene visual features to visual state spaces of different depths, thereby obtaining a first visual state representation vector and a second visual state representation vector, includes: The multi-granularity scene visual features are subjected to layer normalization processing to obtain normalized visual features, and the normalized visual features are input into a linear layer to perform channel projection on the normalized visual features to obtain projected visual features. The projected visual features are input into two paths of the dual-path visual state space network to map the projected visual features to visual state spaces of different depths, thereby obtaining a first visual state representation vector and a second visual state representation vector.

4. The spatial environment perception method for multi-agent systems according to claim 3, characterized in that, The decoder includes multiple upsampling layers and multiple second residual blocks; Accordingly, the step of semantically segmenting the initial scene image using multiple mapped scene visual features to obtain spatial environment perception results includes: The mapped scene visual features are input into the second residual block to perform a nonlinear representation of the mapped scene visual features, thereby obtaining nonlinear visual features; The nonlinear visual features are input into the upsampling layer for upsampling to obtain sampled visual features. The sampled visual features are then used to perform semantic segmentation on different environmental regions in the initial scene images collected by each agent, resulting in spatial environment perception results that include environmental semantic label maps and confidence maps.

5. The spatial environment perception method for multi-agent systems according to claim 4, characterized in that, After obtaining the spatial environment perception result including the environmental semantic label map and the confidence map, the method further includes: Connectivity aggregation and boundary fitting are performed on environmental regions with the same semantic labels in the spatial environment perception results to form candidate task units for task allocation. Generate corresponding task description information for the candidate task units and obtain the current state information of each agent; The task description information and the current state information are input into the task allocation decision model based on reinforcement learning network to generate corresponding execution tasks for each agent and allocate the execution tasks to the corresponding agents.

6. The spatial environment perception method for multi-agent systems according to any one of claims 1 to 5, characterized in that, The step of inputting the initial scene image into the trained spatial environment perception model includes: Each of the initial scene images is subjected to denoising and abnormal pixel removal processing to obtain the scene image after removal. The scene images after removal are subjected to illumination normalization and contrast enhancement processing to obtain enhanced scene images, and all the enhanced scene images are normalized to obtain normalized scene images. The normalized scene image is subjected to distortion correction processing to obtain a preprocessed scene image, and the preprocessed scene image is input into the trained spatial environment perception model.

7. A spatial environment sensing device suitable for multi-agent systems, characterized in that, include: The acquisition module is used to acquire scene images collected by each agent in the multi-agent system in the target task scenario, and obtain multiple initial scene images; The spatial environment perception module is used to input the initial scene image into the trained spatial environment perception model to extract multi-scale features from the multi-granularity visual features in the initial scene image, obtain multi-granularity scene visual features, map the multi-granularity scene visual features to different visual state spaces to obtain mapped scene visual features, and use multiple mapped scene visual features to perform semantic segmentation on the initial scene image to obtain spatial environment perception results. The spatial environment perception model is a model obtained by training a semantic segmentation model consisting of an encoder and a decoder using a historical scene image dataset, wherein the encoder and the decoder adopt a skip connection method. The encoder includes a multi-scale feature extraction network and a dual-path visual state space network. The multi-scale feature extraction network includes multiple target convolutional layers and serpentine convolutional layers with different kernel sizes connected in parallel. The dual-path visual state space network includes two paths that connect multiple visual state space blocks based on the Mamba architecture in parallel. The multi-scale feature extraction network also includes a first residual block; Correspondingly, the spatial environment perception module is specifically used to input the initial scene image into multiple target convolutional layers and serpentine convolutional layers with different kernel sizes in the multi-scale feature extraction network, so as to extract multi-granularity visual features in the initial scene image using convolutional kernels of different sizes, and obtain multiple initial scene visual features of different scales. The initial scene visual features at multiple different scales are weighted and fused to obtain fused visual features. The fused visual features are then input into the first residual block to perform nonlinear representation of the fused scene visual features, thereby obtaining multi-granularity scene visual features.

8. An electronic device, characterized in that, It includes a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the spatial environment perception method for multi-agent systems as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when executed by a processor, the computer programs implement the spatial environment perception method for multi-agent systems as described in any one of claims 1 to 6.

Images