Emergency biomimetic robot collaborative method and multimodal rescue system with task and state collaborative perception

Abstract

This invention provides an emergency bionic robot collaborative method and multimodal rescue system with task and state collaborative perception, relating to the field of emergency bionic robot technology. It acquires environmental perception data streams through multi-source sensing acquisition devices deployed on each robot body in an emergency bionic robot cluster, covering panoramic vision, terrain stiffness distribution, gas composition concentration, and sound wave spectrum data. Each type of data stream is processed to obtain feature segments, and then visual and tactile, olfactory and auditory related segments are spatially superimposed to generate a collaborative scene map. The collaborative scene map undergoes physical field state interference deduction and cross-map temporal correlation fusion to generate a global collaborative perception situational deduction map. Based on this global collaborative perception situational deduction map, the coordinates of the disaster spread source, the leading edge of the spread path, and the robot's pose and state information are determined, generating a distributed rescue collaborative instruction set to achieve efficient robot collaborative rescue.

Description

Technical Field

[0001] This invention relates to the field of emergency bionic robot technology, and more specifically, to an emergency bionic robot collaborative method and multimodal rescue system with task and state collaborative perception. Background Technology

[0002] In emergency rescue scenarios, biomimetic robot swarms, with their flexibility and adaptability, have become an important force in performing complex rescue missions. However, current emergency biomimetic robot swarms face many challenges in collaborative operations.

[0003] On the one hand, existing environmental perception methods are relatively limited. In most cases, robots rely on only a single type of sensor to acquire environmental information, such as using only a visual sensor to acquire scene images or only using a gas sensor to detect gas composition. These single perception methods are insufficient to comprehensively and accurately reflect the multi-dimensional characteristics of complex emergency environments. For example, at a fire scene, visual sensors may be unable to clearly acquire scene information due to smoke obscuring the view, while gas sensors can only provide gas concentration data and cannot intuitively present the location of the fire source and its spread trend.

[0004] On the other hand, the collaboration between robots lacks effective information fusion and situational analysis mechanisms. After each robot independently acquires environmental information, it often simply transmits the information to the central control system for unified processing and decision-making. However, this approach suffers from problems such as information transmission delays and high processing pressure on the central system, and it is difficult to achieve real-time, efficient collaboration between robots. For example, in earthquake rubble rescue, robots in different locations may simultaneously discover trapped individuals, but due to the lack of an effective collaboration mechanism, they cannot quickly determine the optimal rescue path and division of labor, resulting in low rescue efficiency. Summary of the Invention

[0005] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an emergency bionic robot collaborative method with task and state collaborative perception, the method comprising: The environmental perception data stream is acquired by multi-source sensing and acquisition devices of each bionic robot body deployed in the emergency bionic robot cluster. The environmental perception data stream includes panoramic visual data stream, terrain stiffness distribution data stream, gas composition concentration data stream and sound wave spectrum data stream. Visual saliency region segmentation is performed on the panoramic visual data stream to obtain visual saliency region segments; stiffness jump boundary tracking is performed on the terrain stiffness distribution data stream to obtain terrain stiffness jump boundary segments; gas concentration front identification is performed on the gas composition concentration data stream to obtain gas concentration front movement trajectory segments; and anomalous frequency band separation is performed on the acoustic spectrum data stream to obtain anomalous acoustic frequency band change segments. The visually salient region segment and the terrain stiffness jump boundary segment are spatially superimposed to generate a visual-tactile collaborative scene map. The gas concentration front movement trajectory segment and the abnormal sound wave frequency band change segment are spatially superimposed to generate an olfactory-auditory collaborative scene map. Physical field state interference inference processing and cross-graph temporal correlation fusion processing are performed on the visual-tactile collaborative scene map and the olfactory-auditory collaborative scene map to generate a global collaborative perception situation inference map; Based on the global collaborative perception situational simulation map, the spatial coordinates of the disaster spread source and the front line of the evolution trend of the disaster spread path are determined, and the current pose state information of each bionic robot body is determined. Based on the spatial coordinates of the disaster spread source, the front line of the evolution trend and the current pose state information of each bionic robot body, a distributed rescue collaborative instruction set is generated.

[0006] Furthermore, embodiments of the present invention also provide a multimodal rescue system, comprising: A processor; a machine-readable storage medium for storing machine-executable instructions of the processor; wherein the processor is configured to execute the aforementioned emergency bionic robot collaborative method with task and state collaborative awareness by executing the machine-executable instructions.

[0007] In another aspect, embodiments of the present invention also provide a computer program product, the computer program product including machine-executable instructions, the machine-executable instructions being stored in a computer-readable storage medium, the processor of the multimodal rescue system reading the machine-executable instructions from the computer-readable storage medium, the processor executing the machine-executable instructions, causing the multimodal rescue system to execute the above-mentioned emergency bionic robot collaborative method with task and state collaborative perception.

[0008] Based on the above, by deploying multi-source sensing devices on each robot in an emergency bionic robot swarm, environmental perception data streams containing multi-dimensional data such as panoramic vision, terrain stiffness distribution, gas composition concentration, and sound wave spectrum are acquired. Targeted processing of various data streams yields visually salient region segments, terrain stiffness abrupt boundary segments, gas concentration front movement trajectory segments, and abnormal sound wave frequency band change segments, achieving effective extraction and accurate identification of environmental features. By spatially overlaying visual and tactile, and olfactory and auditory related segments, visual-tactile collaborative scene maps and olfactory-auditory collaborative scene maps are generated, achieving preliminary fusion of multimodal information. Further physical field state interference inference processing and cross-map temporal correlation fusion processing are performed on the two collaborative scene maps to generate a global collaborative perception situation inference map, which can comprehensively and dynamically reflect the evolution trend of the disaster environment and the relative states between each robot. Based on the situation simulation map, the spatial coordinates of the disaster spread source, the front line of the evolution trend of the disaster spread path, and the current pose status information of each robot body are determined. A distributed rescue coordination instruction set is generated, realizing real-time and efficient coordination between robots and improving the efficiency and success rate of emergency rescue. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the execution flow of the emergency bionic robot collaborative method with task and state collaborative perception provided in the embodiments of the present invention.

[0010] Figure 2 This is a schematic diagram of exemplary hardware and software components of the multimodal rescue system provided in an embodiment of the present invention. Detailed Implementation

[0011] Figure 1 This is a flowchart illustrating an emergency biomimetic robot collaborative method with task and state collaborative perception provided in one embodiment of the present invention, which will be described in detail below.

[0012] Step S110: Obtain environmental perception data streams through multi-source sensing acquisition devices deployed on each bionic robot body in the emergency bionic robot cluster. The environmental perception data streams include panoramic visual data streams, terrain stiffness distribution data streams, gas composition concentration data streams, and sound wave spectrum data streams.

[0013] The panoramic visual data stream is acquired by multi-view panoramic cameras mounted on the top of each bionic robot's torso. Each robot's panoramic camera outputs a cylindrically projected panoramic image with a horizontal 360-degree and vertical 180-degree field of view at a frame rate Ff. Each frame is a three-channel digital image matrix with H rows and W columns, where the matrix elements are the pixel color component brightness value I. The terrain stiffness distribution data stream is acquired by a distributed pressure sensor array and displacement sensors at the ends of each of the four legs of each bionic robot. Each leg outputs one frame of terrain stiffness distribution map after each ground contact cycle, with the stiffness response value at each ground contact point in the map being Ks = Fpeak / Dnormal. The gas component concentration data stream is acquired by an electrochemical gas sensor array mounted on the front of each robot's torso, outputting the volume concentration values ​​Cgas of various target gas components at a frequency Fg. The acoustic spectrum data stream is collected by four acoustic pickups installed around the torso of each bionic robot. The sound pressure waveform at each pickup position is output at a sampling frequency Fs. The acoustic spectrum vector is obtained by short-time Fourier transform, and the power spectral density value of each frequency band is denoted as Psound.

[0014] Step S120: Perform visual saliency region segmentation processing on the panoramic visual data stream to obtain visual saliency region segments; perform stiffness jump boundary tracking processing on the terrain stiffness distribution data stream to obtain terrain stiffness jump boundary segments; perform gas concentration front identification processing on the gas composition concentration data stream to obtain gas concentration front movement trajectory segments; and perform acoustic abnormal frequency band separation processing on the acoustic spectrum data stream to obtain abnormal acoustic frequency band change segments.

[0015] Step S121: Perform multi-scale visual region segmentation processing on each frame of panoramic visual image in the panoramic visual data stream to generate multiple visual region units. Each visual region unit is composed of all pixels within that visual region unit.

[0016] For a single frame of panoramic visual image It received at time t, a Gaussian pyramid scale space of number G is constructed on the image plane. The g-th scale image is obtained by convolving the original image It with a Gaussian kernel function with standard deviation σg and then downsampling with stride sg. On each scale image, a grid is divided using a fixed-size rectangular window sliding with a stride. Each grid rectangular region is a visual region unit, and the data structure of the visual region unit is a tuple containing a list of coordinate indices and an array of pixel values ​​for all pixels within the unit.

[0017] Step S122: Perform visual contrast calculation processing on each visual region unit, calculate the visual contrast value between each visual region unit and its neighboring visual region units, and the visual contrast value is determined based on the difference between the brightness distribution of the pixels in the visual region unit and the brightness distribution of the pixels in the neighboring visual region units.

[0018] For the u-th visual region unit Ru at the g-th scale, extract its eight neighboring visual region units Rv. Calculate the mean μu and standard deviation σu of the brightness values ​​of all pixels in Ru, and the mean μv and standard deviation σv of Rv. The visual contrast value Cuv = |μu−μv| / ((μu+μv) / 2)+|σu−σv| / (σu+σv).

[0019] Step S123: Construct a visual contrast spatial distribution map based on the visual contrast values ​​corresponding to each visual region unit, and mark the height corresponding to the visual contrast value of each visual region unit in the visual contrast spatial distribution map.

[0020] Using the spatial index coordinates of visual region units as two-dimensional plane coordinates, the maximum value of the visual contrast values ​​of each visual region unit and its eight neighboring visual region units is taken as the representative contrast value of that visual region unit. All the representative contrast values ​​are arranged according to their spatial positions to form a two-dimensional array as a visual contrast spatial distribution map.

[0021] Step S124: Extract visual region units whose visual contrast values ​​exceed the preset visual contrast limit Tvis from the visual contrast spatial distribution map as candidate visual saliency region units, and assign a center seed point mark to the pixel with the highest visual contrast value in each candidate visual saliency region unit.

[0022] Traverse all visual region units in the visual contrast spatial distribution map, and record visual region units with a contrast representative value exceeding Tvis as candidate visual saliency region units. Within each candidate visual saliency region unit, find the image coordinates of the pixel with the highest brightness value among all pixels in that unit, mark this pixel as the center seed point, and record its coordinates and contrast representative value.

[0023] Step S125: Starting from each candidate visual saliency region unit carrying a center seed point marker, perform region growing processing on adjacent visual region units. Adjacent visual region units whose visual contrast values ​​differ from the candidate visual saliency region unit by less than a preset growth tolerance Tgrow are accepted as the same visual saliency connected region. The candidate visual saliency region unit is merged with the accepted adjacent visual region units to obtain a visual saliency connected region. Contour extraction processing is performed on the obtained visual saliency connected region. The outer boundary pixels of the visual saliency connected region are connected to form a closed contour line. The area enclosed by the closed contour line is used as the visual saliency region unit in the panoramic visual image of this frame.

[0024] Starting from the candidate visually saliency region unit where the central seed point is located, a breadth-first search is used to check the absolute value of the difference between the representative contrast value of each of its eight neighboring visual regions and the representative contrast value of the unit where the central seed point is located. If this absolute value is less than Tgrow, the neighboring visual region unit is merged into the current visually saliency connected region and used as a new growth starting point to continue checking outwards. The search continues until the absolute value of the contrast difference of all neighboring units at the boundary of the connected region is not less than Tgrow. The set of pixels in the merged connected region where at least one neighboring visual region unit does not belong to the connected region is taken as the boundary pixel layer. The boundary pixels are connected in clockwise order to form a discrete point sequence and then connected point by point with straight line segments to form a closed contour line. The pixel region enclosed by the closed contour line is a visually saliency region unit.

[0025] Step S126: Perform the multi-scale visual region segmentation process, the visual contrast calculation process, the candidate visual saliency region unit extraction process, the region growth process, and the contour extraction process on all consecutive frames of the panoramic visual image in the panoramic visual data stream to obtain the visual saliency region unit corresponding to each frame of the panoramic visual image. Connect the visual saliency region units with spatial overlapping relationships between consecutive frames according to the timestamp order to form a visual saliency region segment that progresses continuously in the time dimension.

[0026] For each frame of the panoramic visual image It with timestamps from 1 to T, steps S121 to S125 are executed sequentially to obtain the set of visually salient region units for the t-th frame. Spatial overlap matching is performed on the visually salient region units of adjacent frames, and the intersection-over-union (IoU) is calculated. If the IoU exceeds a preset inter-frame IoU threshold, the two units are determined to belong to the same target on the time axis and are assigned to the same visually salient region connection chain. For connection chains that maintain matching across multiple consecutive frames, the visually salient region units corresponding to each frame are arranged in chronological order according to their timestamps to form visually salient region segments.

[0027] Step S127: Extract the stiffness response value Ks of each ground contact point from each frame of the terrain stiffness distribution map in the terrain stiffness distribution data stream, map the stiffness response value of each ground contact point to the spatial coordinates of the ground contact point on the terrain stiffness distribution map, generate a terrain stiffness spatial distribution point matrix, perform stiffness gradient calculation processing on each ground contact point in the terrain stiffness spatial distribution point matrix, calculate the rate of change of stiffness response value between the ground contact point and its adjacent ground contact points, and generate the stiffness gradient vector corresponding to each ground contact point.

[0028] For each frame of the terrain stiffness distribution map, the stiffness response values ​​Ks of all ground contact points are combined with their spatial coordinates to form a terrain stiffness spatial distribution matrix. For a ground contact point with coordinates (x, y) in the matrix, a circular neighborhood with radius R is taken centered on it. The stiffness gradient vector G = (∂Ks / ∂x, ∂Ks / ∂y), where ∂Ks / ∂x is the ratio of the stiffness difference to the coordinate difference between two adjacent points in the X direction, and ∂Ks / ∂y is the ratio of the stiffness difference to the coordinate difference between two adjacent points in the Y direction. The vector magnitude |G| = sqrt((∂Ks / ∂x)² + (∂Ks / ∂y)²).

[0029] Step S128: Select ground contact points from the spatial distribution point matrix of terrain stiffness where the magnitude of the stiffness gradient vector exceeds the preset stiffness abrupt change limit Tgrad as stiffness jump candidate points, and assign a jump seed point label to each stiffness jump candidate point. Taking each stiffness jump candidate point with a jump seed point label as the starting point, sequentially search the stiffness gradient vector of adjacent ground contact points along the direction of the stiffness gradient vector of the stiffness jump candidate point. Connect adjacent ground contact points where the angle between the direction of the stiffness gradient vector and the direction of the stiffness gradient vector of the starting point is less than the preset direction tolerance Tangle and the magnitude of the stiffness gradient vector exceeds Tgrad in sequence to form a stiffness jump chain.

[0030] Step S129: Arrange all ground contact points in the stiffness jump chain in the connection order to obtain the stiffness jump boundary line. The stiffness jump boundary line divides the terrain stiffness distribution map into a first terrain zone and a second terrain zone where the stiffness response value difference exceeds a preset stiffness separation threshold Tsep.

[0031] Take strip-shaped regions of width Wstrip on both sides of the stiffness jump boundary line, and calculate the mean stiffness response values ​​Kleft for the left strip-shaped region and Kright for the right strip-shaped region. If |Kleft−Kright|>Tsep, then the boundary line is confirmed to effectively separate two different terrain zones.

[0032] Step S1210: For all consecutive frames of the terrain stiffness distribution map in the terrain stiffness distribution data stream, perform the following processes: extracting the stiffness response value of the ground contact point, calculating the stiffness gradient, filtering the stiffness jump candidate points, and connecting the stiffness jump chain. This will obtain the stiffness jump boundary line corresponding to each frame of the terrain stiffness distribution map. Connect the stiffness jump boundary lines with spatial continuity between consecutive frames according to the timestamp order to form a terrain stiffness jump boundary segment that evolves continuously in the time dimension.

[0033] Steps S127 to S129 are executed sequentially for each frame of the terrain stiffness distribution map from timestamp 1 to T to obtain the stiffness jump boundary line of frame t. The Euclidean distance between the mean spatial locations of the stiffness jump boundary lines of adjacent frames is calculated. If the distance is less than a preset inter-frame continuity distance threshold, the two boundary lines are determined to be a continuous evolution of the same stiffness jump boundary, and are assigned to the same stiffness jump boundary connection chain. For connection chains that maintain matching across multiple consecutive frames, the corresponding stiffness jump boundary lines of each frame are arranged according to their timestamps to form terrain stiffness jump boundary segments.

[0034] Step S1211: Perform gas concentration front identification processing on the gas component concentration data stream to obtain a gas concentration front movement trajectory segment.

[0035] For each sampling moment, the multi-component gas volume concentration value Cgas output by the biomimetic robot's gas sensor array is used to construct a scalar field of the gas concentration spatial distribution at that moment in the global coordinate system using the Kriging spatial interpolation algorithm. The spatial gradient field is calculated for the scalar field, and spatial locations where the gradient amplitude exceeds a preset concentration gradient threshold Tconc are extracted as gas concentration front points. Adjacent front points are connected to form front lines. This process is repeated for each consecutive sampling moment, connecting the front lines with spatial continuity between consecutive moments in time stamp order to form gas concentration front movement trajectory segments.

[0036] Step S1212: Perform acoustic abnormal frequency band separation processing on the acoustic spectrum data stream to obtain abnormal acoustic frequency band change segments.

[0037] For the acoustic spectrum vector at each pickup location, using the normal ambient background noise spectrum as a reference baseline, calculate the power spectral density ratio Psound / Pref for each frequency band. Frequency bands with ratios exceeding a preset abnormal frequency band threshold Tabn are marked as abnormal frequency bands, and their center frequency, bandwidth, and power ratio are recorded. This process is repeated for each consecutive sampling time, and abnormal frequency bands with center frequency deviations less than a preset frequency drift tolerance between consecutive time points are sequentially linked by timestamp to form abnormal acoustic frequency band variation segments.

[0038] Step S130: The visually salient region segment and the terrain stiffness jump boundary segment are spatially superimposed to generate a visual-tactile collaborative scene map, and the gas concentration front movement trajectory segment and the abnormal sound wave frequency band change segment are spatially superimposed to generate an olfactory-auditory collaborative scene map.

[0039] Step S131: Map the spatial coordinates of the visually salient region units corresponding to each time section in the visually salient region segment and the set of spatial coordinates of the stiffness jump boundary line corresponding to the same time section in the terrain stiffness jump boundary segment to the same spatial coordinate plane in a unified spatial reference system to obtain the spatially registered visually salient region layer and terrain stiffness jump boundary layer.

[0040] A unified spatial reference system is adopted, using a global Cartesian coordinate system with the disaster site reference point as the origin. Through a pre-calibrated panoramic camera extrinsic parameter matrix, the pixel coordinates (u, v) of visually salient region units are mapped to spatial coordinates (X, Y, Z) in the global coordinate system via inverse perspective projection transformation and rigid body coordinate transformation. Using the forward kinematics model of the foot of each bionic robot body, the local coordinates of various ground contact points relative to the bionic robot body on the stiffness jump boundary line are mapped to the same global coordinate system via odometry pose transformation. Pixel-level alignment is achieved on the same spatial coordinate plane after mapping the two data sources.

[0041] Step S132: Perform region coverage marking processing on the regions enclosed by the closed contour lines of each visually salient region unit in the spatially registered visually salient region layer, and assign visual coverage markings to the spatial coordinate points falling within the closed contour lines of each visually salient region unit in the unified spatial reference system; and perform boundary line marking processing on the spatial coordinate points traversed by each stiffness jump boundary line in the spatially registered terrain stiffness jump boundary layer, and assign boundary line markings to the spatial coordinate points traversed by each stiffness jump boundary line in the unified spatial reference system.

[0042] The region coverage marking process employs a ray projection determination algorithm where points lie within polygons. For each spatial coordinate point in a unified spatial reference frame, a ray is emitted from that point in any direction, and the number of intersections between this ray and the closed contour line of the visually salient region unit is counted. If the number of intersections is odd, the point lies inside the closed contour line, and a visual coverage mark is assigned to it. For the boundary line marking process, for each ground contact point on each stiffness jump boundary line, the neighboring spatial coordinate points within a preset linewidth radius centered on that point are uniformly assigned boundary line marks.

[0043] Step S133: In the unified spatial reference system, spatial coordinate points carrying both the visual overlay marker and the boundary line marker are determined as visual terrain-related points, spatial coordinate points carrying only the visual overlay marker but not the boundary line marker are determined as visually independent points, and spatial coordinate points carrying only the boundary line marker but not the visual overlay marker are determined as terrain-independent points. Multimodal association marker processing is performed on the visual terrain-related points, and the visual contrast value of the visual saliency region unit corresponding to the visual terrain-related point and the stiffness gradient vector magnitude of the ground contact point corresponding to the visual terrain-related point are recorded as the visual terrain dual-mode association marker of the visual terrain-related point.

[0044] The visual terrain bimodal association is labeled as a tuple (Cvis, |G|), where Cvis is the visual contrast representative value of the visually salient region unit to which the associated point belongs, and |G| is the magnitude of the stiffness gradient vector at the ground contact point corresponding to the associated point.

[0045] Step S134: Perform visual independent field state assignment processing on the visually independent point, setting the visual field state intensity value of the visually independent point to the field state intensity value corresponding to the visual contrast value of the visual saliency region unit to which the visually independent point belongs. Perform terrain independent field state assignment processing on the terrain independent point, setting the terrain field state intensity value of the terrain independent point to the field state intensity value corresponding to the stiffness gradient vector magnitude of the ground contact point corresponding to the terrain independent point.

[0046] The visual field intensity value Evis = fvis(Cvis), where fvis is a mapping function that linearly normalizes the visual contrast value to the interval [0, 1]. The terrain field intensity value Eter = fter(|G|), where fter is a mapping function that linearly normalizes the stiffness gradient magnitude to the interval [0, 1].

[0047] Step S135: Merge the visual terrain association points, the visual independent points carrying the visual field intensity values, and the terrain independent points carrying the terrain field intensity values ​​in the same spatial coordinate plane to generate a visual-tactile collaborative scene map with spatial coordinate points as the basic unit. Each spatial coordinate point in the visual-tactile collaborative scene map contains at least one of the following labeling information: visual field intensity label, terrain field intensity label, and visual-terrain association intensity label.

[0048] Layer merging overlays all three types of spatial coordinate points into the same spatial raster matrix. Each cell of the raster matrix corresponds to a spatial coordinate point and stores a marker vector containing three components: visual field intensity marker Evis, terrain field intensity marker Eter, and visual-terrain correlation intensity marker Corr. The correlation intensity marker Corr takes the normalized value of the product of the visual contrast value and the stiffness gradient magnitude at visual-terrain correlation points, and takes the value of zero at other points.

[0049] Step S136: Spatial position superposition of the gas concentration front movement trajectory segment and the abnormal sound wave frequency band change segment to generate an olfactory-auditory collaborative scene map.

[0050] The spatial coordinates of the gas concentration front points at each time segment in the gas concentration front movement trajectory segment are mapped to the spatial coordinates of the microphones corresponding to the abnormal frequency bands at the same time segment in the abnormal sound wave frequency band change segment, and processed using the same layer overlay, label assignment, and field state assignment methods as in steps S131 to S135. The odor field state intensity value Eodor at the gas concentration front point is obtained by normalizing the gas concentration gradient amplitude at that point, and the sound wave field state intensity value Eacous at the spatial location point corresponding to the abnormal frequency band is obtained by normalizing the power spectral density ratio at that point. After layer merging, each spatial coordinate point contains at least one of the following label information: odor field state intensity label, sound wave field state intensity label, and odor-sound wave correlation intensity label, forming an olfactory-auditory collaborative scene map.

[0051] Step S140: Perform physical field state interference inference processing and cross-graph temporal correlation fusion processing on the visual-tactile collaborative scene map and the olfactory-auditory collaborative scene map to generate a global collaborative perception situation inference map.

[0052] Step S141: Extract all spatial coordinate points carrying the visual terrain association intensity marker from the visual-tactile collaborative scene graph as physical field state interference seed points. Construct an interference neighborhood window centered on each physical field state interference seed point and covering a preset interference radius. Within the interference neighborhood window, extract all spatial coordinate points carrying the visual field state intensity marker from the visual-tactile collaborative scene graph to form a visual field state point set, and extract all spatial coordinate points carrying the terrain field state intensity marker within the interference neighborhood window to form a terrain field state point set.

[0053] Spatial coordinate points with a visual-topographic correlation strength marker (Corr) greater than zero are selected as seed points for visual-tactile physical field interference. The interference radius (Rinter) is set according to the spatial scale of the disaster site. A square window with a side length of 2 × Rinter, centered on the seed point, serves as the interference neighborhood window. All spatial coordinate points are traversed within the window, and points with a visual field strength marker (Evis) greater than a preset threshold are added to the visual field point set, while points with a topographic field strength marker (Eter) greater than a preset threshold are added to the topographic field point set. The purpose of interference simulation is to model the mutual influence between visual and tactile perception information at the same spatial location. The visual field represents the intensity of the disaster characteristics reflected by the visual saliency of that point, while the topographic field represents the physical boundary intensity reflected by the abrupt change in ground stiffness at that point. When these two are superimposed in space, they can either reinforce or cancel each other out.

[0054] Step S142: Perform field state attenuation processing on the visual field state intensity values ​​of each spatial coordinate point in the visual field state point set. Calculate the attenuated visual field state intensity contribution value based on the spatial distance between each spatial coordinate point and the physical field state interference seed point. The greater the spatial distance, the lower the visual field state intensity contribution value of the spatial coordinate point. Perform field state attenuation processing on the terrain field state intensity values ​​of each spatial coordinate point in the terrain field state point set. Calculate the attenuated terrain field state intensity contribution value based on the spatial distance between each spatial coordinate point and the physical field state interference seed point. The greater the spatial distance, the lower the terrain field state intensity contribution value of the spatial coordinate point.

[0055] The attenuation is simulated using a Gaussian attenuation function to model the gradual weakening of the physical field state with distance diffusion. For the seed point coordinates (x0, y0) and the field state point coordinates (xj, yj), the Euclidean distance between the two points is dj = sqrt((xj−x0)2+(yj−y0)2). The attenuation coefficient wj = exp(−dj2 / (2σ2)), where σ is the attenuation bandwidth parameter, controlling the spatial scale of the field state attenuation. The attenuated visual field state intensity contribution of visual field state point j is Evis_j×wj, and the attenuated terrain field state intensity contribution of terrain field state point j is Eter_j×wj. The attenuation coefficient and contribution value are smaller for field state points farther away from the seed point.

[0056] Step S143: The attenuated visual field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the visual field intensity convergence value. The attenuated terrain field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the terrain field intensity convergence value. The visual field intensity convergence value and the terrain field intensity convergence value are correlated with each other to generate a mutual interference inference point record for the physical field interference seed point. The mutual interference inference point record includes the spatial coordinates of the physical field interference seed point, the visual field intensity convergence value, and the terrain field intensity convergence value.

[0057] The visual field convergence quantity Svis = Σ(Evis_j × wj) is summed and iterated over all points in the visual field point set. The terrain field convergence quantity Ster = Σ(Eter_j × wj) is summed and iterated over all points in the terrain field point set. The field-state inter-perturbation correlation processing combines the two convergence quantities into a tuple (Svis, Ster), which, together with the seed point coordinates (x0, y0), constitutes the inter-perturbation inference point record. This inter-perturbation inference point record quantifies the intensity of the convergence of the visual field and terrain field around the seed point location, and is used to characterize whether the point is simultaneously affected by both visual saliency anomalies and terrain stiffness abrupt changes.

[0058] Step S144: Traverse all physical field state interference seed points in the visual-tactile collaborative scene graph. For each physical field state interference seed point, perform the interference neighborhood window construction process, the visual field state point set and terrain field state point set extraction process, the field state attenuation process, the field state convergence and accumulation process, the field state interference association process, and the interference inference point record generation process to obtain the visual-tactile interference inference result. The visual-tactile interference inference result is composed of the interference inference point records of all physical field state interference seed points.

[0059] The visual-tactile interference simulation result is a list of interference simulation point records for all visual-tactile physical field state interference seed points. Each record in the list includes spatial location coordinates, Svis value, and Ster value. This visual-tactile interference simulation result describes the spatial distribution of mutual perturbation between the visual perception field state and the terrain tactile field state at various locations in the disaster site.

[0060] Step S145: Extract all spatial coordinate points carrying odor sound wave correlation intensity markers from the olfactory-auditory collaborative scene graph as physical field state interference seed points. Construct an interference neighborhood window centered on each physical field state interference seed point and covering a preset interference radius. Within the interference neighborhood window, extract all spatial coordinate points carrying odor field state intensity markers from the olfactory-auditory collaborative scene graph to form an odor field state point set, and extract all spatial coordinate points carrying sound wave field state intensity markers from the interference neighborhood window to form a sound wave field state point set.

[0061] The processing method is the same as in step S141, except that the association intensity marker is replaced with the odor sound wave association intensity marker, and the visual field intensity marker and terrain field intensity marker are replaced with the odor field intensity marker Eodor and the sound wave field intensity marker Eacous, respectively. The construction rules for the odor field point set and the sound wave field point set are the same as those for the visual field point set and the terrain field point set.

[0062] Step S146: Perform field state attenuation processing on the odor field state intensity values ​​of each spatial coordinate point in the odor field state point set, and calculate the attenuated odor field state intensity contribution value based on the spatial distance between each spatial coordinate point and the physical field state interference seed point. The greater the spatial distance, the lower the odor field state intensity contribution value. Perform field state attenuation processing on the sound wave field state intensity values ​​of each spatial coordinate point in the sound wave field state point set, and calculate the attenuated sound wave field state intensity contribution value based on the spatial distance between each spatial coordinate point and the physical field state interference seed point. The greater the spatial distance, the lower the sound wave field state intensity contribution value.

[0063] The attenuation process uses the same Gaussian attenuation function and attenuation bandwidth parameter σ as step S142. The attenuated contribution value of odor field state point j is Eodor_j×wj, and the attenuated contribution value of sound wave field state point j is Eacous_j×wj.

[0064] Step S147: The attenuated odor field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the odor field intensity. The attenuated sound wave field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the sound wave field intensity. The odor field intensity and the sound wave field intensity are correlated with each other to generate a mutual interference prediction point record for the physical field interference seed point. The mutual interference prediction point record includes the spatial coordinates of the physical field interference seed point, the odor field intensity, and the sound wave field intensity.

[0065] The odor field convergence quantity is Sodor = Σ(Eodor_j × wj), and the sound wave field convergence quantity is Sacous = Σ(Eacous_j × wj). The field-state mutual interference correlation processing combines the two into a binary tuple (Sodor, Sacous), which, together with the seed point coordinates, constitutes the mutual interference inference point record.

[0066] Step S148: Traverse all physical field state interference seed points in the olfactory-auditory collaborative scene graph, and perform the interference neighborhood window construction process, the odor field state point set and sound wave field state point set extraction process, the field state attenuation process, the field state convergence and accumulation process, the field state interference association process, and the interference inference point record generation process for each physical field state interference seed point to obtain the olfactory-auditory interference inference result. The olfactory-auditory interference inference result is composed of the interference inference point records of all physical field state interference seed points.

[0067] Step S149: Map the spatial coordinates of each interference simulation point recorded in the visual-tactile interference simulation results and the spatial coordinates of each interference simulation point recorded in the olfactory-auditory interference simulation results to a unified spatial grid coordinate system, and construct a spatial grid array covering the disaster site in the unified spatial grid coordinate system. The spatial grid array consists of multiple spatial grid nodes with a unified grid size.

[0068] The unified spatial grid coordinate system is a regular grid defined on a two-dimensional horizontal plane at the disaster site, with grid cell side length Lgrid. The center point of each grid cell is a spatial grid node. The mutual interference inference results from visual and tactile perception and olfactory and auditory perception are recorded and assigned to the corresponding spatial grid nodes according to their spatial coordinates.

[0069] Step S1410: Perform visual-tactile field-state component mapping processing on each spatial grid node in the spatial grid array. Average aggregation of the visual field-state convergence values ​​recorded by all interfering simulation points falling within the range of the spatial grid node in the visual-tactile interferometry simulation results is performed to obtain the visual field-state convergence component of the spatial grid node. Average aggregation of the terrain field-state convergence values ​​recorded by all interfering simulation points falling within the range of the spatial grid node in the visual-tactile interferometry simulation results is also performed to obtain the terrain field-state convergence component of the spatial grid node.

[0070] For a spatial grid node g, if there are Nvis visual-tactile inter-perturbation inference point records within the coverage area of ​​the node, then the visual field convergence component Vvis=(1 / Nvis)×ΣSvis_i and the terrain field convergence component Vter=(1 / Nvis)×ΣSter_i. Summation is performed to traverse all inter-perturbation inference point records within the coverage area of ​​the node.

[0071] Step S1411: Perform olfactory and auditory field state component mapping processing on each spatial grid node in the spatial grid array. Average aggregation of the odor field state convergence amounts recorded by all mutual interference simulation points falling within the range of the spatial grid node in the olfactory and auditory mutual interference simulation results is performed to obtain the odor field state convergence component of the spatial grid node. Average aggregation of the sound wave field state convergence amounts recorded by all mutual interference simulation points falling within the range of the spatial grid node in the olfactory and auditory mutual interference simulation results is also performed to obtain the sound wave field state convergence component of the spatial grid node.

[0072] For a spatial grid node g, if there are Nodor olfactory and auditory interference inference points recorded within the coverage area of ​​the node, then the odor field convergence component Vodor = (1 / Nodor) × ΣSodor_i, and the sound wave field convergence component Vacous = (1 / Nodor) × ΣSacous_i.

[0073] Step S1412: Perform field state component splicing processing on each spatial grid node in the spatial grid array, splicing the visual field state convergence component, terrain field state convergence component, odor field state convergence component, sound wave field state convergence component, visual field state intensity value of the visually salient region segment corresponding to the spatial grid node, and terrain field state intensity value of the terrain stiffness jump boundary segment corresponding to the spatial grid node along the component channel dimension to generate the cooperative perception situation component of the current time section of the spatial grid node.

[0074] The collaborative sensing situational component is a six-dimensional vector [Vvis, Vter, Vodor, Vacos, V5, V6], where V5 is the visual field intensity value Evis of the visually salient region segment at the node's location, and V6 is the terrain field intensity value Eter of the terrain stiffness jump boundary segment at the node's location. If a node has no corresponding segment at its location, the corresponding component is set to zero. The concatenation operation links the six scalar values ​​end-to-end along the component channel dimension to form a vector of length 6.

[0075] Step S1413: Perform time-series arrangement processing on the collaborative sensing situation components of the same spatial grid node in continuous time sections on the sampling time axis to obtain the evolution sequence of the collaborative sensing situation components of the spatial grid node. The evolution sequence of the collaborative sensing situation components arranges the collaborative sensing situation components of the spatial grid node in each time section in order from early to late according to the timestamp.

[0076] Traverse all spatial grid nodes in the spatial grid array, and perform the visual-tactile field-state component mapping processing, the olfactory-auditory field-state component mapping processing, the field-state component splicing processing, and the temporal arrangement processing on each spatial grid node to generate a global collaborative perception situation projection map. The global collaborative perception situation projection map contains the evolution sequence of the collaborative perception situation components corresponding to each of the spatial grid nodes at the disaster site.

[0077] The data structure of the global collaborative sensing situation projection map is a three-dimensional array. The first dimension is the row index of the spatial grid node, the second dimension is the column index of the spatial grid node, and the third dimension is the time section index. Each element in the array is a six-dimensional collaborative sensing situation component vector.

[0078] Step S150: Determine the spatial coordinates of the disaster spread source and the front line of the evolution trend of the disaster spread path based on the global collaborative perception situational inference map, and determine the current pose state information of each bionic robot body. Generate a distributed rescue collaborative instruction set based on the spatial coordinates of the disaster spread source, the front line of the evolution trend and the current pose state information of each bionic robot body.

[0079] Step S151: Extract the collaborative perception situation component of the current time segment from the evolution sequence of the collaborative perception situation component of each spatial grid node in the global collaborative perception situation deduction diagram, and parse the visual field convergence component, terrain field convergence component, odor field convergence component and sound wave field convergence component of the spatial grid node in the current time segment from the collaborative perception situation component of the current time segment.

[0080] The current time slice is the time slice with the latest timestamp in the evolution sequence of the collaborative sensing situation components. The six-dimensional collaborative sensing situation component vector [Vvis, Vter, Vodor, Vacos, V5, V6] of each spatial grid node of this slice is decomposed into dimensions, and the first four dimension components Vvis, Vter, Vodor, and Vacos are taken as the basic field state values ​​for subsequent fluctuation analysis.

[0081] Step S152: Perform field state fluctuation amplitude calculation processing on the visual field state convergence component and terrain field state convergence component of each spatial grid node in the current time section. Compare the visual field state convergence component of the current time section with the visual field state convergence component of the previous adjacent time section, calculate the time difference amplitude of the visual field state convergence component as the visual fluctuation amplitude, compare the terrain field state convergence component of the current time section with the terrain field state convergence component of the previous adjacent time section, calculate the time difference amplitude of the terrain field state convergence component as the terrain fluctuation amplitude, and use the sum of the visual fluctuation amplitude and the terrain fluctuation amplitude as the visual tactile fluctuation amplitude.

[0082] The preceding adjacent time segment is the time segment immediately preceding the current segment in the evolution sequence of the collaborative perception situational components. Visual fluctuation amplitude ΔVvis = |Vvis(t)−Vvis(t−1)|, terrain fluctuation amplitude ΔVter = |Vter(t)−Vter(t−1)|, and visual-tactile fluctuation amplitude ΔVvis-ter = ΔVvis + ΔVter. A larger visual-tactile fluctuation amplitude indicates more drastic changes in the visual and tactile perception field at that spatial grid node in the time dimension, and more active disaster diffusion activities.

[0083] Step S153: Perform field state fluctuation amplitude calculation processing on the odor field state convergence component and the sound wave field state convergence component of each spatial grid node in the current time section. Compare the odor field state convergence component of the current time section with the odor field state convergence component of the previous adjacent time section, and calculate the time difference amplitude of the odor field state convergence component as the odor fluctuation amplitude. Compare the sound wave field state convergence component of the current time section with the sound wave field state convergence component of the previous adjacent time section, and calculate the time difference amplitude of the sound wave field state convergence component as the sound wave fluctuation amplitude. The sum of the odor fluctuation amplitude and the sound wave fluctuation amplitude is used as the olfactory-auditory fluctuation amplitude. Odor fluctuation amplitude ΔVodor=|Vodor(t)−Vodor(t−1)|, sound wave fluctuation amplitude ΔVacous=|Vacous(t)−Vacous(t−1)|. Olfactory-auditory fluctuation amplitude ΔVodor-acous=ΔVodor+ΔVacous.

[0084] Step S154: Perform fluctuation amplitude fusion processing on the visual tactile fluctuation amplitude and olfactory and auditory fluctuation amplitude of each spatial grid node, take the larger of the visual tactile fluctuation amplitude and olfactory and auditory fluctuation amplitude as the cooperative fluctuation amplitude of the spatial grid node, and mark the fluctuation direction vector of the cooperative fluctuation amplitude on the spatial grid node pointing to the next adjacent time section.

[0085] The co-oscillation amplitude ΔV = max(ΔVvis-ter, ΔVodor-acous). The oscillation direction vector is the unit direction vector from the node to the neighboring node with the largest co-oscillation amplitude in its eight neighborhoods.

[0086] Step S155: Construct a spatial distribution map of field state fluctuations in the current time section based on the coordinated fluctuation amplitudes of all spatial grid nodes. Search for the spatial grid node with the largest coordinated fluctuation amplitude in the spatial distribution map as a candidate node for disaster spread source. Determine the spatial coordinates of the candidate node for disaster spread source as the spatial coordinates of disaster spread source.

[0087] The spatial distribution map of field fluctuations is a two-dimensional matrix, where each element represents the cooperative fluctuation amplitude ΔV of each spatial grid node. A global maximum search algorithm is used to traverse all matrix elements to find the spatial grid node corresponding to the element with the largest ΔV value. The spatial grid center coordinates of this node are the spatial coordinates of the disaster diffusion source.

[0088] Step S156: Starting from the spatial grid node corresponding to the spatial coordinates of the disaster spread source, track the adjacent spatial grid nodes pointed to by the fluctuation direction vector of the spatial grid node one by one, and record the spatial coordinates of each spatial grid node passed in sequence on the tracking path in the tracking order to obtain the front line of the evolution trend of the disaster spread path.

[0089] The tracking employs a greedy step-by-step strategy: starting from the current node, it moves along its fluctuation direction vector to the next node, adds the next node to the tracking path, and repeats this process starting from the new node. Tracking terminates when the amplitude of the current node's co-fluctuation is lower than a preset termination threshold Tstop, or when the number of tracking steps reaches a preset maximum number of tracking steps Mmax. The spatial coordinate sequence of all nodes on the tracking path forms the leading edge of the evolutionary trend.

[0090] Step S157: The spatial distribution map of field fluctuations, spatial coordinates of disaster spread sources, and position lines of evolution trend fronts of the current time segment are associated and stored in the disaster situation evolution record. As the acquisition time segment progresses, the collaborative sensing situation component analysis processing, the field fluctuation amplitude calculation processing, the fluctuation amplitude fusion processing, the disaster spread source spatial coordinate determination processing, and the tracking processing are repeatedly performed on each new time segment to obtain updated disaster spread source spatial coordinates and updated evolution trend front lines.

[0091] Step S158: Obtain the body posture data stream of each bionic robot body from the bionic motion posture sensing unit deployed on each bionic robot body. The body posture data stream includes the torso spatial orientation data and bionic limb joint angle data of each bionic robot body. Extract the evolution sequence of the collaborative perception situational component of each bionic robot body from the global collaborative perception situational projection map. Determine the local visual-tactile field state component and the local olfactory-auditory field state component of the current location of each bionic robot body based on the evolution sequence of the collaborative perception situational component of each bionic robot body.

[0092] The torso spatial orientation data represents the direction vector of the torso's principal axis in the global coordinate system, and the bionic limb joint angle data represents the angle values ​​of each moving leg joint. The spatial grid node where the bionic robot body is located is determined by projecting the global positioning coordinates of the bionic robot body onto the spatial grid array. The local visual and tactile field state components are taken from the first two dimensions of the collaborative perception situation component of the current time section of the node, and the local olfactory and auditory field state components are taken from the third and fourth dimensions.

[0093] Step S159: Perform orientation field state mapping processing on the torso spatial orientation data of each bionic robot body and the local visual-tactile field state component of the position of the bionic robot body. Transform the torso spatial orientation data to the spatial coordinate reference system where the local visual-tactile field state component is located, and calculate the angle between the torso spatial orientation and the field state gradient of the visual-tactile field state component as the orientation field state deviation angle. Perform limb tactile mapping processing on the bionic limb joint angle data of each bionic robot body and the local olfactory-auditory field state component of the position of the bionic robot body. Project the spatial position coordinates of each bionic limb end in the bionic limb joint angle data onto the spatial grid node where the local olfactory-auditory field state component is located, and extract the olfactory-auditory field state component of the spatial grid node projected by each bionic limb end as the limb tactile perception component.

[0094] The field state gradient of the visual-tactile field state component is obtained by calculating the spatial difference between Vvis and Vter at the current node and its neighboring nodes. The orientation field state deviation angle is the angle between the torso orientation vector and the field state gradient vector. The limb tactile mapping is performed by calculating the spatial coordinates of each foot end through the forward kinematics model of each bionic robot, projecting them onto the spatial grid node, and then reading the Vodor and Vacous components of that node as the limb tactile perception component.

[0095] Step S1510: Collect the torso spatial orientation data, bionic limb joint angle data, orientation field deviation angle, and limb tactile perception components of each bionic robot body according to the bionic robot body identifier, and construct the body pose perception description set of each bionic robot body in the current time segment. Summarize the body pose perception description sets of all bionic robot bodies in the bionic robot cluster to obtain the cluster pose state set in the current time segment.

[0096] Step S1511: Compare the pose change of the cluster pose state set of the current time segment with the pose change of the cluster pose state set of the previous adjacent time segment, calculate the change in spatial positioning coordinates and the change in torso orientation angle of each bionic robot body, and generate the current pose state information of each bionic robot body. The current pose state information includes the spatial position coordinate change vector and the torso orientation change rate.

[0097] The spatial position coordinate change vector is the vector difference between the current sectional coordinates and the previous sectional coordinates. The rate of change of torso orientation is the quotient of the change in torso orientation angle divided by the time interval between the two sectional planes.

[0098] For example, a distributed rescue coordination instruction set is generated based on the spatial coordinates of the disaster spread source, the position line of the evolution trend front, and the current pose state information of each bionic robot body. This includes: Step S1512: Using the spatial coordinates of the disaster spread source as the radiation center and the position line of the evolution trend front as the diffusion direction reference axis, search and rescue task allocation sectors are divided within the disaster site space. A sector identifier and sector boundary coordinate sequence are set for each search and rescue task allocation sector. The spatial position coordinates corresponding to the spatial position coordinate change vector in the current pose state information of each bionic robot body are spatially matched with the sector boundary coordinate sequence of the search and rescue task allocation sector to determine the sector identifier of the search and rescue task allocation sector to which each bionic robot body belongs.

[0099] Centered on the spatial coordinates of the disaster spread source, and with the direction of the evolution trend front line as the 0-degree reference axis, the perimeter is divided into Ns equal sectors at fixed angular intervals Δθ, where Ns = 2π / Δθ (rounded down). The sector boundary of each sector is enclosed by two rays emanating from the center point along the sector boundary direction and a preset sector radius Rsec. Spatial attribution matching determines which sector's boundary range the bionic robot's spatial coordinates fall within.

[0100] Step S1513: Based on the sector identifier and evolution trend front position line of the search and rescue task allocation sector to which each bionic robot body belongs, generate a sequence of search and rescue waypoints for each bionic robot body, starting from the current spatial position coordinates and advancing in a direction perpendicular to the evolution trend front position line within the search and rescue task allocation sector. The search and rescue waypoint sequence includes at least one search and rescue waypoint and its corresponding target spatial coordinates. Perform waypoint transition time allocation processing between adjacent search and rescue waypoints in the search and rescue waypoint sequence of each bionic robot body, and calculate the estimated movement time of the bionic robot body between two adjacent search and rescue waypoints based on the rate of change of the torso orientation of the bionic robot body and the joint bending angle of each bionic limb joint.

[0101] The search and rescue waypoint sequence is equidistantly arranged along the radial direction of the sector perpendicular to the leading edge of the evolution trend. The distance between adjacent waypoints is the distance traveled by the bionic robot in a single gait cycle. The estimated movement time is calculated by dividing the spatial distance between adjacent waypoints by the movement speed of the bionic robot at its current position. The movement speed is estimated from its gait parameters and terrain conditions.

[0102] Step S1514: Perform cooperative coverage analysis on the search and rescue waypoint sequences of adjacent bionic robot bodies within the same search and rescue task allocation sector. When the search and rescue waypoints of two bionic robot bodies overlap on the same spatial grid node, group the two bionic robot bodies together to generate cooperative formation configuration parameters including formation following distance and formation arrangement. Package the sector identifier, search and rescue waypoint sequence, task execution timing constraint parameters, and cooperative formation configuration parameters of each bionic robot body according to the bionic robot body identifier to generate the rescue task instruction unit of that bionic robot body. Traverse all bionic robot bodies to generate each rescue task instruction unit, summarize them to form a distributed rescue cooperative instruction set, and distribute it to each bionic robot body.

[0103] For example, the method may further include: step S210: extracting the collaborative perception situation components of each spatial grid node in the current time segment from the evolution sequence of the collaborative perception situation components of each spatial grid node in the global collaborative perception situation deduction diagram; performing dual-channel separation of the collaborative perception situation components of each spatial grid node according to the visual-tactile field state components and the olfactory-auditory field state components; arranging the visual-tactile field state components of each spatial grid node along the spatial position of the spatial grid node to form the visual-tactile field state spatial matrix of the current time segment; and arranging the olfactory-auditory field state components of each spatial grid node along the spatial position of the spatial grid node to form the olfactory-auditory field state spatial matrix of the current time segment.

[0104] The visual-tactile field state components are obtained by summing the convergent components of the visual field state and the convergent components of the terrain field state in the six-dimensional vector of the collaborative perception situational component. The olfactory-auditory field state components are obtained by summing the convergent components of the odor field state and the convergent components of the sound wave field state. The visual-tactile field state component values ​​of all spatial grid nodes are arranged into a two-dimensional matrix with the same shape as the spatial grid array according to the node row index and column index, forming the visual-tactile field state spatial matrix. The row and column indices of the matrix elements correspond to the spatial position coordinates of the spatial grid nodes. The olfactory-auditory field state spatial matrix is ​​constructed in the same way, and the matrix elements are the olfactory-auditory field state component values ​​of each node.

[0105] Step S211: Perform field state valley and ridge structure detection processing on the visual tactile field state space matrix. Scan the variation curves of visual tactile field state components along each row and column of the visual tactile field state space matrix. Mark the spatial locations where local minima appear in the variation curves of visual tactile field state components as field state valley points and mark the spatial locations where local maxima appear as field state ridge points. Obtain the visual tactile field state valley line formed by connecting all field state valley points and the visual tactile field state ridge line formed by connecting all field state ridge points.

[0106] For each row of the matrix, the visual-tactile field component values ​​of all columns in that row are arranged into a one-dimensional variation curve by column index. The curve is slid across a preset window length, comparing the value of the central element with the values ​​of other elements within each window. If the central element value is the minimum within the window and lower than the window mean minus the window standard deviation, the corresponding spatial grid node is marked as a field valley. If the central element value is the maximum within the window and higher than the window mean plus the window standard deviation, it is marked as a field ridge. The same operation is repeated for each column. All detected field valleys in rows and columns are connected into lines based on spatial eight-neighbor connectivity, forming visual-tactile field valley lines. These visual-tactile field valley lines represent low-value channels in space for visual and tactile perception fields, reflecting possible confluence paths for disaster diffusion characteristics. Similarly, field ridges are connected to form visual-tactile field ridge lines, representing high-value barriers and reflecting the boundary locations where disaster diffusion is hindered.

[0107] Step S212: Perform field state valley and ridge structure detection processing on the olfactory and auditory field state space matrix. Scan the variation curves of the olfactory and auditory field state components along each row and column of the olfactory and auditory field state space matrix. Mark the spatial locations where local minima appear in the variation curves of the olfactory and auditory field state components as field state valley points and mark the spatial locations where local maxima appear as field state ridge points. Obtain the olfactory and auditory field state valley line formed by connecting all field state valley points and the olfactory and auditory field state ridge line formed by connecting all field state ridge points.

[0108] Using the same valley-ridge detection method as in step S211, the processing objects are replaced with the rows and columns of the olfactory-auditory field state spatial matrix. The resulting olfactory-auditory field state valleys represent the low-value confluence channels of odor and sound wave perception fields in space, while the olfactory-auditory field state ridges represent the high-value diffusion barrier boundaries of odor and sound wave fields.

[0109] Step S213: Perform spatial superposition and comparison processing on the visual-tactile field valley line and the olfactory-auditory field valley line, extract the overlapping segment of the valley line where the visual-tactile field valley line and the olfactory-auditory field valley line coincide in spatial position, perform field gradient direction comparison analysis on each spatial grid node on the overlapping segment of the valley line, and calculate the angle between the gradient direction vector of the visual-tactile field state component and the gradient direction vector of the olfactory-auditory field state component at the spatial grid node.

[0110] Two valley lines are plotted on the same spatial coordinate grid. The nodes that the two valley lines pass through simultaneously are retrieved; these are the nodes where the positions of the two valley lines coincide within a preset spatial tolerance range. These consecutive coinciding nodes constitute the valley line overlap segment. For each node on the valley line overlap segment, the gradient direction vector of the visual-tactile field component is calculated. This is done by finding the neighborhood direction with the largest difference in visual-tactile field component values ​​among the eight neighboring grid nodes of that node. The unit vector pointing from that node to that neighboring node is the gradient direction vector. The gradient direction vectors of the olfactory-auditory field components are calculated similarly. The angle between the two gradient direction vectors is calculated by dividing their dot product by their magnitude product and then taking the inverse cosine function. The smaller the angle, the more consistent the changes in the two field states at the overlapped valley line, and the more pronounced the transmission trend of disaster spread along that overlapped valley line.

[0111] Step S214: Select spatial grid nodes with directional angles less than a preset valley line direction synchronization threshold from the overlapping valley line segments as disaster source confluence channel nodes. Connect consecutive disaster source confluence channel nodes sequentially along the extension direction of the overlapping valley line segments to form disaster source confluence channels. Determine the disaster diffusion barrier line based on the visual-tactile field ridge line and the olfactory-auditory field ridge line. Perform safety constraint correction on the distributed rescue collaborative instruction set based on the spatial positional relationship between the disaster source confluence channels and the disaster diffusion barrier line.

[0112] A preset valley-line direction synchronization threshold is used to determine whether the two field states truly coordinate at the overlapping valley lines. The selected nodes indicate that visual-tactile anomalies and olfactory-auditory anomalies converge in the same direction at that location, representing the most likely path for the spread of hazardous materials. These nodes are connected according to their spatial adjacency to form the hazardous source confluence channel. The hazardous spread barrier line is formed by connecting nodes where the visual-tactile field state ridge line and the olfactory-auditory field state ridge line spatially coincide, and the angle between the gradient directions of the two field states at the ridge line nodes is less than the preset ridge-line direction synchronization threshold. This represents a field state barrier boundary that hazardous materials are unlikely to cross.

[0113] Step S2141: Perform spatial superposition and comparison processing on the visual-tactile field ridge line and the olfactory-auditory field ridge line, extract the overlapping segments of the ridge lines that coincide in spatial position, perform field gradient direction comparison analysis on each spatial grid node on the overlapping segments, and calculate the angle between the gradient direction vector of the visual-tactile field component and the gradient direction vector of the olfactory-auditory field component at the spatial grid node.

[0114] The visual-tactile field ridge and the olfactory-auditory field ridge are superimposed on the same spatial coordinate grid. Spatial grid nodes that the two ridges pass through within a preset spatial tolerance range are retrieved. These consecutive overlapping nodes constitute overlapping ridge segments. For each node on the overlapping ridge segment, the direction of the maximum difference between its visual-tactile field components in its eight-neighbor direction is calculated as the visual-tactile gradient direction vector, and the direction of the maximum difference between its olfactory-auditory field components in its eight-neighbor direction is calculated as the olfactory-auditory gradient direction vector. The angle between the two gradient direction vectors is obtained using the inverse cosine function.

[0115] Step S2142: Select spatial grid nodes with directional angles less than a preset ridge direction synchronization threshold from the overlapping ridge segments as disaster diffusion barrier nodes, and connect the continuous disaster diffusion barrier nodes sequentially along the extension direction of the overlapping ridge segments to form a disaster diffusion barrier line. The disaster diffusion barrier line represents the field state barrier boundary that prevents the disaster diffusion material from spreading outward.

[0116] A preset ridge direction synchronization threshold is used to determine whether two field states truly have a consistent blocking effect at the overlapping ridge line location. Nodes with an angle between their directions less than this threshold indicate that the visual-tactile field state and the olfactory-auditory field state synchronously form a diffusion barrier at that location. Connecting these nodes according to their spatial adjacency forms the disaster diffusion barrier line, making it difficult for disaster-spreading substances to cross this barrier line and spread outward.

[0117] Step S2143: Determine the safe area of ​​the rescue channel based on the spatial relationship between the disaster source confluence channel and the disaster diffusion barrier line. Define the spatial grid nodes located on the side of the disaster diffusion barrier line away from the disaster source confluence channel and whose spatial distance from the disaster diffusion barrier line exceeds the preset safe buffer distance as safe nodes of the rescue channel. Connect all the safe nodes of the rescue channel along the extension direction of the front line of the evolution trend of the disaster diffusion path to form a safe rescue channel corridor.

[0118] The disaster source confluence channel is the confluence path of disaster diffusion materials obtained in step S214, and the disaster diffusion barrier line is the field-state barrier boundary that hinders disaster diffusion. The safe zone should be located on the side of the diffusion barrier line away from the confluence channel, that is, the area where the impact of disaster diffusion is minimal after being blocked by the barrier line. The safe buffer distance is the additional safety margin space reserved. Nodes that meet the conditions are connected to form a rescue safety corridor, which is the safe activity space for rescue operations.

[0119] Step S2144: Using the rescue safety passage corridor as the constraint boundary, perform safety constraint correction processing on the search and rescue waypoint sequence of each bionic robot body in the distributed rescue collaborative instruction set. Project the search and rescue waypoints in the search and rescue waypoint sequence of each bionic robot body that fall outside the rescue safety passage corridor onto the safety passage boundary node closest to the search and rescue waypoint within the rescue safety passage corridor, to obtain the search and rescue waypoint sequence after safety constraint correction.

[0120] For each search and rescue waypoint of each biomimetic robot body, determine whether its spatial coordinates are within the boundary range of the rescue safety corridor. If they are within the corridor range, they remain unchanged. If they are outside the corridor range, calculate the Euclidean distance from the waypoint to all nodes on the corridor boundary, find the nearest corridor boundary node, and replace the waypoint coordinates with the coordinates of that boundary node. After processing, the search and rescue waypoint sequence after safety constraint correction is obtained.

[0121] Step S310: Extract the collaborative perception situation components of all time sections in the evolution sequence of the collaborative perception situation components of each spatial grid node from the global collaborative perception situation projection map. Construct a collaborative perception situation component time series vector along the time dimension for each spatial grid node. The collaborative perception situation component time series vector includes the visual tactile field state component, terrain field state intensity value, odor field state intensity value and sound wave field state intensity value of the spatial grid node in each time section.

[0122] For each spatial grid node, its six-dimensional collaborative sensing situational components across all time planes are arranged into a matrix in temporal order. Four field-state dimensions are extracted to form a time-series vector, which consists of the visual-tactile field-state component, the terrain field-state intensity value, the odor field-state intensity value, and the sound wave field-state intensity value. This time-series vector is used to analyze the fluctuation rhythm characteristics of each field-state component over time.

[0123] Step S311: Perform field state fluctuation rhythm extraction processing on the time series vector of the collaborative sensing situation component for each spatial grid node. Perform spectral decomposition on the time series of the visual-tactile field state component in the collaborative sensing situation component time series vector, and extract the dominant frequency component of the visual-tactile field state component as the visual-tactile dominant rhythm. Perform spectral decomposition on the time series of the olfactory-auditory field state component, and extract the dominant frequency component of the olfactory-auditory field state component as the olfactory-auditory dominant rhythm.

[0124] The visual-tactile field-state component time series is a one-dimensional time-domain sequence formed by summing the convergent components of the visual field state and the convergent components of the topographic field state at each time section. A Fast Fourier Transform is performed on this one-dimensional time-domain sequence to obtain the spectrum. The frequency component with the largest amplitude in the spectrum is taken as the dominant visual-tactile frequency. The reciprocal of this dominant visual-tactile frequency is the dominant visual-tactile precession period, and the corresponding phase angle is the dominant visual-tactile rhythm phase. The olfactory-auditory field-state component time series is formed by summing the convergent components of the odor field state and the convergent components of the sound wave field state. The same processing is performed to obtain the dominant olfactory-auditory frequencies and the dominant olfactory-auditory rhythm phases. The dominant frequency reflects the most important rhythmic period of the physical field state fluctuations at that spatial location and is used to analyze the periodic behavior of disaster spread.

[0125] Step S312: Perform rhythm phase synchronization analysis on the visual-tactile dominant rhythm and the olfactory-auditory dominant rhythm of each spatial grid node, calculate the phase difference between the phase angle of the visual-tactile dominant rhythm and the phase angle of the olfactory-auditory dominant rhythm in the same time plane, and take the change sequence of the phase difference in the continuous time plane as the cross-field rhythm synchronization phase offset sequence of the spatial grid node.

[0126] On the same time plane, instantaneous phase calculations are performed at each time point using the dominant frequencies of visual-tactile and olfactory-auditory perceptions. The difference between these two calculations represents the cross-field rhythm phase difference for that time plane. Arranging the phase differences from each time plane sequentially forms a cross-field rhythm synchronization phase shift sequence. This sequence reflects the degree of rhythmic synchronization between the visual-tactile and olfactory-auditory fields in the time dimension. A more stable phase difference indicates greater synchronization between the two field rhythms, suggesting a more stable disaster spread at that location. Larger fluctuations in the phase difference indicate a loss of synchronization between the two field rhythms, potentially indicating a disaster spread front or disturbance zone.

[0127] Step S313: Perform spatial clustering on the cross-field rhythmic synchronization phase offset sequences of all spatial grid nodes, group spatial grid nodes that are spatially adjacent and whose phase difference changes in the cross-field rhythmic synchronization phase offset sequences have the same trend into the same rhythmic synchronization cluster, and assign a rhythmic synchronization cluster identifier to each rhythmic synchronization cluster.

[0128] Spatial clustering employs a region growing algorithm. For each unassigned cluster node, a new cluster is created as a seed point. Its four neighboring grid nodes are checked. If the absolute difference in phase between a neighboring node and the seed point on the same time plane is less than a preset phase synchronization tolerance, and this condition is met on multiple consecutive planes, then the neighboring nodes are merged into the current cluster, and growth continues outwards. After growth is complete, a unique identifier is assigned to the cluster.

[0129] Step S314: Calculate the cluster average phase offset of the cross-field rhythmic synchronization phase offset sequence of all spatial grid nodes in each rhythmic synchronization cluster. Arrange the cluster average phase offset of each rhythmic synchronization cluster in the order of time section to obtain the cluster-level phase offset time evolution line of each rhythmic synchronization cluster. Based on the cluster-level phase offset time evolution line, identify the out-of-phase rhythmic clusters. According to the identification results, perform rhythmic mutation and anomaly constraint mapping processing on the search and rescue waypoint sequence of each bionic robot body.

[0130] Step S3141: Calculate the cluster average phase shift of the cross-field rhythm synchronization phase shift sequence of all spatial grid nodes in each rhythm synchronization cluster, arrange the cluster average phase shift of each rhythm synchronization cluster in time section order, and obtain the cluster-level phase shift time evolution line of each rhythm synchronization cluster.

[0131] For each rhythm synchronization cluster, the arithmetic mean of the cross-field rhythm synchronization phase offset values ​​of all spatial grid nodes within the cluster on each time plane is calculated to obtain the cluster-average phase offset of the cluster on that plane. Arranging the cluster-average phase offsets of each time plane in chronological order forms the cluster-level phase offset time evolution line of the rhythm synchronization cluster.

[0132] Step S3142: Analyze the inter-cluster phase deviation degree of the cluster-level phase offset time evolution line of different rhythmic synchronization clusters, calculate the absolute value of the difference between the average cluster phase offset of any two rhythmic synchronization clusters in the same time section as the inter-cluster phase deviation degree, and mark the two rhythmic synchronization clusters whose inter-cluster phase deviation degree exceeds the preset phase deviation threshold as out-of-phase rhythmic cluster pairs.

[0133] The larger the absolute value of the difference in the average phase offset between two clusters on the same time plane, the more asynchronous the field rhythms of the two clusters are on that plane. If the difference exceeds the threshold on multiple consecutive planes, the two clusters are determined to be a pair of out-of-phase rhythm clusters.

[0134] Step S3143: Extract the spatial range boundaries of the two rhythm synchronization clusters belonging to the same heterophase rhythm cluster pair from the global collaborative perception situation inference map, connect the adjacent boundary space grid nodes between the spatial range boundaries of the two rhythm synchronization clusters to form a transition zone between heterophase rhythm clusters. The transition zone between heterophase rhythm clusters represents the spatial transition region where the physical field state of the disaster site undergoes abrupt changes in the rhythm dimension.

[0135] The spatial boundaries of the two rhythm synchronization clusters are closed contours formed by the outermost grid nodes of each cluster. Between the two contours, one node from each of the adjacent grid node pairs is taken and connected to form a line segment. Multiple adjacent line segments are connected to form a transition zone between the out-of-phase rhythm clusters.

[0136] Step S3144: Calculate the spatial distance between the spatial position coordinates of the transition zone between the heterogeneous rhythm clusters and the spatial position coordinate change vector in the current pose state information of each bionic robot body. Mark the bionic robot body whose next expected position pointed to by the spatial position coordinate change vector falls within the range of the transition zone between the heterogeneous rhythm clusters as a rhythm mutation risk body.

[0137] The spatial position coordinate change vector is obtained by adding the spatial position coordinate change vector to the current spatial coordinates of the bionic robot body to obtain the next expected position coordinates. If the coordinates are located within the spatial range of the transition zone between heterogeneous rhythm clusters, it is marked as a body at risk of rhythmic abrupt change.

[0138] Step S3145: Perform rhythm mutation and anomaly constraint mapping processing on the search and rescue waypoint sequence of the bionic robot body marked as the rhythm mutation and anomaly risk body. Insert the spatial position coordinates of the transition zone between the out-of-phase rhythm clusters into the original search and rescue waypoint sequence as the rhythm mutation early warning waypoint. Set the bionic limb joint angle adjustment parameters and trunk spatial orientation adjustment parameters for the bionic robot body at the rhythm mutation early warning waypoint.

[0139] The bionic limb joint angle adjustment parameters enable the bionic robot to reduce the stride length and increase the foot support area when passing through the transition zone. The trunk spatial orientation adjustment parameters keep the trunk orientation parallel to the direction of the transition zone to reduce cross-band truncation.

[0140] Step S410: Combine the collaborative perception situation components of each spatial grid node in the current time section of the global collaborative perception situation projection map with the spatial coordinates of the spatial grid node to construct a collaborative perception situation snapshot map of the current time section. Arrange the collaborative perception situation snapshot maps of multiple consecutive time sections in chronological order of the time sections to form a sequence of collaborative perception situation snapshot maps.

[0141] For the current time slice, the six-dimensional collaborative sensing situation component vectors of each spatial grid node in the global collaborative sensing situation projection map are combined with the spatial coordinates of that node into a recording unit. All recording units are arranged into a two-dimensional matrix according to the spatial grid nodes, with each matrix element containing spatial coordinates and a six-dimensional component vector, forming a snapshot of the collaborative sensing situation at the current time slice. The snapshots of the collaborative sensing situation at multiple consecutive time slices are arranged into a sequence according to the timestamp, with the index of the snapshot sequence being the time slice number, and each sequence element corresponding to the global state snapshot data of a time slice.

[0142] Step S411: Perform collaborative situation differential processing on the collaborative situation snapshot images of every two adjacent time segments in the collaborative situation snapshot image sequence, calculate the component change between the collaborative situation component of each spatial grid node in the later time segment and the collaborative situation component of the previous time segment, and obtain the collaborative situation differential vector of each spatial grid node. The collaborative situation differential vector includes the change of visual tactile field state component, the change of terrain field state intensity value, the change of odor field state intensity value, and the change of sound wave field state intensity value.

[0143] For the snapshot images of frame t and frame (t+1) in the snapshot image sequence, the cooperative sensing situational component vector of frame (t+1) is subtracted from the corresponding vector of frame t for each spatial grid node, resulting in the cooperative sensing situational difference vector of that node between the two time planes. The four dimensions of this cooperative sensing situational difference vector correspond to the changes in the convergent components of the visual field (ΔVvis), the topographic field (ΔVter), the odor field (ΔVodor), and the acoustic field (ΔVacous), respectively.

[0144] Step S412: Perform situation change direction encoding processing on the collaborative perception situation difference vector of each spatial grid node. Determine the visual-tactile joint change direction encoding based on the positive and negative signs of the changes in the visual-tactile field state component and the terrain field state intensity value. Determine the olfactory-auditory joint change direction encoding based on the positive and negative signs of the changes in the odor field state intensity value and the sound wave field state intensity value. Concatenate the visual-tactile joint change direction encoding and the olfactory-auditory joint change direction encoding to obtain the situation change direction joint encoding.

[0145] For the visual-tactile field state component change ΔVvis, if the value is positive, the direction code for that dimension is 1; if it is negative, the direction code is 0. The terrain field state intensity change ΔVter is encoded similarly. The two direction codes are concatenated to form the visual-tactile joint change direction code. The odor component change ΔVodor and the sound wave component change ΔVacous are encoded similarly to obtain the olfactory-auditory joint change direction code. The two codes are concatenated to form the situational change direction joint code.

[0146] Step S413: Arrange the joint encoding of the situation change direction of all spatial grid nodes according to the spatial position of the spatial grid nodes to construct the joint encoding map of the situation change direction of the current time section. Identify the connected spatial grid node regions with the same joint encoding of situation change direction in the joint encoding map of situation change direction to obtain multiple situation cooperative change blocks with the same situation change trend. Adjust the situation disturbance avoidance of the search and rescue waypoint sequence of each bionic robot body based on the predicted spatial boundary contour of the situation cooperative change blocks.

[0147] A four-neighbor connected component labeling algorithm is used to traverse the encoded graph, grouping spatially adjacent nodes with the same encoded value into the same connected component. Each connected component represents a situational coordination change block. Based on the predicted spatial boundary contours of these situational coordination change blocks, the search and rescue waypoint sequences of each biomimetic robot are adjusted to avoid situational disturbances.

[0148] Step S4131: Extract the spatial boundary contour of each situational change block in each time section from the sequence of collaborative sensing situational snapshot images. Perform trend extrapolation on the area change of the spatial boundary contour of each situational change block in the continuous time section to obtain the predicted spatial boundary contour of the situational change block in the subsequent time section.

[0149] For each situational change block, the spatial boundary nodes of its connected domains are extracted on each time plane and connected in clockwise order to form a closed contour line. The area enclosed by the contour line is calculated. The area values ​​of multiple consecutive time planes are arranged into an area time series, and the first-order difference of the area between adjacent planes is calculated. Linear regression is used to fit a straight line to the area values ​​of the K most recent planes, and the predicted area of ​​the next H planes is extrapolated. Based on the scaling ratio between the predicted area and the current contour area, the current boundary contour is scaled proportionally with the block center as the scaling origin to obtain the predicted spatial boundary contour of subsequent time planes.

[0150] Step S4132: Perform spatiotemporal collision prediction processing on the predicted spatial boundary contours of each situational coordination change block in the subsequent time section and the search and rescue waypoint sequence of each bionic robot body. When the target spatial coordinates of the search and rescue waypoint of a certain bionic robot body in a certain subsequent time section fall within the predicted spatial boundary contour of any situational coordination change block in that subsequent time section, mark the bionic robot body as a situational change disturbed body.

[0151] Step S4133: For the bionic robot body marked as a situational change disturbed body, determine the type of situational change disturbance experienced by the bionic robot body based on the joint encoding of the situational change direction of the situational cooperative change block into which it falls. Extract the visual-tactile field state component change trend and the olfactory-auditory field state component change trend from the cooperative perception situational component evolution sequence of the situational cooperative change block. Generate a situational disturbance avoidance direction vector for the bionic robot body based on the synthesis direction of the visual-tactile field state component change trend and the olfactory-auditory field state component change trend.

[0152] The high two bits of the joint encoding for the direction of situational change are the joint visual-tactile encoding, and the low two bits are the joint olfactory-auditory encoding, used to determine the type of disturbance. The visual-tactile and olfactory-auditory field-state components from the most recent segments of the collaborative sensing situational component evolution sequence of the block are fitted with their respective slopes. These two slopes form a trend vector, and the composite direction is determined by the direction of the sum of these two slope vectors. The situational disturbance avoidance direction vector is the opposite direction of this composite direction.

[0153] Step S4134: Spatially offset the original search and rescue waypoints in the search and rescue waypoint sequence of the bionic robot body along the situational disturbance avoidance direction vector, and shift the original search and rescue waypoints away from the predicted spatial boundary contour of the situational cooperative change block by a preset avoidance offset amount to obtain the situational disturbance avoidance search and rescue waypoint sequence of the bionic robot body.

[0154] For each search waypoint in the search and rescue waypoint sequence marked as a disturbance risk waypoint, the original waypoint spatial coordinates are translated by a preset avoidance offset along the situation disturbance avoidance direction vector generated in step S4133. This preset avoidance offset is taken as the predicted boundary contour radius of the situation cooperative change block in the translation direction multiplied by a safety factor. Search and rescue waypoints of non-disturbance risk waypoints retain their original coordinates. After translation, all waypoints are arranged in chronological order to form a situation disturbance avoidance search and rescue waypoint sequence. This situation disturbance avoidance search and rescue waypoint sequence ensures that the trajectory of the bionic robot avoids the area covered by the future expansion of the situation cooperative change block.

[0155] Step S4135: Replace the original search and rescue waypoint sequence corresponding to the distributed rescue coordination instruction set with the situation disturbance avoidance search and rescue waypoint sequence of all situation change disturbed entities, and attach the predicted spatial boundary contour of each situation coordination change block as the coordination perception situation warning boundary to the distributed rescue coordination instruction set.

[0156] For each bionic robot body marked as a situational change-affected body in the distributed rescue coordination instruction set, its situational perturbation avoidance search and rescue waypoint sequence is used to replace the search and rescue waypoint sequence in the original rescue mission instruction unit. The predicted spatial boundary contour data of all situational coordination change blocks are encapsulated into a collaborative perception situational warning boundary field and appended to the additional information segment of the distributed rescue coordination instruction set for continuous reference and avoidance by each bionic robot body during the execution of the rescue mission.

[0157] Based on the same inventive concept, please refer to Figure 2The diagram illustrates a schematic block diagram of the multimodal rescue system provided in this application embodiment, which includes a central processing unit, a system memory including random access memory and read-only memory, and a system bus connecting the system memory and the central processing unit. The emergency bionic robot collaboration system with task and status collaborative perception also includes a basic input / output system to facilitate information transmission between various devices within the computer, and a large-capacity storage device for storing the operating system, applications, and other program modules.

[0158] A basic input / output system includes a display for showing information and input devices such as a mouse and keyboard for user input. Both the display and the input devices are connected to the central processing unit via an input / output controller connected to the system bus. The basic input / output system may also include an input / output controller for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller also provides output to a display screen, printer, or other types of output devices.

[0159] A mass storage device is connected to the central processing unit via a mass storage controller connected to the system bus. The mass storage device and its associated computer-readable medium provide non-volatile storage for collaborative emergency bionic robots with task and state awareness. Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. According to various embodiments of this application, collaborative emergency bionic robots with task and state awareness can also operate by connecting to a remote computer on a network, such as the Internet. That is, collaborative emergency bionic robots with task and state awareness can connect to a network via a network interface unit connected to the system bus, or they can use the network interface unit to connect to other types of networks or remote computer systems.

[0160] In addition, in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0161] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. An emergency biomimetic robot collaborative method with task and state collaborative perception, characterized in that, The method includes: The environmental perception data stream is acquired by multi-source sensing and acquisition devices of each bionic robot body deployed in the emergency bionic robot cluster. The environmental perception data stream includes panoramic visual data stream, terrain stiffness distribution data stream, gas composition concentration data stream and sound wave spectrum data stream. Visual saliency region segmentation is performed on the panoramic visual data stream to obtain visual saliency region segments; stiffness jump boundary tracking is performed on the terrain stiffness distribution data stream to obtain terrain stiffness jump boundary segments; gas concentration front identification is performed on the gas composition concentration data stream to obtain gas concentration front movement trajectory segments; and anomalous frequency band separation is performed on the acoustic spectrum data stream to obtain anomalous acoustic frequency band change segments. The visually salient region segment and the terrain stiffness jump boundary segment are spatially superimposed to generate a visual-tactile collaborative scene map. The gas concentration front movement trajectory segment and the abnormal sound wave frequency band change segment are spatially superimposed to generate an olfactory-auditory collaborative scene map. Physical field state interference inference processing and cross-graph temporal correlation fusion processing are performed on the visual-tactile collaborative scene map and the olfactory-auditory collaborative scene map to generate a global collaborative perception situation inference map; Based on the global collaborative perception situational simulation map, the spatial coordinates of the disaster spread source and the front line of the evolution trend of the disaster spread path are determined, and the current pose state information of each bionic robot body is determined. Based on the spatial coordinates of the disaster spread source, the front line of the evolution trend and the current pose state information of each bionic robot body, a distributed rescue collaborative instruction set is generated.

2. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, The panoramic visual data stream is subjected to visual saliency region segmentation processing to obtain visual saliency region segments, including: Perform multi-scale visual region segmentation processing on each frame of panoramic visual image in the panoramic visual data stream to generate multiple visual region units. Each visual region unit is composed of all pixels within that visual region unit. For each visual region unit, a visual contrast calculation is performed to calculate the visual contrast value between each visual region unit and its neighboring visual region units. The visual contrast value is determined based on the difference between the brightness distribution of pixels in the visual region unit and the brightness distribution of pixels in the neighboring visual region units. A spatial distribution map of visual contrast is constructed based on the visual contrast values ​​corresponding to each visual region unit, and the height marker corresponding to the visual contrast value of each visual region unit is marked on the spatial distribution map of visual contrast. Visual region units whose visual contrast values ​​exceed a preset visual contrast limit are extracted from the visual contrast spatial distribution map as candidate visual saliency region units, and the pixel with the highest visual contrast value in each candidate visual saliency region unit is marked with a center seed point. Starting with each candidate visual saliency region unit carrying a center seed point marker, region growing is performed on adjacent visual region units. Adjacent visual region units whose visual contrast values ​​differ from the candidate visual saliency region unit by less than a preset growth tolerance are accepted as the same visual saliency connected region. The candidate visual saliency region unit is then merged with the accepted adjacent visual region units to obtain a visual saliency connected region. Contour extraction is performed on the obtained visual saliency connected region, and the outer boundary pixels of the visual saliency connected region are connected to form a closed contour line. The area enclosed by the closed contour line is used as the visual saliency region unit in the panoramic visual image of that frame. The multi-scale visual region segmentation process, the visual contrast calculation process, the candidate visual saliency region unit extraction process, the region growing process, and the contour extraction process are performed on all consecutive frames of the panoramic visual image in the panoramic visual data stream to obtain the visual saliency region unit corresponding to each frame of the panoramic visual image. The visual saliency region units with spatial overlapping relationships between consecutive frames are connected in time stamp order to form a visual saliency region segment that progresses continuously in the time dimension. Each time segment of the visual saliency region corresponds to a visual saliency region unit and its spatial position coordinates in the panoramic visual image.

3. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, Perform stiffness jump boundary tracing processing on the terrain stiffness distribution data stream to obtain terrain stiffness jump boundary segments, including: The stiffness response values ​​of each ground contact point are extracted from each frame of the terrain stiffness distribution map in the terrain stiffness distribution data stream. The stiffness response value of each ground contact point is mapped to the spatial coordinates of the ground contact point on the terrain stiffness distribution map to generate a terrain stiffness spatial distribution point matrix. Stiffness gradient calculation is performed on each ground contact point in the terrain stiffness spatial distribution point matrix to calculate the rate of change of stiffness response value between the ground contact point and its adjacent ground contact points, and a stiffness gradient vector corresponding to each ground contact point is generated. The direction of the stiffness gradient vector points to the direction of the fastest change in stiffness response value. Ground contact points whose stiffness gradient vector magnitude exceeds a preset stiffness abrupt change limit are selected from the spatial distribution point matrix of the terrain stiffness as stiffness jump candidate points. Each stiffness jump candidate point is assigned a jump seed point label. Taking each stiffness jump candidate point with a jump seed point label as the starting point, the stiffness gradient vectors of adjacent ground contact points are sequentially searched along the direction of the stiffness gradient vector of the stiffness jump candidate point. Adjacent ground contact points whose stiffness gradient vector direction is less than a preset direction tolerance and whose stiffness gradient vector magnitude exceeds the preset stiffness abrupt change limit are sequentially connected to form a stiffness jump chain. The stiffness jump boundary line is obtained by arranging all ground contact points in the stiffness jump chain in the connection order. The stiffness jump boundary line divides the terrain stiffness distribution map into a first terrain zone and a second terrain zone where the difference in stiffness response values ​​exceeds a preset stiffness separation threshold. For all consecutive frames of the terrain stiffness distribution map in the terrain stiffness distribution data stream, the stiffness response value extraction process of the ground contact point, the stiffness gradient calculation process, the stiffness jump candidate point screening process, and the stiffness jump chain connection process are performed to obtain the stiffness jump boundary line corresponding to each frame of the terrain stiffness distribution map. By connecting the stiffness jump boundary lines that have a spatial continuity relationship between consecutive frames in the order of timestamps, a terrain stiffness jump boundary segment that evolves continuously in the time dimension is formed. Each time section of the terrain stiffness jump boundary segment corresponds to a stiffness jump boundary line and its set of spatial position coordinates in the terrain stiffness distribution map.

4. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, The visually salient region segment and the terrain stiffness abrupt boundary segment are spatially superimposed to generate a visual-tactile collaborative scene map, including: The spatial coordinates of the visual saliency region units corresponding to each time section in the visual saliency region segment and the set of spatial coordinates of the stiffness jump boundary line corresponding to the same time section in the terrain stiffness jump boundary segment are mapped to the same spatial coordinate plane in a unified spatial reference system to obtain the spatially registered visual saliency region layer and terrain stiffness jump boundary layer. The regions enclosed by the closed contour lines of each visually salient region unit in the spatially registered visually salient region layer are subjected to region coverage marking processing, and spatial coordinate points falling within the closed contour lines of each visually salient region unit are assigned visual coverage markings in the unified spatial reference system; and the spatial coordinate points traversed by each stiffness jump boundary line in the spatially registered terrain stiffness jump boundary layer are subjected to boundary line marking processing, and spatial coordinate points traversed by each stiffness jump boundary line are assigned boundary line markings in the unified spatial reference system. In the unified spatial reference system, spatial coordinate points that simultaneously carry the visual overlay marker and the boundary line marker are determined as visual terrain-related points, spatial coordinate points that only carry the visual overlay marker and not the boundary line marker are determined as visually independent points, and spatial coordinate points that only carry the boundary line marker and not the visual overlay marker are determined as terrain-independent points. Multimodal association marker processing is performed on the visual terrain-related points, and the visual contrast value of the visual saliency region unit corresponding to the visual terrain-related point and the stiffness gradient vector magnitude of the ground contact point corresponding to the visual terrain-related point are recorded as the visual terrain dual-mode association marker of the visual terrain-related point. Visual independent field state assignment processing is performed on the visual independent point, and the visual field state intensity value of the visual independent point is set to the field state intensity value corresponding to the visual contrast value of the visual saliency region unit to which the visual independent point belongs. Terrain independent field state assignment processing is performed on the terrain independent point, and the terrain field state intensity value of the terrain independent point is set to the field state intensity value corresponding to the stiffness gradient vector magnitude of the ground contact point corresponding to the terrain independent point. The visual terrain associated points, the visual independent points carrying visual field intensity values, and the terrain independent points carrying terrain field intensity values ​​are merged into layers in the same spatial coordinate plane to generate a visual-tactile collaborative scene map with spatial coordinate points as basic units. Each spatial coordinate point in the visual-tactile collaborative scene map contains at least one of the following labeling information: visual field intensity label, terrain field intensity label, and visual-terrain associated intensity label.

5. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, Perform physical field state interference deduction processing on the visual-tactile collaborative scene graph, including: All spatial coordinate points carrying visual terrain association intensity markers are extracted from the visual-tactile collaborative scene graph as physical field state interference seed points. For each physical field state interference seed point, an interference neighborhood window is constructed with the seed point as its center and covering a preset interference radius. Within the interference neighborhood window, all spatial coordinate points carrying visual field state intensity markers within the interference neighborhood window are extracted from the visual-tactile collaborative scene graph to form a visual field state point set, and all spatial coordinate points carrying terrain field state intensity markers within the interference neighborhood window are extracted to form a terrain field state point set. The visual field intensity value of each spatial coordinate point in the visual field point set is subjected to field state attenuation processing. The attenuated visual field intensity contribution value is calculated based on the spatial distance between each spatial coordinate point and the physical field state interference seed point. The greater the spatial distance, the lower the visual field intensity contribution value of the spatial coordinate point. The terrain field intensity value of each spatial coordinate point in the terrain field state point set is subjected to field state attenuation processing. The attenuated terrain field intensity contribution value is calculated based on the spatial distance between each spatial coordinate point and the physical field state interference seed point. The spatial coordinate point with a greater spatial distance has a lower terrain field intensity contribution value. The attenuated visual field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the visual field intensity convergence value. The attenuated terrain field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the terrain field intensity convergence value. The visual field intensity convergence value and the terrain field intensity convergence value are correlated with each other to generate a mutual interference inference point record for the physical field interference seed point. The mutual interference inference point record includes the spatial coordinates of the physical field interference seed point, the visual field intensity convergence value, and the terrain field intensity convergence value. Traverse all physical field state interference seed points in the visual-tactile collaborative scene graph, and perform the following processes for each physical field state interference seed point: interference neighborhood window construction, visual field state point set and terrain field state point set extraction, field state attenuation, field state convergence and accumulation, field state interference association, and interference inference point record generation, to obtain the visual-tactile interference inference result. The visual-tactile interference inference result is composed of the interference inference point records of all physical field state interference seed points.

6. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, Perform physical field state interference deduction processing on the aforementioned olfactory-auditory collaborative scene diagram, including: All spatial coordinate points carrying odor sound wave correlation intensity markers are extracted from the olfactory-auditory collaborative scene graph as physical field state interference seed points. For each physical field state interference seed point, an interference neighborhood window is constructed with the seed point as its center and covering a preset interference radius. Within the interference neighborhood window, all spatial coordinate points carrying odor field state intensity markers within the interference neighborhood window are extracted from the olfactory-auditory collaborative scene graph to form an odor field state point set, and all spatial coordinate points carrying sound wave field state intensity markers within the interference neighborhood window are extracted to form a sound wave field state point set. The odor field intensity value of each spatial coordinate point in the odor field state point set is subjected to field state attenuation processing. The attenuated odor field intensity contribution value is calculated based on the spatial distance between each spatial coordinate point and the physical field state interference seed point. The spatial coordinate point with a greater spatial distance has a lower odor field intensity contribution value. The acoustic field intensity values ​​of each spatial coordinate point in the acoustic field state point set are subjected to field state attenuation processing. The acoustic field intensity contribution value after attenuation is calculated based on the spatial distance between each spatial coordinate point and the physical field state mutual interference seed point. The greater the spatial distance, the lower the acoustic field intensity contribution value of the spatial coordinate point. The attenuated odor field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the odor field intensity convergence. The attenuated sound wave field intensity contribution values ​​of all spatial coordinate points within the mutual interference neighborhood window are accumulated at the physical field interference seed point to obtain the sound wave field intensity convergence. The odor field intensity convergence and the sound wave field intensity convergence are correlated with each other to generate a mutual interference prediction point record for the physical field interference seed point. The mutual interference prediction point record includes the spatial coordinates of the physical field interference seed point, the odor field intensity convergence, and the sound wave field intensity convergence. Traverse all physical field state interference seed points in the olfactory-auditory collaborative scene graph, and perform the following processes for each physical field state interference seed point: interference neighborhood window construction, odor field state point set and sound wave field state point set extraction, field state attenuation, field state convergence and accumulation, field state interference association, and interference inference point record generation, to obtain the olfactory-auditory interference inference result. The olfactory-auditory interference inference result is composed of the interference inference point records of all physical field state interference seed points.

7. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, The visual-tactile interference simulation results and the olfactory-auditory interference simulation results are subjected to cross-graph temporal correlation fusion processing to generate a global collaborative perception situational simulation map, including: The spatial coordinates of each interference simulation point recorded in the visual-tactile interference simulation results and the spatial coordinates of each interference simulation point recorded in the olfactory-auditory interference simulation results are mapped to a unified spatial grid coordinate system. A spatial grid array covering the disaster site is constructed in the unified spatial grid coordinate system. The spatial grid array is composed of multiple spatial grid nodes with a unified grid size. For each spatial grid node in the spatial grid array, a visual-tactile field state component mapping process is performed. The visual field state convergence amount recorded by all mutual interference inference points falling within the range of the spatial grid node in the visual-tactile mutual interference inference results is averaged and aggregated to obtain the visual field state convergence component of the spatial grid node. The terrain field state convergence amount recorded by all mutual interference inference points falling within the range of the spatial grid node in the visual-tactile mutual interference inference results is averaged and aggregated to obtain the terrain field state convergence component of the spatial grid node. For each spatial grid node in the spatial grid array, perform olfactory and auditory field state component mapping processing. The odor field state convergence amount recorded by all mutual interference inference points falling within the range of the spatial grid node in the olfactory and auditory mutual interference inference results is averaged and aggregated to obtain the odor field state convergence component of the spatial grid node. The sound wave field state convergence amount recorded by all mutual interference inference points falling within the range of the spatial grid node in the olfactory and auditory mutual interference inference results is averaged and aggregated to obtain the sound wave field state convergence component of the spatial grid node. For each spatial grid node in the spatial grid array, field state component splicing processing is performed. The visual field state convergence component, terrain field state convergence component, odor field state convergence component, sound wave field state convergence component, visual field state intensity value of the visual saliency region segment corresponding to the spatial grid node, and terrain field state intensity value of the terrain stiffness jump boundary segment corresponding to the spatial grid node are spliced ​​along the component channel dimension to generate the cooperative perception situation component of the current time section of the spatial grid node. The collaborative sensing situational components of the same spatial grid node in continuous time sections on the sampling time axis are processed by time sequence arrangement to obtain the evolution sequence of the collaborative sensing situational components of the spatial grid node. The evolution sequence of the collaborative sensing situational components arranges the collaborative sensing situational components of the spatial grid node in each time section in order from early to late timestamp. Traverse all spatial grid nodes in the spatial grid array, and perform the visual-tactile field-state component mapping processing, the olfactory-auditory field-state component mapping processing, the field-state component splicing processing, and the temporal arrangement processing on each spatial grid node to generate a global collaborative perception situation projection map. The global collaborative perception situation projection map contains the evolution sequence of the collaborative perception situation components corresponding to each of the spatial grid nodes at the disaster site.

8. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, Based on the aforementioned global collaborative perception situational awareness map, the spatial coordinates of the disaster spread source and the leading edge of the disaster spread path evolution trend are determined, including: Extract the collaborative perception situation components of the current time segment from the evolution sequence of the collaborative perception situation components of each spatial grid node in the global collaborative perception situation deduction diagram, and parse the visual field convergence component, terrain field convergence component, odor field convergence component and sound wave field convergence component of the spatial grid node in the current time segment from the collaborative perception situation components of the current time segment. For each spatial grid node in the current time section, the visual field convergence component and the terrain field convergence component are processed to calculate the field state fluctuation amplitude. The visual field convergence component of the current time section is compared with the visual field convergence component of the previous adjacent time section, and the time difference amplitude of the visual field convergence component is calculated as the visual fluctuation amplitude. The terrain field convergence component of the current time section is compared with the terrain field convergence component of the previous adjacent time section, and the time difference amplitude of the terrain field convergence component is calculated as the terrain fluctuation amplitude. The sum of the visual fluctuation amplitude and the terrain fluctuation amplitude is used as the visual tactile fluctuation amplitude. For the odor field convergence component and sound wave field convergence component of each spatial grid node in the current time section, the field state fluctuation amplitude is calculated. The odor field convergence component in the current time section is compared with the odor field convergence component in the previous adjacent time section, and the time difference amplitude of the odor field convergence component is calculated as the odor fluctuation amplitude. The sound wave field convergence component in the current time section is compared with the sound wave field convergence component in the previous adjacent time section, and the time difference amplitude of the sound wave field convergence component is calculated as the sound wave fluctuation amplitude. The sum of the odor fluctuation amplitude and the sound wave fluctuation amplitude is used as the olfactory-auditory fluctuation amplitude. The visual-tactile fluctuation amplitude and the olfactory-auditory fluctuation amplitude of each spatial grid node are fused. The larger of the visual-tactile fluctuation amplitude and the olfactory-auditory fluctuation amplitude is taken as the cooperative fluctuation amplitude of the spatial grid node, and the fluctuation direction vector of the cooperative fluctuation amplitude on the spatial grid node pointing to the next adjacent time section is marked. Construct a spatial distribution map of field state fluctuations in the current time section based on the coordinated fluctuation amplitudes of all spatial grid nodes. Search for the spatial grid node with the largest coordinated fluctuation amplitude in the spatial distribution map as a candidate node for disaster spread source. Determine the spatial coordinates of the candidate node for disaster spread source as the spatial coordinates of disaster spread source. Starting from the spatial grid node corresponding to the spatial coordinates of the disaster spread source, the adjacent spatial grid nodes pointed to by the fluctuation direction vector of the spatial grid node are traced node by node. The spatial coordinates of each spatial grid node passed in sequence on the tracing path are recorded in the tracing order to obtain the front line of the evolution trend of the disaster spread path. The spatial distribution map of field fluctuations in the current time segment, the spatial coordinates of the disaster spread source, and the position line of the evolution trend front are associated and stored in the disaster situation evolution record. As the acquisition time segment progresses, the collaborative perception situation component analysis processing, the field fluctuation amplitude calculation processing, the fluctuation amplitude fusion processing, the disaster spread source spatial coordinate determination processing, and the tracking processing are repeatedly performed on each new time segment to obtain the updated disaster spread source spatial coordinates and the updated evolution trend front position line.

9. The emergency bionic robot collaborative method with task and state collaborative perception as described in claim 1, characterized in that, Determine the current pose state information of each bionic robot body, including: The body posture data stream of each bionic robot body is obtained from the bionic motion posture sensing unit deployed on each bionic robot body. The body posture data stream includes the torso spatial orientation data and bionic limb joint angle data of each bionic robot body. Extract the evolution sequence of the collaborative perception situation components of each bionic robot body in the spatial grid node where each bionic robot body is located from the global collaborative perception situation inference map. Determine the local visual and tactile field state components and the local olfactory and auditory field state components of the current location of each bionic robot body based on the evolution sequence of the collaborative perception situation components of each current spatial grid node where the bionic robot body is located. The orientation field state mapping process is performed on the torso spatial orientation data of each bionic robot body and the local visual-tactile field state component of the position of the bionic robot body. The torso spatial orientation data is transformed into the spatial coordinate reference system where the local visual-tactile field state component is located. The angle between the torso spatial orientation and the field state gradient of the visual-tactile field state component is calculated as the orientation field state deviation angle. The bionic limb joint angle data of each bionic robot body and the local olfactory and auditory field state components of the bionic robot body are subjected to limb tactile mapping processing. The spatial position coordinates of each bionic limb end in the bionic limb joint angle data are projected onto the spatial grid node where the local olfactory and auditory field state components are located. The olfactory and auditory field state components of the spatial grid nodes projected by each bionic limb end are extracted as limb tactile perception components. The torso spatial orientation data, bionic limb joint angle data, orientation field deviation angle, and limb tactile perception components of each bionic robot body are collected according to the bionic robot body identifier to construct the body pose perception description set of each bionic robot body in the current time plane. The body pose perception description set includes the spatial positioning coordinates of the bionic robot body, torso orientation angle, joint bending angle of each bionic limb joint, orientation field deviation angle, and limb tactile perception components corresponding to each bionic limb end. The pose perception description sets of all bionic robot bodies in the bionic robot cluster are summarized to obtain the cluster pose state set in the current time segment. The cluster pose state set includes the body identifier of each bionic robot body and its pose perception description set in the current time segment. The pose state set of the current time segment is compared with the pose state set of the previous adjacent time segment. The change in spatial positioning coordinates and the change in torso orientation angle of each bionic robot body are calculated to generate the current pose state information of each bionic robot body. The current pose state information includes the spatial position coordinate change vector and the torso orientation change rate.

10. A multimodal rescue system, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to execute the emergency bionic robot collaborative method with task and state collaborative awareness as described in any one of claims 1 to 9 by executing the machine-executable instructions.

Images