A rescue robot path planning method and system based on sound source positioning

Abstract

The application discloses a kind of rescue robot path planning method and system based on sound source positioning, it is related to robot path planning technical field, the method real-time collection stereo sound format rescue environment audio data;Adopt pre-training sound source positioning model analysis rescue environment audio data's acoustic characteristics, output the sound event type and position data contained in audio;According to the output result of pre-training sound source positioning model, determine rescue target, obtain the environment model of rescue area by fusing the data collected by rescue unmanned aerial vehicle;Based on rescue target position information and rescue area environment model, using quadratic optimization path planning algorithm to formulate the rescue route of rescue robot, and realize dynamic obstacle avoidance in the advancing process of robot by dynamic window method.The problem of insufficient accuracy of rescue robot path planning in post-disaster environment with limited vision is solved, the method of determining rescue target position information from the auditory perspective is expanded, and the efficiency of robot path planning is improved.

Description

Technical Field

[0001] This application relates to the field of robot path planning technology, and in particular to a path planning method and system for rescue robots based on sound source localization. Background Technology

[0002] Rescue robots are intelligent sensing and control devices specifically designed for disaster site rescue. Equipped with a variety of sensors, they can perceive their surroundings in complex environments, detect signs of life, and perform tasks such as demolition, transportation, and rescue. Application scenarios include earthquake, fire, mine, and building collapse rescue sites.

[0003] Path planning is a key aspect of a rescue robot's autonomous navigation and mission completion. Proper path planning can reduce energy consumption and travel distance, enabling the robot to reach the target location faster and improving rescue speed.

[0004] Existing path planning methods for rescue robots employ map- and sensor-based approaches. Given the target location and the surrounding map, they perform a global search to find the optimal path and then adjust and optimize based on real-time environmental information perceived by sensors. However, existing perception methods determine the target location for path planning solely from a visual perspective. In post-disaster sites, the severely destructive environment creates a poor visual environment, significantly increasing the difficulty of navigation and path planning for rescue robots. Therefore, it is necessary to utilize multimodal fusion methods and incorporate audio recognition technology to obtain comprehensive positioning information from an auditory perspective when image data cannot provide accurate location data, thereby improving the accuracy of path planning for rescue robots. Summary of the Invention

[0005] To address the technical problems of the prior art, this application provides a path planning method and system for rescue robots based on sound source localization. The method utilizes sound source localization to analyze audio data, classify and locate sound events to obtain the location information of the rescue target, and combines this with environmental perception information provided by multiple sensors to obtain accurate rescue target location data for the rescue robot's rescue route planning. Furthermore, an improved path planning algorithm based on secondary optimization and dynamic windows is employed to enhance the efficiency of the rescue robot's path planning and enable emergency dynamic obstacle avoidance.

[0006] This application provides a path planning method for rescue robots based on sound source localization, including:

[0007] (1) Install a microphone array on the rescue robot to collect real-time stereo audio data of the rescue environment; (2) Analyze the acoustic characteristics of the audio data of the rescue environment by using the pre-trained sound source localization model embedded in the rescue robot, and output the sound event types and location data contained in the audio; (3) Determine the rescue target based on the output of the pre-trained sound source localization model, and integrate the data collected by the rescue drone to perceive and map the environment of the rescue site to obtain the environmental model of the rescue area; (4) Based on the location information of the rescue target and the environmental model of the rescue area, the rescue route of the rescue robot is formulated by the secondary optimization path planning algorithm, and the dynamic obstacle avoidance of the robot during the movement is realized by the dynamic window method.

[0008] Furthermore, in buildings affected by disasters such as fires, floods, and earthquakes, rescue robots cannot directly detect targets calling for help in other rooms or invisible areas of the building through sensors such as vision, ultrasound, and infrared due to smoke, wall collapse, etc. However, sound signals, unlike light, ultrasound, and infrared signals, have the characteristic of being able to propagate around obstacles. Auditory sensors can compensate for the limitations of other sensors, such as limited field of view and inability to penetrate non-transparent obstacles. Auditory navigation can be used to determine the location of rescue targets. Therefore, sound source localization is a key link in realizing auditory navigation.

[0009] Sound source localization addresses the problem of locating multi-channel spatial audio signals relative to a recording microphone array. Its purpose is to estimate the accurate location of one or more sound sources relative to a reference location, providing auxiliary localization information for path planning in rescue robots. Sound source localization is simplified to estimating the direction of arrival (DOA) of the sound source, i.e., the azimuth and elevation angles of the sound source relative to the microphone array, without involving the estimation of the distance to arrival. Traditional sound source localization methods are based on signal and channel models and signal processing techniques. With technological advancements, methods based on deep neural networks are increasingly prevalent. These methods use the audio spectrum and sound intensity vector as input features, employing a convolutional recurrent neural network architecture to estimate the DOA of multiple sound sources, significantly improving the accuracy of sound source localization.

[0010] Furthermore, while the sound source localization model performs direction-of-arrival estimation, a sound event detection task is added to predict the type and start / end time of sound events. This provides the rescue robot with accurate information about the rescue target, ignoring interference information and helping to determine the urgency of the rescue, thus improving the accuracy of target identification. The sound source localization model used in this invention combines sound event detection and direction-of-arrival estimation tasks, and incorporates a pre-trained speech enhancement network at the beginning of the model to remove noise interference from the data, achieving accurate acquisition of rescue target information and improving the efficiency of path planning for the rescue robot.

[0011] This application also provides a path planning system for a rescue robot based on sound source localization, including: Audio acquisition module: used to acquire real-time stereo audio data of the rescue environment through a microphone array installed on the rescue robot; Sound source localization module: used to analyze the acoustic characteristics of audio data in the rescue environment through a pre-trained sound source localization model embedded in the rescue robot, and output the sound event types and location data contained in the audio; The environmental mapping module is used to determine the rescue target based on the output of the pre-trained sound source localization model, and to perceive and map the environment of the rescue site by integrating data collected by the rescue drone, so as to obtain an environmental model of the rescue area. Path planning module: Based on the location information of the rescue target and the environmental model of the rescue area, it uses a secondary optimization path planning algorithm to formulate the rescue route of the rescue robot, and uses a dynamic window method to realize dynamic obstacle avoidance during the robot's movement.

[0012] This application discloses the following technical effects: This application proposes a path planning method and system for rescue robots based on sound source localization. Utilizing intelligent sensing and control devices on the robot, it provides navigation information for path planning from an auditory perspective, even in environments with poor visual perception. This invention employs a pre-trained sound source localization model based on deep learning. Leveraging the powerful feature learning capabilities of neural networks, it performs deep analysis on audio data collected in the rescue environment, extracting acoustic features, learning the spectral distribution patterns and spatial structure information of each sound event, and removing noise interference. Furthermore, based on the feature information learned by the model, it determines the rescue target and obtains the target's location data. In addition, based on the localization results output by the sound source localization model, this invention determines the rescue area for the rescue robot's mission and, combined with terrain and obstacle distribution data perceived by the rescue drone, establishes a rescue area environment model. The robot's route is planned on this environmental model, and a secondary optimization genetic algorithm is used to determine the optimal path for rescue action, guiding the robot to reach the target location in a timely manner. A dynamic window method is incorporated during the robot's movement to achieve dynamic obstacle avoidance, improving the robot's emergency response capabilities and further enhancing its rescue efficiency, ensuring the smooth execution of rescue missions. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0014] Figure 1This is a flowchart illustrating a path planning method for a rescue robot based on sound source localization, provided in an embodiment of this application.

[0015] Figure 2 This is a schematic diagram of the structure of the pre-trained sound source localization model provided in the embodiments of this application.

[0016] Figure 3 This is a schematic diagram of a path planning system for a rescue robot based on sound source localization, provided in an embodiment of this application. Detailed Implementation

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application.

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the following description, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0020] Example 1: This application provides a path planning method for a rescue robot based on sound source localization, such as... Figure 1 As shown, the method includes: Step S10: Install a microphone array on the rescue robot to collect real-time stereo audio data of the rescue environment.

[0021] In this embodiment, the rescue robot is equipped with a high-channel counting spherical microphone array to acquire multi-channel first-order Ambisonics format stereo signals and record audio information of the rescue environment: The high-channel counting spherical microphone array consists of multiple microphones evenly distributed on a spherical structure to capture audio signals from all directions and record multi-channel information of audio in multiple directions in the Cartesian coordinate system. Each microphone is connected to an audio acquisition module and is set to a sampling rate of 24kHz to independently acquire audio signals from the rescue environment. The acquired analog audio signals from the rescue environment are converted into digital signals and amplified to highlight the amplitude and phase information of each time component. Multi-channel first-order Ambisonics format stereo signals are used to record and play back three-dimensional sound fields. They include omnidirectional channels, X channels, Y channels, and Z channels. These channels are combined through differential gain and phase relationships to represent a complete three-dimensional sound field and record omnidirectional spatial information of multiple sound sources in the rescue environment.

[0022] Step S20: Analyze the acoustic characteristics of the audio data of the rescue environment through the pre-trained sound source localization model embedded inside the rescue robot, and output the sound event types and location data contained in the audio.

[0023] In this embodiment, the acoustic features extracted from the rescue environment audio data include a multi-channel Log linear spectrum and a normalized sound intensity vector. The extraction process of the multi-channel Log linear spectrum is as follows: set up Given the number of sound sources, the short-time Fourier transform (STFT) signal of the ambient audio signal observed in the time-frequency domain within a single audio channel is expressed by the formula:

[0024] in, This represents the complex spectrum of the audio signal in a single-channel rescue environment. and These are time and frequency indices, respectively. Indicating the first in the rescue environment The complex spectrum of a sound source signal, The complex spectrum of noise; Indicates the first The direction of the sound source wave The corresponding number The steering vector of each frequency component and They represent the first The azimuth and elevation angles of each sound source are determined; further, the logarithmic linear spectrum of the single-channel audio signal is obtained using the following formula. :

[0025] in, This represents the L2 norm. The above formula can be simply described as follows: First, square the complex spectrum of each channel signal to obtain the power spectrum. Then, take the logarithm of the power spectrum to obtain the single-channel Log linear spectrum of the rescue environment audio signal. Concatenating the Log linear spectra of each channel yields the multi-channel Log linear spectrum of the rescue environment audio data, with a size of... , This represents the number of microphone channels in the microphone array. Indicates the number of time frames. Number of frequency bands; The sound intensity vector carries important information related to the direction of sound propagation and is suitable for audio signals in FOA format. The instantaneous sound intensity vector is represented as follows: ,in Indicates sound pressure. Represents the particle velocity vector. , and These are the velocity components of the X, Y, and Z channels, respectively. This indicates a transpose operation; the sound intensity vector carries the energy direction information of the sound wave, and its inverse direction is interpreted as the direction of arrival of the sound source. In the STFT domain, the sound intensity vector of the rescue environment audio signal... It can be calculated using the following formula:

[0026] in, and These represent atmospheric density and the speed of sound, respectively. Represents the real part of a complex number; This is the complex spectrum of the omnidirectional channel components of the audio signal. Indicates complex conjugate; , and Represent the complex spectra of the X, Y, and Z channel components of the rescue audio signal, respectively; normalized sound intensity vector. Represented as:

[0027] in, This indicates a modulo operation.

[0028] The above-mentioned process of extracting acoustic features is implemented through the acoustic feature extractor in the sound source localization model. It does not participate in the model training process. The Hanning window used for short-time Fourier transform in the extractor has a window length of 512 sampling points and a frame shift of 300 sampling points. Without considering the spectral information of the omnidirectional channel, the final result is the three-channel Log linear spectrum and the four-channel normalized sound intensity vector. The two acoustic features are concatenated according to the channel dimension to obtain the first feature information used for analysis and processing of the sound source localization model.

[0029] The detailed process of obtaining a pre-trained sound source localization model includes: The system collects sound events in the rescue environment used to identify rescue targets, including cries for help, screams, knocking sounds, and glass breaking sounds. It then spatializes the sound events using room impulse response and synthesizes audio data containing three sound events, each with three sound sources in different spatial locations. The audio data of sound events of the rescue target are labeled, and the sound event type and the direction of arrival (azimuth and elevation) of the sound source are labeled at each moment in the audio signal. For each sound event, the sound source is divided into stationary and moving sound sources, and its start and end times are recorded. The labeled audio data of the rescue target sound events and their corresponding real label information constitute the dataset for training the sound source localization model. It contains 400 training records, 100 test records and 100 validation records. Each record lasts for 1 minute and includes different combinations of sound event types and sound sources in different spatial locations. A sound source localization model is established to simultaneously handle sound event detection and direction-of-arrival estimation tasks. The model includes an acoustic feature extractor, a speech enhancement network, a convolutional neural network, and a recurrent neural network. The acoustic feature extractor and the speech enhancement network do not require training, and their network parameters are fixed. Hyperparameters of each convolutional layer, pooling layer, recurrent unit, and fully connected layer in the model are set. During model training, the AdamW optimizer was used with an initial learning rate of 0.0003 and a cosine-cooled dynamic learning rate strategy. The binarization threshold for sound event detection was set to 0.3. The overall model and training code were implemented using the PyTorch framework. Binary cross-entropy loss and mean squared error loss were selected as the loss functions for sound event detection and direction of arrival estimation, respectively. During training, the changing trend of the loss function was recorded, and the model parameters were updated by backpropagation using stochastic gradient descent. The model performance was evaluated by error rate and recall rate. When the loss function converged, the model parameters with the best performance were saved as the pre-trained sound source localization model.

[0030] Step S30: Determine the rescue target based on the output of the pre-trained sound source localization model, and use the data collected by the rescue drone to perceive and map the environment of the rescue site to obtain an environmental model of the rescue area.

[0031] In this embodiment, the pre-trained sound source localization model outputs the localization information of multiple sound sources, transmits the localization information to the lidar, obtains the distance information of each sound source by emitting and receiving lasers, determines the rescue target closest to the rescue robot as the first endpoint position, and defines the positions of other rescue targets as the second and third endpoint positions in sequence according to their distance, which is used to formulate the rescue route of the rescue robot. The urgency of the rescue is determined according to the sound event type and duration output by the model, and the order of endpoint positions is dynamically adjusted.

[0032] The process of determining the rescue area based on the location of the rescue target and establishing an environmental model of the rescue area is as follows: The system acquires image data from cameras mounted on rescue robots and rescue drones, and combines this with ranging data from lidar on the rescue drones to determine the distribution of obstacles in the rescue area and mark the locations of the obstacles. An environmental model of the rescue area is constructed using the grid method. The two-dimensional plane of the rescue area is represented as a grid diagram. The rescue robot is defined as a point mass in the diagram with constant velocity, and obstacles are static with fixed size and position. Label each grid in the rescue area grid map, using black grids to represent obstacles and white grids to represent free passage areas. Let the size of the rescue area be... , No. The grid numbers are Corresponding grid position coordinates Represented as:

[0033] in, This represents the modulo operation. Indicates rounding down; Before using a genetic algorithm for path planning, the grid number is encoded as a chromosome gene. Each chromosome obtained by the algorithm corresponds to a feasible path from the starting point to the target point, forming a set of sequence numbers.

[0034] Step S40: Based on the location information of the rescue target and the environmental model of the rescue area, a secondary optimization path planning algorithm is used to formulate the rescue route of the rescue robot, and dynamic obstacle avoidance is achieved during the robot's movement through a dynamic window method.

[0035] In this embodiment, the location of the rescue target output by the sound source localization model is marked in the rescue area environment model, with the location of the rescue robot as the starting point. The first endpoint is the first endpoint of the path planning. An improved quadratic optimization genetic algorithm is used to plan the rescue route for the rescue robot: An initial population is established using a heuristic feasible point insertion method, with the population size set to [value missing]. Generate an initial path chromosome, setting the starting point as... As the first gene Add a path chromosome and randomly search for a point in the reachable region as the next point of the path, passing through... Displacement reaches the first endpoint , denoted as the last gene The initial path chromosome is represented by its sequence number as follows: ; Using fitness safety evaluation function Determining the quality of path chromosomes within a population includes a path length evaluation score. and path smoothness evaluation It can be expressed by the following formula:

[0036] in, and They are respectively and The higher the weight of the fitness safety evaluation function, the better the path planning result; the path length evaluation function... It is calculated using the following formula:

[0037] in, This represents the total number of nodes in the path. and The first in the path and the Coordinates of each grid cell; path smoothness evaluation score It is calculated using the following formula:

[0038] in, It is the first The weighted angle of each node is represented as the product of the robot's turning angle and the penalty weight at each node; turning angle The possible values ​​include , , , and Set the maximum threshold for the turning angle as ,like ,but The smaller the corresponding penalty weight; The gene nodes of the path chromosome in the population are updated by selection, crossover and mutation operations. Nodes with the same index in two paths are swapped and nodes are changed by neighborhood single-point mutation. The shortest path is left. When the fitness value remains unchanged, the iteration stops and the optimal path sequence number is obtained, which is used to plan the rescue route of the rescue robot. If a node in the optimal path can be connected to subsequent nodes (non-adjacent nodes) without obstacles, then the intermediate node between the two points is a redundant node. The redundant node is deleted, and the rescue route is optimized a second time.

[0039] The rescue robot determines the specific route to the first destination based on the path sequence number obtained by the path planning algorithm. While traveling along the route, when it detects unknown obstacles, it records the current position and the next position as vectors in the rescue area environment model. and (Including horizontal coordinates and steering angle), local obstacle avoidance is achieved through a dynamic window method: First, determine the local path guide node. Then determine the path nodes. Finally, the new driving node was determined. Update the rescue robot's route:

[0040]

[0041]

[0042] in, For unknown obstacle nodes, Let be the set of all nodes in the driving route. For nodes in the set, yes and The resultant vector, This is the unit vector representing the linear velocity of the rescue robot when it enters the unreachable area of ​​the guide node. This is the unit vector in the direction from the guide node to the path node. This indicates the transpose operation.

[0043] After the rescue robot reaches the first endpoint and completes the rescue mission, it takes the current position as the new starting point and successively determines the second and third endpoints as the endpoints. It also uses a path planning method based on a secondary optimization genetic algorithm to formulate the driving route and carry out rescue operations on all rescue targets within the rescue area.

[0044] Example 2: This embodiment of the invention provides a detailed structure of a pre-trained sound source localization model for locating rescue targets, as follows: Figure 2As shown: In this embodiment, the real-time collected audio data of the rescue environment is first sent to the acoustic feature extractor to obtain the first feature information composed of a multi-channel Log linear spectrum and a normalized sound intensity vector. This feature information contains noise, which affects the classification and positioning results. Secondly, the first feature information is fed into a pre-trained speech enhancement network to remove noise interference. This network adopts a DPCRN structure and only selects the encoder part of the structure to separate clean data and noise data in the acoustic features of the rescue environment audio data, and obtains the acoustic features of the clean signal as the second feature information. Next, the second feature information is fed into a feature extraction module consisting of three stacked layers. This module comprises depthwise separable convolutional layers, ReLU activation functions, and max pooling layers. It extracts feature information from the rescue environment audio data from the second feature information and matches the extracted feature information with the rescue target sound event feature knowledge learned by the model to determine whether the second feature information contains sound events used to identify the rescue target. The max pooling layer downsamples the feature map to reduce the computational complexity of the model. After three downsampling operations, the feature map size becomes 1 / 8 of its original size. Then, the downsampled features are fed into a bidirectional gated recurrent unit for temporal context modeling to obtain the temporal correlation of the features, which is used to determine the duration and location change of the rescue target sound event, which is beneficial to obtain accurate positioning of the rescue target. The features after temporal context modeling are used to establish a nonlinear relationship through the tanh activation function; the bidirectional gated recurrent unit and the tanh activation function are stacked in two layers. Finally, the features modeled after the temporal context are fed into a two-branch structure, which is then processed by a fully connected layer, a sigmoid activation function, and a tanh activation function to obtain the prediction results for sound event detection and direction of arrival estimation. These results are used to process the sound event detection and direction of arrival estimation tasks, and to obtain the sound event types, temporal trajectories, and corresponding location data of multiple active sound sources in the rescue environment audio data.

[0045] Example 3: The rescue robot path planning system based on sound source localization provided in this embodiment of the invention can execute the rescue robot path planning method based on sound source localization provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method, such as... Figure 3 As shown, it includes the following modules: Audio acquisition module: used to acquire real-time stereo audio data of the rescue environment through a microphone array installed on the rescue robot; Sound source localization module: used to analyze the acoustic characteristics of audio data in the rescue environment through a pre-trained sound source localization model embedded in the rescue robot, and output the sound event types and location data contained in the audio; The environmental mapping module is used to determine the rescue target based on the output of the pre-trained sound source localization model, and to perceive and map the environment of the rescue site by integrating data collected by the rescue drone, so as to obtain an environmental model of the rescue area. Path planning module: Based on the location information of the rescue target and the environmental model of the rescue area, it uses a secondary optimization path planning algorithm to formulate the rescue route of the rescue robot, and uses a dynamic window method to realize dynamic obstacle avoidance during the robot's movement.

[0046] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0047] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than in the embodiments and still achieve the desired results.

Claims

1. A path planning method for a rescue robot based on sound source localization, characterized in that, The method includes: (1) Install a microphone array on the rescue robot to collect real-time stereo audio data of the rescue environment; (2) Analyze the acoustic characteristics of the audio data of the rescue environment by using the pre-trained sound source localization model embedded in the rescue robot, and output the sound event types and location data contained in the audio; The process of obtaining a pre-trained sound source localization model is as follows: Collect sound event types in the rescue environment to identify rescue targets, and spatialize the sound events through room impulse response to synthesize multi-source audio data; The audio data of sound events of the rescue target are labeled, and the sound event type and direction of arrival of the sound source are labeled at each moment in the audio signal. The labeled audio data and its corresponding real label information constitute the dataset for training the sound source localization model, which includes different combinations of sound event types and sound sources in different spatial locations. A sound source localization model is established to simultaneously handle sound event detection and direction-of-arrival estimation tasks. The model includes an acoustic feature extractor, a speech enhancement network, a convolutional neural network, and a recurrent neural network. The acoustic feature extractor and the speech enhancement network do not require training, and their network parameters are fixed. Binary cross-entropy loss and mean squared error loss were selected as the loss functions for sound event detection and direction of arrival estimation, respectively. During training, the changing trend of the loss function was recorded, the model parameters were updated by backpropagation, and the model performance was evaluated. When the loss function converged, the model parameters with the best performance were saved as the pre-trained sound source localization model. (3) Determine the rescue target based on the output of the pre-trained sound source localization model, and integrate the data collected by the rescue drone to perceive and map the environment of the rescue site to obtain the environmental model of the rescue area; (4) Based on the location information of the rescue target and the environmental model of the rescue area, the rescue route of the rescue robot is formulated by the secondary optimization path planning algorithm, and the dynamic obstacle avoidance of the robot during the movement is realized by the dynamic window method.

2. The path planning method for a rescue robot based on sound source localization as described in claim 1, characterized in that, In step (1), the microphone array is a high-channel counting spherical microphone array, which consists of multiple microphones evenly distributed on the spherical structure to capture audio signals from all directions; each microphone is connected to an audio acquisition module to independently acquire audio signals from the rescue environment; The stereo format is a multi-channel first-order Ambisonics format, which includes omnidirectional channels, X channels, Y channels and Z channels. These channels are combined through differential gain and phase relationship to represent a complete three-dimensional sound field and record the omnidirectional spatial information of multiple sound sources in the rescue environment.

3. The path planning method for a rescue robot based on sound source localization as described in claim 1, characterized in that, In step (2), the acoustic features of the rescue environment audio data include a multi-channel Log linear spectrum and a normalized sound intensity vector. The extraction process of the multi-channel Log linear spectrum is as follows: The power spectrum is obtained by squaring the complex spectrum of each channel of the audio data from the rescue environment. Taking the logarithm of the power spectrum yields the single-channel logarithmic linear spectrum of the audio signal from the rescue environment. Within a single audio channel, the logarithmic linear spectrum of the audio signal is... Represented as: in, and They are time and frequency indices, respectively. This represents the short-time Fourier transform signal of the environmental audio data observed in the time-frequency domain. This represents the L2 norm; concatenating the Log linear spectra of each channel yields the multi-channel Log linear spectrum of the rescue environment audio data, with a size of [missing value]. , This represents the number of microphone channels in the microphone array. Indicates the number of time frames. This represents the number of frequency bands.

4. The path planning method for a rescue robot based on sound source localization as described in claim 3, characterized in that, The normalized sound intensity vector carries the energy direction information of the sound wave, and its inverse direction is interpreted as the direction of arrival of the sound source. In the frequency domain, the sound intensity vector of the rescue environment audio data... It can be calculated using the following formula: in, and These represent atmospheric density and the speed of sound, respectively. Represents the real part of a complex number; This is the complex spectrum of the omnidirectional channel components of the audio signal. Indicates complex conjugate; , and Represent the complex spectra of the X, Y, and Z channel components of the rescue audio signal, respectively; normalized sound intensity vector. Represented as: in, This indicates a modulo operation.

5. The path planning method for a rescue robot based on sound source localization as described in claim 1, characterized in that, In step (3), the process of determining the rescue area and establishing the environmental model of the rescue area based on the location of the rescue target output by the sound source localization model is as follows: The system acquires image data from cameras mounted on rescue robots and rescue drones, and combines this with ranging data from lidar on the rescue drones to determine the distribution of obstacles in the rescue area and mark the locations of the obstacles. An environmental model of the rescue area is constructed using the grid method. The two-dimensional plane of the rescue area is represented as a grid diagram. The rescue robot is defined as a point mass in the diagram with constant velocity, and obstacles are static with fixed size and position. Each grid in the rescue area grid map is labeled, with black grids representing obstacles and white grids representing free passage areas. The position coordinates of each grid are calculated. Before using a genetic algorithm for path planning, the grid number is encoded as a chromosome gene. Each chromosome obtained by the algorithm corresponds to a feasible path from the starting point to the target point, forming a set of sequence numbers.

6. The path planning method for a rescue robot based on sound source localization as described in claim 1, characterized in that, In step (4), a genetic algorithm with secondary optimization is used to plan the route for the rescue robot: An initial population is established using a heuristic feasible point insertion method, an initial path chromosome is generated, the starting point is added to the path chromosome as the first gene, and a point is randomly searched in the reachable region as the next point of the path until the target node is found. The initial path chromosome is represented by a set of sequence numbers. The fitness safety evaluation function is used to judge the quality of path chromosomes in the population. The gene nodes of the path chromosomes in the population are updated and iterated through selection, crossover and mutation operations. The path with the smallest fitness value is left. When the fitness value remains unchanged, the iteration stops and the optimal path sequence number is obtained, which is used to plan the rescue route of the rescue robot. If a node in the optimal path can connect to subsequent non-adjacent nodes without obstacles, then the intermediate node between the two points is a redundant node. The redundant node is deleted, and the rescue route is further optimized.

7. The path planning method for a rescue robot based on sound source localization as described in claim 1, characterized in that, The dynamic obstacle avoidance of the rescue robot during its movement is achieved through the following steps: When the rescue robot detects an unknown obstacle, its current position and next position are recorded as vectors in the rescue area environment model. and Local obstacle avoidance is achieved through a dynamic window method: First, determine the local path guide node. Then determine the path nodes. Finally, the new driving node was determined. Update the rescue robot's route: in, For unknown obstacle nodes, Let be the set of all nodes in the driving route. For nodes in the set, yes and The resultant vector, This is the unit vector in the linear velocity direction when the rescue robot enters the unreachable area of ​​the guide node. This is the unit vector in the direction from the guide node to the path node. This indicates the transpose operation.

8. A path planning system for a rescue robot based on sound source localization, characterized in that, The system is used to implement the path planning method for a rescue robot based on sound source localization as described in any one of claims 1-7, and the system comprises: Audio acquisition module: used to acquire real-time stereo audio data of the rescue environment through a microphone array installed on the rescue robot; Sound source localization module: used to analyze the acoustic characteristics of audio data in the rescue environment through a pre-trained sound source localization model embedded in the rescue robot, and output the sound event types and location data contained in the audio; The environmental mapping module is used to determine the rescue target based on the output of the pre-trained sound source localization model, and to perceive and map the environment of the rescue site by integrating data collected by the rescue drone, so as to obtain an environmental model of the rescue area. Path planning module: Based on the location information of the rescue target and the environmental model of the rescue area, it uses a secondary optimization path planning algorithm to formulate the rescue route of the rescue robot, and uses a dynamic window method to realize dynamic obstacle avoidance during the robot's movement.

Images