Robot inspection method, device, storage medium and robot

Abstract

A robot inspection method, device, storage medium and robot. The method includes obtaining a current scene sound image of the current scene in response to an abnormal sound in the current scene; determining the current position of the sound source corresponding to the abnormal sound according to the current scene sound image; driving the robot to move toward the sound source according to the current position; imaging the sound source to obtain a sound source inspection image of the sound source, wherein the sound source inspection image includes a sound source visual image and a sound source sound image. The present application can improve the efficiency of robot inspection.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a robot inspection method, device, storage medium and robot. Background Art

[0002] In industrial environments such as substations, mining areas, and factories, it is necessary to regularly inspect the facilities in the industrial environment to ensure the normal operation of the facilities.

[0003] With the rapid development of robots, considering the environmental complexity and inspection danger of industrial environments, robots are now used in industrial environments to replace manual inspections.

[0004] During robot inspection, it is necessary to manually remotely control the robot to move to a specified position in an industrial environment, which affects the inspection efficiency of the robot. Summary of the invention

[0005] The present application provides a robot inspection method, device, storage medium and robot, which can improve the robot's inspection efficiency.

[0006] The robot inspection method provided by this application includes:

[0007] In response to an abnormal sound of a current scene, obtaining a current scene sound-image of the current scene;

[0008] The current scene sound image, to determine the current location of the abnormal sound corresponding to the sound source;

[0009] According to the current position, drive the robot to move towards the sound source;

[0010] The sound source is imaged to obtain a sound source inspection image of the sound source, which includes a sound source visual image and a sound source acoustic image.

[0011] The robot inspection device provided in this application includes:

[0012] A data acquisition module, for acquiring a current scene sound-image of the current scene in response to an abnormal sound of the current scene;

[0013] The sound source localization module is used to determine the current location of the sound source corresponding to the abnormal sound based on the sound image of the current scene;

[0014] The robot control module is used to drive the robot to move towards the sound source according to the current position;

[0015] The imaging control module is used to image the sound source to obtain a sound source inspection image of the sound source, wherein the sound source inspection image includes a sound source visual image and a sound source acoustic image.

[0016] The storage medium provided in the present application stores a computer program thereon, and when the computer program is loaded by a processor, the steps in the robot inspection method provided in the present application are executed.

[0017] The robot provided in the present application includes a processor and a memory, the memory stores a computer program, and the processor is used to execute the steps in the robot inspection method provided in the present application by loading the computer program.

[0018] In the present application, when there is an abnormal sound in the current scene, the current scene sound-image diagram of the current scene is obtained, and the current position of the sound source corresponding to the abnormal sound is determined according to the current scene sound-image diagram. Afterwards, based on the current position of the sound source in the current scene, the robot is driven to move toward the sound source, so that the robot is close to the sound source to image the sound source. Specifically, by imaging the sound source, a sound source inspection image of the sound source is obtained, and the sound source inspection image includes a sound source visual image and a sound source sound-image diagram, so that the abnormal cause of the sound source is obtained by analyzing the sound source visual image and the sound source sound-image diagram. The present application can autonomously find the sound source of the abnormal sound through the robot, realize the automatic inspection of the robot, and does not require manual intervention to control the movement of the robot. Compared with the prior art, the present application avoids the need for manual uninterrupted control of the robot movement, improves the efficiency of the robot inspection, and can perform the inspection task more flexibly according to the current environment. In addition, it also avoids the disadvantage of incomplete inspection caused by subjective judgment when manually controlling the robot, so as to inspect the current scene more comprehensively and improve the effectiveness of the robot inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 Schematic diagram of the robot inspection system provided in an embodiment of the present application.

[0021] Figure 2 Another schematic diagram of the robot inspection system provided in an embodiment of the present application.

[0022] Figure 3 A flowchart of a robot inspection method provided in an embodiment of the present application.

[0023] Figure 4 A schematic diagram of determining the current position of a sound source based on a world coordinate system in a robot inspection method.

[0024] Figure 5Schematic diagram of the imaging distance in the robot inspection method.

[0025] Figure 6 Schematic diagram of determining imaging parameters based on the sound source environment in the robot inspection method.

[0026] Figure 7 This is a schematic diagram of planning the robot's moving path according to the current position in the robot inspection method.

[0027] Figure 8 A schematic diagram of determining the moving path of the robot in the robot inspection method.

[0028] Fig. 9 Another schematic diagram of determining the moving path of the robot in the robot inspection method.

[0029] Fig.10 This is another schematic diagram of determining the moving path of the robot in the robot inspection method.

[0030] Fig.11 This is a schematic diagram of driving the robot through an external device in the robot inspection method.

[0031] Fig.12 Detailed flowchart of the robot inspection method.

[0032] Fig.13 It is a structural block diagram of the robot inspection device provided in an embodiment of the present application.

[0033] Fig.14 It is a structural block diagram of the robot provided in the embodiment of the present application. DETAILED DESCRIPTION

[0034] It should be noted that the principles of the present application are illustrated by implementing in an appropriate computing environment. The following description is based on the illustrated specific embodiments of the present application, which should not be construed as limiting other specific embodiments not described in detail herein. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0035] The relational terms such as first and second involved in the following embodiments of the present application are only used to distinguish one object or operation from another object or operation, and are not used to limit the actual sequential relationship between these objects or operations. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0036] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0037] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes machine learning (ML) technology, among which deep learning (DL) is a new research direction in machine learning. It is introduced into machine learning to make it closer to the original goal, namely artificial intelligence. At present, deep learning is mainly used in computer vision, natural language processing and other fields.

[0038] A robot is an intelligent machine that can work semi-autonomously or fully autonomously. A robot can perform tasks such as work or movement through programming and automatic control. Among them, robots can be divided into inspection robots, entertainment robots, sweeping robots, etc. according to their uses. Robots can also be named according to their shapes, such as robot dogs, robot cats, etc.

[0039] In this application, the robot can perform inspection tasks in various scenarios. For example, in the power inspection scenario, the robot inspects the power supply of the power supply equipment, wherein the abnormal conditions in the power inspection scenario include discharge conditions such as corona, electric traces, and arcs in the power supply equipment, as well as leakage, fire, hanging foreign objects, and instrument alarms in the power supply line. For another example, in the tunnel inspection scenario, the robot inspects the internal conditions of the tunnel, wherein the abnormal conditions in the tunnel inspection scenario include tunnel collapse, surface water seepage, etc. For another example, in the pipeline inspection scenario, the robot inspects the transportation conditions of the pipeline, wherein the abnormal conditions in the pipeline inspection scenario include gas or liquid leakage, loose valves, and foreign object alarms in the pipeline. For another example, in the industrial inspection scenario, the robot inspects the operation of industrial equipment and the industrial environment, wherein the abnormal conditions in the industrial inspection scenario include industrial equipment failure alarms, abnormal noises during the operation of industrial equipment, fire or explosion warnings in industrial scenarios, noise pollution, etc.

[0040] Among them, using robots to replace manual inspections can save manpower on the one hand, and ensure the safety of personnel on the other hand. However, the relevant technology also requires manual remote control of robots to perform inspection tasks, which affects the inspection efficiency. In order to improve the efficiency of robot inspections, this application introduces acoustic imaging into robot inspections, and accordingly provides a robot inspection method, a robot inspection device, a storage medium, and a robot. Among them, the robot inspection method can be executed by a robot.

[0041] Among them, acoustic imaging is based on microphone array measurement technology. By measuring the signal phase difference of sound waves reaching each microphone in a certain space, the position of the sound source is determined according to the phased array principle, the amplitude of the sound source is measured, and the distribution of the sound source in space is displayed in the form of an image, that is, a spatial sound field distribution cloud map - an acoustic image map is obtained, in which the color and brightness of the image represent the strength. Among them, the acoustic imaging component integrates the acoustic imaging function.

[0042] In this application, the inspection scene where the robot is located is referred to as the current scene. The acoustic imaging component and the robot can be set as one or separately. When the acoustic imaging component and the robot are set as one, please refer to Figure 1 , Figure 1 Schematic diagram of the robot inspection system provided in the embodiment of the present application. The robot inspection system includes a robot 110, and the robot 110 is used to locate the sound source of the current scene through its own acoustic imaging component 120 to obtain the current scene sound image of the current scene. For example, the sound sources distributed in the current scene are detected, and the spatial distribution of the sound sources is imaged, and the spatial distribution of the sound sources in the current scene is displayed in the form of a current scene sound image.

[0043] Optionally, the current scene can also be photographed by the acoustic imaging component 120 to obtain a current scene visual image of the current scene, wherein the current scene visual image can be an image or a video. The current scene visual image of the current scene and the current scene acoustic image are the imaging results of imaging the current scene, and the imaging result is called the scene inspection image of the current scene.

[0044] Afterwards, the robot 110 determines the current position of the sound source through the current scene audio-visual image of the current scene, and then moves toward the sound source to image the sound source and obtain a sound source inspection image of the sound source, which includes a sound source visual image and a sound source audio-visual image.

[0045] In addition, the robot inspection system also includes external equipment, Figure 1In the embodiment, the mobile phone 130 is used as an external device, and the robot 110 can be remotely controlled through the mobile phone 130. Moreover, the robot 110 can send at least one of the image data such as the current scene visual image, the current scene sound image, the sound source visual image, and the sound source sound image to the mobile phone 130, and the mobile phone 130 presents such image data in the corresponding application interface after receiving it. Among them, the mobile phone 130 is in communication connection with the robot 110.

[0046] Optionally, the mobile phone 130 can also be connected to a remote control handle 140, and the application loaded on the mobile phone 130 can be controlled by the remote control handle 140, thereby remotely controlling the robot 110. The remote control handle 140 is connected to the mobile phone 130, and the connection method can be WIFI connection, Bluetooth connection, NFC connection, etc.

[0047] When the acoustic imaging component is set up separately from the robot, refer to Figure 2 , Figure 2 Another schematic diagram of a robot inspection system provided in an embodiment of the present application. The robot inspection system includes an acoustic imaging component 120, which is used to receive a control signal from the robot 110 to image the current scene and obtain a scene inspection image of the current scene. Alternatively, it is used to image the sound source and obtain a sound source inspection image of the sound source, and the sound source inspection image includes a sound source visual image and a sound source sound image. In addition, the acoustic imaging component 120 is also used to send image data such as the current scene visual image (optional), the current scene sound image, the sound source visual image, and the sound source sound image to the robot 110.

[0048] In addition, the robot inspection system also includes a robot 110, which is used to power the acoustic imaging component 120, and the two are electrically connected. The robot 110 is also used to receive the current scene visual image (optional) and the current scene sound image sent from the acoustic imaging component 120, and determine the current position of the sound source in the current scene according to the current scene sound image, so as to move to the current position of the sound source, thereby controlling the acoustic imaging component 120 to image the sound source, and receiving the sound source visual image and the sound source sound image sent from the acoustic imaging component 120 to determine the abnormal cause of the sound source.

[0049] Optionally, the robot inspection system may further include external equipment, Figure 2 In the example, the mobile phone 130 is used as an external device, and the mobile phone 130 is used to receive image data such as the current scene visual image (optional), the sound source visual image, and the sound source audio-visual image sent by the robot 110. After receiving such image data, the mobile phone 130 presents it in the corresponding application interface.

[0050] Optionally, the robot inspection system may further include a remote control handle 140 , and the remote control handle 140 is used to control an application loaded on the mobile phone 130 , thereby remotely controlling the robot 110 and / or the acoustic imaging component 120 .

[0051] It should be noted that Figure 1 and Figure 2 The schematic diagram of the robot inspection system shown is only an example. The robot inspection system described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. It is known to those skilled in the art that with the evolution of the robot inspection system, the technical solution provided by the embodiment of the present application is also applicable to similar technical problems. Figure 2 The robot inspection system shown is used as an example to explain the robot inspection method provided in an embodiment of the present application.

[0052] Please refer to Figure 3 , Figure 3 The following is a flow chart of the robot inspection method provided in the embodiment of the present application. Figure 3 As shown, the process of the robot inspection method provided in the embodiment of the present application can be as follows:

[0053] In S210 , in response to an abnormal sound of a current scene, a current scene sound image of the current scene is acquired.

[0054] The robot collects the sounds in the current scene through the acoustic imaging component, and analyzes the sounds in the current scene to determine whether they are abnormal sounds. In the inspection scene, abnormal sounds can be noise, abnormal noises of equipment, leakage sounds, etc.

[0055] In this embodiment, the frequency or frequency band of the abnormal sound can be preset, and then it is determined whether the collected sound matches the preset abnormal sound. If so, the sound is determined to be an abnormal sound. Alternatively, the collected sound can be input into a pre-trained neural network model for discrimination to determine whether the sound is an abnormal sound. It can be understood that there are many ways to determine whether the sound in the current scene is an abnormal sound, which are not listed here.

[0056] When the robot determines that an abnormal sound is detected, the robot detects the sound source in the current scene through the acoustic imaging component, and visualizes the spatial distribution of the sound source through the imaging algorithm to obtain the current scene acoustic image of the current scene, wherein the current scene acoustic image can reflect the current position of the sound source in the current scene.

[0057] Exemplarily, the acoustic imaging component can also shoot the current scene while detecting the sound source of the current scene to obtain a scene inspection image of the current scene, which includes a visual image of the current scene and an acoustic image of the current scene. The acoustic imaging component can also image the current scene to obtain a scene inspection image of the current scene when an abnormal sound is detected. It can be understood that when the acoustic imaging component can detect the sound source distributed in the current scene, the sound source can be displayed through the current scene acoustic image. Therefore, in a further example, when an abnormal sound is detected, the direction of the abnormal sound is located, and then the robot is controlled to face the direction, and then the current scene is imaged, so that a scene inspection image showing the sound source can be obtained.

[0058] Among them, the current scene audio-visual image and the current scene visual image of the current scene can be stored locally or in the cloud.

[0059] In S220, the current position of the sound source corresponding to the abnormal sound is determined according to the current scene sound image.

[0060] Among them, the current scene sound image shows the spatial distribution of the sound source in the current scene. According to the spatial distribution of the sound source, the spatial position of the sound source relative to the robot can be determined, wherein the spatial position of the sound source relative to the robot is called the current position of the sound source.

[0061] In S230 , the robot is driven to move toward the sound source according to the current position.

[0062] After the current position of the sound source is determined, the robot can be driven to move toward the sound source. The robot can move by rollers, walking, crawling, etc., depending on the driving structure of the robot.

[0063] In this embodiment, by driving the robot to move toward the sound source, the distance between the robot and the sound source can be shortened, so that the robot can approach the sound source to detect it and identify the details of the sound source, which can improve the accuracy of abnormal analysis of the sound source.

[0064] In S240, the sound source is imaged to obtain a sound source inspection image of the sound source, where the sound source inspection image includes a sound source visual image and a sound source acoustic image.

[0065] The robot images the sound source by controlling the acoustic imaging component to obtain a sound source inspection image of the sound source, wherein the sound source inspection image includes a sound source visual image that displays the sound source content, and the sound source visual image displays the spatial distribution of the sound source. The sound source inspection image can be an image before the fusion or superposition of the sound source visual image and the sound source acoustic image, or an image after the fusion or superposition, and the fusion or superposition process can be implemented by the acoustic imaging component or by the robot, which is not limited here.

[0066] As described above, the robot inspection method provided in the embodiment of the present application can obtain more details of the sound source by imaging the sound source at a close distance, thereby facilitating the analysis of the abnormal cause of the sound source. Compared with imaging the sound source at a long distance, the sound inspection image obtained in this embodiment can show more details and features of the sound source, and when performing abnormal analysis on the sound source, the accuracy of the abnormal analysis can be improved. In addition, this embodiment can accurately and quickly find abnormal sound sources by automatically searching for sound sources and imaging them through the robot, avoiding the ineffective movement of the robot in the current scene, and avoiding the omission of abnormal sound sources, greatly improving the efficiency of robot inspections, realizing all-round abnormal inspections, and improving the effectiveness of inspections.

[0067] In specific implementation, the present application is not limited by the execution order of the various steps described. If no conflict occurs, some steps can be performed in other orders or simultaneously.

[0068] In an optional embodiment, determining the current position of the sound source corresponding to the abnormal sound according to the current scene sound image includes:

[0069] Construct a world coordinate system based on the robot's position;

[0070] and the current scene sound image to determine the current position of the sound source in the world coordinate system.

[0071] In this embodiment, the current position of the sound source is described by constructing a world coordinate system. Figure 4 , Figure 4 The figure is a schematic diagram of determining the current position of the sound source based on the world coordinate system in the robot inspection method. The world coordinate system is a three-axis spatial coordinate system, which takes the position of the robot as the origin. During the movement of the robot, the origin remains unchanged. The sound image of the current scene is transformed from the image coordinate system to the world coordinate system to obtain the position of the sound source in the world coordinate system, wherein the current position of the sound source is represented by the coordinate value in the world coordinate system. Figure 4 As shown, in the world coordinate system, the robot moves on the plane formed by the xy axes, the coordinate of the robot on the z axis represents its height, and the height of the sound source and the height of the robot may be different.

[0072] For example, driving the robot to move with the current position as a reference means driving the robot to move on a two-dimensional plane (the xy plane of the world coordinate system) with the coordinate value of the sound source (three-dimensional space coordinates) as a reference. Of course, if the robot is an intelligent machine such as a drone that does not rely on the ground to move, it means driving the robot to move in space.

[0073] In an optional embodiment, imaging the sound source to obtain a sound source inspection image of the sound source includes:

[0074] Determine the imaging distance to the sound source;

[0075] If the real-time distance between the robot and the sound source reaches the imaging distance, the sound source is imaged to obtain a sound source inspection image of the sound source.

[0076] After determining the imaging distance, the robot moves toward the sound source, and the real-time distance between the two tends to decrease. When the real-time distance is equal to or less than the imaging distance, the sound source can be imaged. It can be understood that with the sound source as the dot and the imaging distance as the radius, an area can be circled on the xy plane, which is called the imaging area, and any coordinate point on and within the imaging area can be called an imaging point.

[0077] It can be understood that when the purpose of determining the imaging area is to instruct the robot to move to any position within the imaging area (including the edge of the imaging area), the sound source can be imaged without limiting the position, viewing angle, time, etc. of imaging the sound source, and the sound source can be imaged more flexibly.

[0078] Among them, the imaging distance refers to the distance between the position where the robot images the sound source and the sound source. The imaging distance can be a straight-line distance in three-dimensional space or a distance on the xy plane (that is, the distance between the projection of the imaging position on the xy plane and the projection of the sound source on the xy plane).

[0079] See also Figure 5 , Figure 5 Figure 1 is a schematic diagram of the imaging distance in the robot inspection method. Taking any imaging point on the imaging area as an example, when the robot moves to the imaging point, the real-time distance between the robot and the sound source is the imaging distance. Figure 5 (a) The imaging distance shown is the straight-line distance in three-dimensional space. Figure 5 (b) The imaging distance shown is the distance on the xy plane, where the current position of the sound source is projected on the xy plane to obtain the projection coordinates (represented by dots in the figure), and the distance between the projection coordinates and the imaging point is the imaging distance.

[0080] Exemplarily, there are many ways to determine the imaging distance, such as setting a fixed imaging distance. For another example, multiple imaging distances are set, and the mapping relationship between the imaging distance and the current position of the sound source is set, and then a target imaging distance is matched according to the current position of the sound source, so that when the real-time distance between the robot and the sound source is less than or equal to the target imaging distance, the sound source is imaged. For another example, the imaging distance of the sound source can also be set by the user, or the imaging distance of the sound source can be predicted by a pre-trained neural network model. Since there are many ways to determine the imaging distance, they are not limited here. It can be understood that any method that can determine the imaging distance can be used in the embodiments of the present application and belongs to the protection scope required by the present application.

[0081] In an optional embodiment, after determining the imaging distance to the sound source, the method further includes:

[0082] Determine an imaging area according to the imaging distance, and determine a target imaging point from the imaging area;

[0083] When the robot moves to the target imaging point, the sound source is imaged to obtain a sound source inspection image of the sound source.

[0084] As described above, an imaging area can be determined according to the imaging distance, and the robot can move to any imaging point in the imaging area to image the sound source. In this embodiment, a solution for determining a target imaging point is provided.

[0085] For example, the imaging point that overlaps with the projection of the current position of the sound source on the xy plane is used as the target imaging point. For another example, the sound source is identified according to the current position of the sound source, and then the preset target imaging point of the sound source is obtained. For another example, according to the current position of the robot, the imaging point closest to the position is used as the target imaging point. For another example, in combination with the current scene, the imaging point with a better imaging perspective is used as the target imaging point, where the basis for a better imaging perspective includes: clear imaging, more imaging details, etc.

[0086] It can be understood that there are one or more target imaging points. When there are multiple target imaging points, the robot can move to each target imaging point one by one to image the sound source.

[0087] In this embodiment, by determining the target imaging point from the imaging area, and then accurately driving the robot to move to the target imaging point, the sound source is imaged, thereby avoiding the need to continuously image the sound source, avoiding the generation of a large number of invalid sound source inspection images, saving power consumption for robot movement, power consumption for imaging by the sound imaging component, and also saving storage space for sound source inspection images, as well as saving the inspection time of the sound source, thereby improving inspection efficiency.

[0088] In an optional embodiment, the sound source is imaged to obtain a sound source inspection image of the sound source, the sound source inspection image including a sound source visual image and a sound source acoustic image, including:

[0089] Identify the sound source environment in which the sound source is located;

[0090] Determine the imaging parameters of the sound source according to the sound source environment;

[0091] The sound source is imaged according to the imaging parameters to obtain a sound source inspection image of the sound source.

[0092] In this embodiment, a method for imaging a sound source is provided.

[0093] The robot identifies the sound source environment where the sound source is located. Specifically, by identifying the location of the sound source in the sound source environment, the sound source's obstructions, the front and side information of the sound source, and whether there is a location in the sound source environment suitable for imaging the sound source, in other words, whether there is a location in the sound source environment suitable for accommodating the robot.

[0094] Exemplarily, the imaging parameters may be determined according to preset conditions. The imaging parameters may include imaging viewing angle, imaging focal length, etc. When the imaging parameters include imaging viewing angle, the preset conditions include: selecting an imaging viewing angle without obstructions, selecting an imaging viewing angle with more sound source details, and selecting an imaging viewing angle in front of the sound source. It is understandable that the position and posture of the robot can be changed. After determining the imaging viewing angle of the robot when imaging the sound source, the robot can change its position (the changed position is regarded as the target imaging point) and posture so that the sound source inspection image can be obtained at this imaging viewing angle. Please refer to Figure 6 , Figure 6 This is a schematic diagram of determining imaging parameters according to the sound source environment in the robot inspection method. If an imaging angle without obstructions is selected, Figure 6 In the figure, the side of the sound source without any obstruction is indicated by the arrow. By driving the robot to move to the direction indicated by the arrow to image the sound source, an inspection image of the sound source without any obstruction can be obtained. After the robot moves to the direction indicated by the arrow, its position can be fine-tuned, or its posture can be adjusted to image the sound source.

[0095] In addition, when the imaging parameters include imaging focal length, the preset conditions include: selecting a focal length that only includes the sound source, selecting a focal length that includes local details of the sound source, etc. It can be understood that after determining different imaging focal lengths, sound inspection images showing sound sources of different sizes can be obtained.

[0096] It can be understood that after the imaging parameters are determined, the robot can be driven to move or adjust its posture in the imaging area according to the imaging parameters to capture the sound source and obtain a high-quality sound inspection image.

[0097] The embodiment of the present application can improve the imaging quality by determining imaging parameters and then imaging the sound source according to the imaging parameters, so as to obtain richer information through the sound source inspection image to detect abnormalities of the sound source, thereby improving the accuracy of abnormality detection.

[0098] In an optional embodiment, driving the robot to move toward the sound source according to the current position includes:

[0099] Get the map data of the current scene;

[0100] Determine the robot's moving path based on the world coordinate system, map data, and current position;

[0101] According to the moving path, the robot is driven to move towards the sound source.

[0102] Among them, the map data can display the environment, facilities, obstacles and other contents. After the robot obtains the map data of the current scene, it can perform coordinate system conversion on the map data to convert it into a scene map in the world coordinate system. The type of the scene map can be a grid map, a feature map, etc. Afterwards, one or more movement paths are planned according to the scene map in the world coordinate system and the current position of the sound source. After obtaining the movement path of the robot, the robot moves on the movement path by adjusting its movement direction, so as to move toward the sound source through automatic navigation.

[0103] For example, when planning the movement path of the robot, the robot's location can be used as the starting point and the target imaging point can be used as the end point. After determining the starting point and the end point, the movement path can be planned according to the scene map, wherein obstacle avoidance can also be performed according to the road conditions during planning.

[0104] See also Figure 7 , Figure 7 The figure is a schematic diagram of planning the robot's moving path according to the current position in the robot inspection method. The robot's position is the starting point, and the target imaging point is the end point (represented by a circle in the figure, there can be one or more target imaging points). According to the starting point and the end point, one or more moving paths can be planned on the scene map (only one moving path is shown in the figure). During the movement, the robot adjusts its moving posture to move along the moving path to the end point of the moving path, thereby moving close to the sound source.

[0105] Of course, when there are multiple moving paths, one or more preferred paths can be selected from the multiple moving paths, wherein the preferred path can meet the conditions of shorter moving time, less moving difficulty, and wider shooting angle.

[0106] It can be understood that there may be one or more sound sources in the current scene displayed by the scene inspection image. When there is only one sound source, the above process S210-S240 can be executed. When there are multiple sound sources, the movement path can be planned according to the spatial distribution of the multiple sound sources, and then the robot can be driven to move toward each sound source one by one according to the movement path, and then each sound source is imaged. In the following embodiment, the situation with multiple sound sources is described.

[0107] In an optional embodiment, determining the movement path of the robot according to the world coordinate system, the map data and the current position includes:

[0108] If the current positions of the multiple sound sources are determined, the distance to be moved from the robot to each sound source is determined based on the world coordinate system, the map data, and the current position of each sound source;

[0109] Determine the robot's moving path based on the world coordinate system, map data, the current position of each sound source, and the distance to be moved from the robot to each sound source;

[0110] According to the moving path, drive the robot to move towards the sound source, including:

[0111] According to the moving path, the robot is driven to move towards each sound source one by one.

[0112] In this embodiment, after knowing the current positions of the multiple sound sources, the distribution of each sound source in the world coordinate system can be determined. According to the world coordinate system and the distribution of each sound source, the target imaging point of each sound source can be determined. Then, the distance to be moved from the robot to each sound source is determined according to the target imaging point of each sound source.

[0113] For example, the straight-line distance between the target imaging point and the robot can be used as the distance to be moved between the sound source corresponding to the target imaging point and the robot. According to the coordinates of each target imaging point and the robot in the world coordinate system, the path from the robot to the target imaging point can be found on the scene map, and the path is called the distance to be moved from the robot to each sound source, wherein the path is the path that the robot can move in the current scene and avoid obstacles.

[0114] After the distance to be moved is determined, the distances to be moved can be arranged in ascending order according to their sizes, and the moving path is planned in ascending order, wherein the moving path is planned from near to far according to the distance to be moved, that is, the planned moving path instructs the robot to move first to the target imaging point closest to it. It can be understood that the moving path includes the target imaging points of each sound source, and obstacle avoidance is also performed according to the scene map, so that the robot can move smoothly in the moving path.

[0115] See also Figure 8 , Figure 8 This is a schematic diagram of determining the robot's moving path in the robot inspection method. Figure 8 As shown, a scene map, the current position of each sound source, the robot's position as the origin, the target imaging point of each sound source (taking one sound source as an example, one target imaging point) and the moving path are displayed in the world coordinate system. When the robot moves along the moving path by adjusting its moving direction and posture, each time it reaches a target detection point, it can image the corresponding sound source at the target detection point to obtain a sound inspection image of the sound source.

[0116] This embodiment plans a moving path from near to far according to the distance to be moved between each target detection point and the robot, so that the robot can sequentially image each sound source, thereby optimizing the robot's moving path and improving the robot's inspection efficiency.

[0117] In an optional embodiment, determining the movement path of the robot according to the world coordinate system, the map data and the current position includes:

[0118] If the current positions of the multiple sound sources are determined, the urgency of each sound source is evaluated;

[0119] Determine the robot's movement path based on the world coordinate system, map data, and the current location and urgency of each sound source;

[0120] According to the moving path, drive the robot to move towards the sound source, including:

[0121] According to the moving path, the robot is driven to move towards each sound source one by one.

[0122] Among them, the urgency of each sound source can be determined based on its abnormal sound or its properties. For example, if the sound of the sound source reflects a violent noise of the sound source, it can be determined that the urgency of the sound source is high. If the sound of the sound source reflects that the sound source is loud but not violent, it can be determined that the urgency of the sound source is average. For another example, if the sound source indicates that it is an important device, the urgency of the important device is high. If the sound source indicates that it is a non-important device, the urgency of the non-important device is low. It can be understood that the way to evaluate the urgency of the sound source can be to analyze the sound of the sound source or to analyze the properties of the sound source, and then match the urgency of the corresponding sound source from the preset urgency according to the analysis results.

[0123] In this embodiment, after the degree of urgency is identified, the sound sources can be sorted according to the degree of urgency, with the sound sources with the highest degree of urgency sorted first, and then as mentioned in the above example, a moving path is planned according to the sorting of the degrees of urgency, the target imaging points corresponding to each sound source, the scene map, etc., and then the robot is driven to move to each target detection point along the moving path to image the corresponding sound source at each target detection point to obtain a sound source inspection image of each sound source.

[0124] See also Fig. 9 , Fig. 9 Another schematic diagram for determining the movement path of the robot in the robot inspection method. In the world coordinate system, a scene map, the current position of each sound source, the robot's position as the origin, the target imaging point of each sound source (taking one sound source as an example, one target imaging point), and the movement path are displayed. When the robot moves along the movement path by adjusting its movement direction and posture, it moves to each target detection point one by one according to the urgency, so as to obtain the sound inspection images of the sound sources with decreasing urgency. Fig. 9 Among them, the black star has the highest urgency, so the target detection point of the black star is taken as the first imaging point.

[0125] In this embodiment, in the process of moving along the inspection path to image the sound source, the sound source inspection image of the sound source with the highest urgency is obtained first, so that by analyzing the sound source inspection image, the abnormal cause of the sound source with the highest urgency can be obtained, and timely measures can be taken to deal with it, which can timely avoid major abnormalities or greatly reduce the impact of the abnormalities.

[0126] In an optional embodiment, determining the movement path of the robot according to the world coordinate system, the map data and the current position includes:

[0127] If the current positions of the multiple sound sources are determined, the urgency of each sound source is evaluated;

[0128] For at least two sound sources of the same urgency level, determining the distances to be moved by the robot to the at least two sound sources;

[0129] Determine a movement path of the robot according to the world coordinate system, the map data, the current position and urgency of each sound source, and the distance to be moved from the robot to at least two sound sources;

[0130] According to the moving path, drive the robot to move towards the sound source, including:

[0131] According to the moving path, the robot is driven to move towards each sound source one by one.

[0132] This embodiment combines the urgency of each sound source and the distance to be moved between each sound source and the robot. When planning the moving path, the urgency is considered first, and the target imaging point corresponding to the sound source with the highest urgency is set at the position closest to the starting point of the moving path. For sound sources with the same urgency, the target imaging point corresponding to the sound source with the closest distance to be moved between the robot and the robot is set at the position closest to the starting point. In this way, the moving path can be reasonably planned. On the one hand, it takes into account the emergency situation during inspection in the current scene, and on the other hand, it takes into account the optimization of the number of robot movements, so that the inspection task can be performed more reasonably.

[0133] See also Fig.10 , Fig.10 Another schematic diagram of determining the moving path of the robot in the robot inspection method. Fig. 9 and Figure 8 In terms of Fig.10 The displayed moving path can provide a more reasonable inspection method for the robot. Fig. 9 and Figure 8 Advantages of mobile paths.

[0134] In an optional embodiment, the method further includes:

[0135] In the process of driving the robot to move toward the sound source, a real-time scene sound image of the real-time scene where the robot is located is obtained;

[0136] Determine the real-time location of the sound source based on the real-time scene sound image;

[0137] Update the movement path based on real-time position, world coordinate system and map data.

[0138] Among them, in the process of driving the robot to move towards the sound source, the moving path referenced by the robot is dynamically changed according to the real-time position of the sound source in the world coordinate system, that is, the moving path is gradually shortened. When the robot moves, the real-time scene where the robot is located is changing. During the movement of the robot, the sound source of the real-time scene can be continuously detected by the acoustic imaging component to obtain a real-time scene sound image of the real-time scene, and then the real-time position of the sound source can be determined according to the real-time scene sound image. After the real-time position of the sound source is determined, the real-time position of the sound source can be displayed on the world coordinate system, and the moving path can be updated according to the real-time scene map on the world coordinate system, so that the robot can continue to navigate autonomously.

[0139] It can be understood that when there are multiple sound sources, the moving path can be updated according to the real-time position of each sound source, as well as the world coordinate system and map data, so that the robot moves toward each sound source, and then images each sound source to obtain a sound source inspection image of each sound source.

[0140] In an optional embodiment, after determining the movement path of the robot according to the world coordinate system, the map data and the current position, the method further includes:

[0141] sending the moving path to a first external device;

[0142] In response to the correction path returned by the first external device according to the moving path, the robot is driven to move according to the correction path.

[0143] Among them, Figure 2 As shown in the figure, the robot inspection system also includes external devices, through which users can remotely control the robot. Fig.11 , Fig.11 The schematic diagram of driving the robot by an external device in the robot inspection method is shown in the figure. In the figure, the external device is controlled by the user, and the external device is connected to the robot for communication. The robot can send the scene inspection image and sound source inspection image it obtains to the external device, and can also send the planned moving path to the external device, and can also send the video image of the robot moving along the moving path to the external device.

[0144] The external device can receive user manipulation to control the robot, for example, control the robot to pause movement, control the robot to pose in various postures, control the robot to move in a specified direction, etc. For another example, the robot's moving path can also be controlled to be adjusted. The adjustment method is, for example, to adjust the moving path received by the external device, and then send the adjusted corrected path to the robot to drive the robot to move along the corrected path. Alternatively, by selecting a position point in the video image presented on the external device, the position point is used as a correction instruction. When the robot receives the correction instruction, the moving path is corrected according to the position point, specifically, the moving path includes the position point. For another example, the robot's imaging parameters can also be controlled to be adjusted, for example, the shooting angle is determined by the external device, and then the robot is controlled to image the sound source according to the shooting angle. Since there are many ways to interact with the robot through an external device, they are not listed here.

[0145] As mentioned above, when there is no abnormal sound in the current scene, the inspection based on the preset inspection path can be restored, and the robot can be driven to move along the preset inspection path. Specifically, there is a preset inspection path, and the robot performs the inspection task according to the preset inspection path. When an abnormal sound is detected in the current scene, the inspection based on the preset inspection path is suspended. In addition, the robot is driven to move toward the sound source corresponding to the abnormal sound. In the process of moving close to the sound source, it can also be detected whether there is a new sound source corresponding to the abnormal sound in the real-time scene, and then the robot is continuously driven to move according to the new sound source appearing in the real-time scene. When there is no new sound source corresponding to the abnormal sound in the real-time scene, the inspection based on the preset inspection path is restored until the sound source corresponding to the new abnormal sound is detected or the inspection of the preset inspection path is completed.

[0146] In an optional embodiment, during the process of driving the robot to move toward the sound source, the method further includes:

[0147] A real-time scene visual image of the real-time scene is acquired, and the real-time scene visual image is sent to a second external device for display.

[0148] In the process of driving the robot to move toward the sound source, the real-time scene can also be photographed by the acoustic imaging component to obtain a real-time scene visual image of the real-time scene, and then the real-time scene visual image is sent to an external device for display, so that the user can remotely view the real-time scene where the robot is located. Of course, the real-time scene visual image and the real-time scene acoustic image of the real-time scene can also be fused and sent to an external device for display, so that the user can view the real-time scene where the robot is located and the location of the sound source in the real-time scene through the external device.

[0149] Furthermore, during the process of detecting and / or photographing the sound source of the real-time scene through the acoustic imaging component, a remote control operation sent by an external device can also be received, thereby driving the robot to perform the remote control operation.

[0150] In an optional embodiment, after imaging the sound source and obtaining the sound source inspection image of the sound source, the method further includes:

[0151] sending the sound source inspection image to a third external device for display;

[0152] In response to the visual control operation returned by the third external device based on the sound source inspection image, the robot is driven to change its posture.

[0153] Specifically, by sending the sound source inspection image to an external device for display, the sound source acoustic image and the sound source visual image can be displayed on the external device, and the user can remotely control the robot by operating the external device.

[0154] Here is an example. For example, the sound source visual image displayed on the external device shows the current scene where the robot is located. The user selects a point on the sound source visual image and then uses the point as the robot's position end point. After sending a visual control operation to the robot based on the point, when the robot receives the visual control operation, it can move toward the end point indicated by the visual control operation. Alternatively, the user indicates a direction for the robot on the external device, and the external device sends a visual control operation to the robot based on the direction. When the robot receives the visual control operation, it can control its posture to change to move the perspective to the direction. It can be understood that there are many ways to remotely control the robot based on the sound source visual image displayed by the external device. In addition to controlling the robot's posture to change, it is also possible to control the sensor on the robot to enable, or control the robot's shooting parameters to change, or control the robot to make a specified posture, etc. Since there are many controllable methods, they are not listed here.

[0155] In summary, please refer to Fig.12 , Fig.12 The figure is a detailed flow chart of the robot inspection method. When the robot starts to move, it moves normally according to the preset inspection path. During the movement, the robot also detects whether there is any abnormal sound in the current scene. If not, it continues to move normally. If so, it determines the current position of the sound source corresponding to the abnormal sound and moves towards the sound source. After moving to the target imaging point, the sound source can be imaged at the target imaging point through the sound imaging component to obtain the sound inspection image of the sound source. This cycle is repeated until no abnormal sound is detected, and the movement towards the sound source ends.

[0156] The robot inspection method provided in the embodiment of the present application can, on the one hand, enable the robot to autonomously move in the current scene by searching for the source of abnormal sound, thereby improving the flexibility, comprehensiveness, effectiveness and efficiency of the inspection. On the other hand, by setting imaging parameters, moving paths, etc., to optimize the moving path of the robot, while improving the inspection efficiency, a higher quality sound source inspection image can be obtained, which is convenient for abnormal analysis of the sound source, improves the accuracy of determining the cause of the abnormal sound source, and improves the efficiency of emergency treatment of the sound source by optimizing the inspection path.

[0157] Please refer to Fig.13 In order to better implement the robot inspection method provided by the present application, the present application further provides a robot inspection device, such as Fig.13 As shown, the robot inspection device 300 includes:

[0158] The data acquisition module 310 is used to acquire a current scene sound image of the current scene in response to an abnormal sound of the current scene;

[0159] The sound source localization module 320 is used to determine the current position of the sound source corresponding to the abnormal sound according to the sound image of the current scene;

[0160] The robot control module 330 is used to drive the robot to move toward the sound source according to the current position;

[0161] The imaging control module 340 is used to image the sound source to obtain a sound source inspection image of the sound source, wherein the sound source inspection image includes a sound source visual image and a sound source acoustic image.

[0162] In an optional embodiment, the sound source localization module 320 is further configured to:

[0163] Construct a world coordinate system based on the robot's position;

[0164] According to the current scene sound image, determine the current position of the sound source in the world coordinate system.

[0165] In an optional embodiment, the robot control module 330 is further configured to:

[0166] Get the map data of the current scene;

[0167] Determine the robot's moving path based on the world coordinate system, map data, and current position;

[0168] According to the moving path, the robot is driven to move towards the sound source.

[0169] In an optional embodiment, the robot control module 330 is further configured to:

[0170] If the current positions of the multiple sound sources are determined, the distance to be moved from the robot to each sound source is determined based on the world coordinate system, the map data, and the current position of each sound source;

[0171] Determine the robot's moving path based on the world coordinate system, map data, the current position of each sound source, and the distance to be moved from the robot to each sound source;

[0172] According to the moving path, the robot is driven to move towards each sound source one by one.

[0173] In an optional embodiment, the robot control module 330 is further configured to:

[0174] If the current positions of the multiple sound sources are determined, the urgency of each sound source is evaluated;

[0175] Determine the robot's movement path based on the world coordinate system, map data, and the current location and urgency of each sound source;

[0176] According to the moving path, the robot is driven to move towards each sound source one by one.

[0177] In an optional embodiment, the robot control module 330 is further configured to:

[0178] Sending the movement path to an external device;

[0179] In response to the correction path returned by the external device according to the moving path, the robot is driven to move according to the correction path.

[0180] In an optional embodiment, the robot control module 330 is further configured to:

[0181] In the process of driving the robot to move toward the sound source, a real-time scene sound image of the real-time scene where the robot is located is obtained;

[0182] Determine the real-time location of the sound source based on the real-time scene sound image;

[0183] Update the movement path based on real-time position, world coordinate system and map data.

[0184] In an optional embodiment, the imaging control module 340 is further configured to:

[0185] Determine the imaging distance to the sound source;

[0186] If the real-time distance between the robot and the sound source reaches the imaging distance, the sound source is imaged to obtain a sound source inspection image of the sound source.

[0187] In an optional embodiment, the imaging control module 340 is further configured to:

[0188] Identify the sound source environment in which the sound source is located;

[0189] Determine the imaging parameters of the sound source according to the sound source environment;

[0190] The sound source is imaged according to the imaging parameters to obtain a sound source inspection image of the sound source.

[0191] It should be noted that the robot inspection device 300 provided in the embodiment of the present application belongs to the same concept as the robot inspection method in the above embodiment, and its specific implementation process is detailed in the above related embodiments and will not be repeated here.

[0192] The present application also provides a robot, including a memory and a processor, wherein the processor is used to execute the steps of the robot inspection method provided in the present embodiment by calling a computer program stored in the memory. The movement mode of the robot includes but is not limited to roller movement, walking movement, crawling movement, etc.

[0193] Please refer to Fig.14 , Fig.14 A schematic diagram of the structure of the robot 110 provided in an embodiment of the present application.

[0194] The robot 110 may include components such as a network interface 150, a memory 160, a processor 170, and a screen assembly. Those skilled in the art will appreciate that Fig.14 The structure of the robot 110 shown in the figure does not constitute a limitation on the robot 110, and the robot 110 may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0195] The network interface 150 may be used to establish a network connection between devices.

[0196] The memory 160 may be used to store computer programs and data. The computer programs stored in the memory 160 include executable codes. The computer programs may be divided into various functional modules. The processor 170 executes various functional applications and data processing by running the computer programs stored in the memory 160.

[0197] The processor 170 is the control center of the robot 110. It uses various interfaces and lines to connect various parts of the entire robot 110, and executes various functions of the robot 110 and processes data by running or executing computer programs stored in the memory 160 and calling data stored in the memory 160, thereby controlling the robot 110 as a whole.

[0198] In the embodiment of the present application, the processor 170 in the robot 110 will load the executable code corresponding to one or more computer programs into the memory 160 according to the following instructions, and the processor 170 will execute the steps in the robot inspection method provided by the present application, for example:

[0199] In response to an abnormal sound of a current scene, obtaining a current scene sound-image of the current scene;

[0200] According to the current scene sound image, determine the current position of the sound source corresponding to the abnormal sound;

[0201] According to the current position, the robot 110 is driven to move toward the sound source;

[0202] The sound source is imaged to obtain a sound source inspection image of the sound source, which includes a sound source visual image and a sound source acoustic image.

[0203] In addition, the robot 110 is also electrically connected to the acoustic imaging component, and controls the acoustic imaging component to detect and / or photograph sound sources, and obtain acoustic images and / or visual images obtained by the acoustic imaging component.

[0204] although Fig.14 Not shown, the robot 110 may further include a camera, which is used to photograph the environment in which the robot is located, or to perform scene recognition.

[0205] The robot 110 may also include sensors, a positioning module, a communication module, etc., which will not be described in detail here.

[0206] It should be noted that the robot 110 provided in the embodiment of the present application belongs to the same concept as the robot inspection method in the above embodiment, and its specific implementation process is detailed in the above related embodiments and will not be repeated here.

[0207] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program stored therein is executed on the processor of the robot provided in the embodiment of the present application, the processor of the robot executes any of the steps in the above robot inspection method suitable for the robot. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0208] The above is a detailed introduction to a robot inspection method, device, storage medium and robot provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A robot inspection method, characterized in that: include: In response to an abnormal sound of a current scene, obtaining a current scene sound-image diagram of the current scene; Determining the current position of the sound source corresponding to the abnormal sound according to the current scene sound image; According to the current position, driving the robot to move toward the sound source; Imaging the sound source to obtain a sound source inspection image of the sound source, wherein the sound source inspection image includes a sound source visual image and a sound source acoustic image; Wherein, determining the current position of the sound source corresponding to the abnormal sound according to the current scene sound image includes: Constructing a world coordinate system based on the position of the robot; Determining the current position of the sound source in the world coordinate system according to the current scene sound image; The step of driving the robot to move toward the sound source according to the current position includes: Obtaining map data of the current scene; Determining a moving path of the robot according to the world coordinate system, the map data and the current position; According to the moving path, driving the robot to move toward each of the sound sources one by one; Wherein, determining the movement path of the robot according to the world coordinate system, the map data and the current position includes: If the current positions of the multiple sound sources are determined, the urgency of each sound source is evaluated according to the loudness of the sound source or the importance of the device corresponding to the sound source; Determining a moving path of the robot according to the world coordinate system, the map data, and the current position and urgency of each of the sound sources; Alternatively, if the current positions of the plurality of sound sources are determined, the distance to be moved from the robot to each of the sound sources is determined according to the world coordinate system, the map data and the current position of each of the sound sources; The moving path of the robot is determined according to the world coordinate system, the map data, the current position of each of the sound sources, and the distance to be moved from the robot to each of the sound sources.

2. The robot inspection method according to claim 1, characterized in that: The step of imaging the sound source to obtain a sound source inspection image of the sound source includes: determining an imaging distance to the sound source; If the real-time distance between the robot and the sound source reaches the imaging distance, the sound source is imaged to obtain a sound source inspection image of the sound source.

3. The robot inspection method according to claim 1, characterized in that: The step of imaging the sound source to obtain a sound source inspection image of the sound source includes: Identifying a sound source environment in which the sound source is located; Determining imaging parameters of the sound source according to the sound source environment; The sound source is imaged according to the imaging parameters to obtain a sound source inspection image of the sound source.

4. The robot inspection method according to claim 1, characterized in that: The method further comprises: In the process of driving the robot to move toward the sound source, obtaining a real-time scene sound image of the real-time scene where the robot is located; Determining the real-time position of the sound source according to the real-time scene sound image; The moving path is updated according to the real-time position, the world coordinate system and the map data.

5. The robot inspection method according to claim 1, characterized in that: After determining the moving path of the robot according to the world coordinate system, the map data and the current position, the method further includes: sending the moving path to a first external device; In response to the correction path returned by the first external device according to the moving path, the robot is driven to move according to the correction path.

6. The robot inspection method according to any one of claims 1 to 3, characterized in that: The method further comprises: In the process of driving the robot to move toward the sound source, a real-time scene visual image of the real-time scene where the robot is located is obtained, and the real-time scene visual image is sent to a second external device for display.

7. The robot inspection method according to any one of claims 1 to 3, characterized in that: After imaging the sound source to obtain a sound source inspection image of the sound source, the method further includes: Sending the sound source inspection image to a third external device for display; In response to the visual control operation returned by the third external device according to the sound source inspection image, the robot is driven to change its posture.

8. A robot inspection device, characterized in that: include: A data acquisition module, configured to acquire a current scene sound-image of the current scene in response to an abnormal sound of the current scene; A sound source localization module, used to determine the current position of the sound source corresponding to the abnormal sound according to the current scene sound image; A robot control module, used for driving the robot to move toward the sound source according to the current position; An imaging control module, used for imaging the sound source to obtain a sound source inspection image of the sound source, wherein the sound source inspection image includes a sound source visual image and a sound source acoustic image; Wherein, the sound source localization module is specifically used for: Constructing a world coordinate system based on the position of the robot; Determining the current position of the sound source in the world coordinate system according to the current scene sound image; The robot control module is specifically used for: Obtaining map data of the current scene; Determining a moving path of the robot according to the world coordinate system, the map data and the current position; According to the moving path, driving the robot to move toward each of the sound sources one by one; Wherein, determining the movement path of the robot according to the world coordinate system, the map data and the current position includes: If the current positions of the multiple sound sources are determined, the urgency of each sound source is evaluated according to the loudness of the sound source or the importance of the device corresponding to the sound source; Determining a moving path of the robot according to the world coordinate system, the map data, and the current position and urgency of each of the sound sources; Alternatively, if the current positions of the plurality of sound sources are determined, the distance to be moved from the robot to each of the sound sources is determined according to the world coordinate system, the map data and the current position of each of the sound sources; The moving path of the robot is determined according to the world coordinate system, the map data, the current position of each of the sound sources, and the distance to be moved from the robot to each of the sound sources.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded by the processor, the steps of the robot inspection method according to any one of claims 1 to 7 are executed.

10. A robot comprising a processor and a memory, wherein the memory stores a computer program, wherein: The processor is used to execute the steps in the robot inspection method according to any one of claims 1 to 7 by loading the computer program.

Images