Safety inspection method based on industrial cleaning robot

Abstract

The invention relates to the technical field of industrial intelligent automation, in particular to a safety inspection method based on an industrial cleaning robot, and the method comprises the following steps: obtaining a key part list which stores the position data of each key part; controlling the cleaning robot to scan in the to-be-inspected area, acquiring an environment image of the to-be-inspected area through a visual acquisition device configured on the cleaning robot in the scanning process, acquiring real-time audio data through a microphone array group configured on the cleaning robot, and analyzing whether an abnormal equipment part exists or not; comparing the position data of each key component in the environment image with the position data of each key component in the key component list, and generating a comparison result; and according to the comparison result, whether the abnormal displacement condition exists in each key part or not is analyzed. By adopting the scheme, the cost caused by arranging a large number of monitoring devices can be reduced, and meanwhile, the state monitoring of key parts in a factory can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial intelligent automation, and in particular to a safety inspection method based on an industrial cleaning robot. Background Art

[0002] In modern industrial production, the stable operation of production equipment is the key to ensuring production efficiency and product quality. However, the damage, drop or incompleteness of key components often cause serious damage to the production process. Traditional equipment detection methods can usually only detect the existence of problems after these key components have caused damage to other facilities and equipment, and remedial measures cannot be implemented in time. This lag not only affects production efficiency, but may also lead to greater safety hazards.

[0003] In order to solve the above problems, the existing technology usually collects and analyzes the audio signals generated by various equipment in the factory during operation to achieve real-time monitoring of the status of key components and fault warning. Specifically, the existing technology often uses a high-sensitivity sound sensor array arranged near the key equipment to capture the audio signals generated by the equipment during operation. Then, advanced signal processing algorithms are used to extract features and recognize patterns on these audio signals to determine whether the operating status of the equipment is normal. Once an abnormality is detected, the system will immediately issue an alarm and provide specific fault location information so that the staff can take timely measures to deal with it. However, there are a large number of key components or parts. If monitoring and detection devices are installed on each equipment component, the cost of production safety will be too high.

[0004] In addition, for some special types of factory equipment parts, such as the hissing sound made when gas or liquid is ejected from a broken pipe, the traditional factory detection method usually determines whether there is a problem by monitoring whether the pipeline pressure valve is within the normal threshold range. However, this method cannot directly locate the specific coordinates of the broken pipeline parts, resulting in low maintenance efficiency. Moreover, if the leaked gas is harmful to the human body, there is still unsafety in monitoring and positioning.

[0005] Therefore, it is particularly important to achieve status monitoring of key components in the factory while reducing the cost of deploying a large number of monitoring devices. Summary of the invention

[0006] The present invention provides a safety inspection method based on an industrial cleaning robot, which can realize status monitoring of key components in a factory while reducing the cost of deploying a large number of monitoring devices.

[0007] In order to achieve the above objectives, this application provides the following technical solutions:

[0008] A safety inspection method based on an industrial cleaning robot comprises the following steps:

[0009] Obtaining a key component list, wherein the key component list stores location data of each key component;

[0010] Control the cleaning robot to patrol the area to be inspected, and during the patrol process, obtain the environmental image of the area to be inspected through the visual acquisition device configured on the cleaning robot;

[0011] Compare the position data of each key component in the environment image with the position data of each key component in the key component list, and generate a comparison result;

[0012] According to the comparison results, each key component is analyzed to see whether there is abnormal displacement. If so, a position abnormality alarm is generated.

[0013] Further, the patrol and sweeping includes the first patrol and sweeping;

[0014] Control the cleaning robot to patrol the area to be inspected, including:

[0015] Obtain map information of the area to be inspected and the corresponding inspection route;

[0016] The cleaning robot is controlled to perform the first inspection in the area to be inspected according to the map information and the inspection route.

[0017] Furthermore, it also includes:

[0018] Construct a 3D fusion feature map based on the environmental image of the area to be inspected.

[0019] Further, the patrol sweep also includes several conventional patrol sweeps;

[0020] Controlling the cleaning robot to patrol the area to be inspected also includes:

[0021] Based on the 3D fusion feature map constructed during the last inspection, the cleaning robot is controlled to perform routine inspections in the area to be inspected.

[0022] Furthermore, obstacle information is obtained in real time during the patrol and sweeping process, and the patrol and sweeping route is adjusted in real time according to the obstacle information.

[0023] Furthermore, it also includes:

[0024] Collect real-time audio data during the inspection process to analyze whether there are any abnormal equipment components.

[0025] Furthermore, real-time audio data is collected during the patrol process to analyze whether there are abnormal equipment components, including:

[0026] Collect real-time audio data during the patrol process to analyze whether there is abnormal noise; if so, analyze the direction of the sound source corresponding to the abnormal noise;

[0027] Controlling the visual collection device provided on the cleaning robot to turn to the direction of the sound source corresponding to the abnormal noise, and collecting the environmental image of the direction of the sound source;

[0028] Based on real-time audio data and environmental images of the sound source location, analyze abnormal equipment components that emit abnormal noise and generate abnormal operation alarms.

[0029] Furthermore, the cleaning robot is also provided with a microphone array group; real-time audio data is collected by the microphone array group configured on the cleaning robot.

[0030] The principles and advantages of the present invention are:

[0031] First, by configuring a visual acquisition device and a microphone array group on the cleaning robot, environmental images and real-time audio data can be obtained while patrolling, so as to locate abnormal components that are displaced, damaged, or operating abnormally. Compared with the prior art in which monitoring devices are set at the locations of each key component, this solution utilizes the cleaning characteristics of the cleaning robot, so that it can detect abnormalities while completing the cleaning, without the need to set up multiple monitoring devices, reducing monitoring costs, and being able to locate abnormal components in a timely manner, thereby improving the efficiency and accuracy of safety inspections.

[0032] Secondly, the method also includes the step of constructing a 3D fusion feature map. By analyzing and processing the environmental image of the inspection area, a 3D fusion feature map containing key component location data is generated and updated in real time, which is then used to guide the cleaning robot to perform regular inspections in the inspection area to improve the accuracy and efficiency of the inspection.

[0033] In addition, this method also takes into account the impact of obstacles. During the inspection process, the cleaning robot will obtain obstacle information in real time and adjust the inspection route in real time according to the obstacle information to avoid collision with obstacles and ensure the safety of the inspection process.

[0034] Finally, the method also detects abnormal equipment parts by analyzing real-time audio data. When the cleaning robot collects abnormal noise, it analyzes the sound source location corresponding to the abnormal noise and controls the visual acquisition device to turn to the sound source location to collect environmental images. By combining real-time audio data and environmental images of the sound source location, the abnormal equipment parts that emit abnormal noise can be analyzed and abnormal operation alarms can be generated. In this way, the accuracy and timeliness of detecting abnormal equipment parts can be improved.

[0035] In summary, this solution can reduce monitoring costs, improve inspection efficiency and accuracy, consider the impact of obstacles, and improve the accuracy of abnormal equipment component detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flowchart of an embodiment of a safety inspection method based on an industrial cleaning robot.

[0037] Figure 2 The present invention is a schematic diagram of the patrol process in an embodiment of a safety patrol method based on an industrial cleaning robot.

[0038] Figure 3 This is a signal flow diagram of a cleaning robot in an embodiment of a safety inspection method based on an industrial cleaning robot of the present invention. DETAILED DESCRIPTION

[0039] The following is further described in detail through specific implementation methods:

[0040] Embodiment 1:

[0041] Embodiment 1 is basically as attached Figure 1 As shown:

[0042] Industrial cleaning robots are automated devices designed for industrial environments for sweeping, vacuuming or other cleaning tasks. They are usually used in large indoor or outdoor places such as factory workshops, warehouses, and logistics centers. They can improve cleaning efficiency, reduce labor costs, and ensure a clean and safe working environment. Industrial cleaning robots are generally equipped with advanced navigation systems, such as laser radar (LIDAE), visual sensors, etc., to implement common SLAM (simultaneous localization and mapping) technology.

[0043] like Figure 3 As shown, the cleaning robot in this scheme includes a controller, a motion device (including a cleaning module and a mobile module), an automatic adjustment device and an alarm device. The top of the cleaning robot is also equipped with a visual acquisition device and a microphone array group; the visual acquisition device, the microphone array group, the automatic adjustment device, the alarm device, the motion device and the cloud server are all electrically connected to the controller. In this embodiment, the controller specifically adopts a remote controller with a built-in communication module for transmitting data with the cloud server, so that the cloud server can collect and process data signals detected by each sensor, and control the motion device and automatic adjustment device of the cleaning robot. The automatic adjustment device adopts a pan-tilt, which is installed on the top of the cleaning robot to drive the visual acquisition device to move, so as to collect images of different areas.

[0044] Specifically, the cleaning module is used to sweep, wash, vacuum or perform other cleaning-related tasks on the ground; the mobile module is used to control the movement and travel of the robot. The module can also cooperate with the visual acquisition device and be used for obstacle avoidance under the control of the controller; the visual acquisition device is used to collect environmental images during the operation process, and specifically a depth camera can be used to collect color images and depth information; the microphone array group is used to monitor the noise with warning or danger signals in the factory environment, and locate the specific location of the factory equipment components that generate noise, such as pipeline gas leakage. The information collected by sensors such as the visual acquisition device and the microphone array group can be used for the judgment of warning signals after processing and processing. If the above-mentioned cleaning robot detects a warning signal during the operation of the factory, it can directly alarm through the alarm device, or send an early warning or alarm signal to the user's mobile APP through the cloud server.

[0045] In actual production, industrial cleaning robots are equipped with a variety of cleaning modes according to the real environment, with the most common mode being patrol mode. Its main tasks include planning the best or most economical cleaning path based on the site map information and preset algorithms; using sensors to identify surrounding obstacles and adjusting the route to avoid these obstacles when necessary; sweeping, vacuuming or mopping the floor with built-in brushes, vacuuming devices or other special tools; automatically returning to the charging station to charge when the battery level is below the set threshold; and recording various data during the cleaning process and feeding it back to the management server.

[0046] In this scheme, a safety inspection method based on an industrial cleaning robot is used. Figure 1 As shown, the following steps are included:

[0047] Obtain a list of key components, which stores the location data of each key component; in this solution, the list of key components is entered by the management personnel to check the accuracy of the location of its equipment components, directory information (mainly including number, name and location description, etc.), in this embodiment, the location description is separated by a unified specific symbol. At the same time, the management personnel also need to enter the map information and patrol route of the area to be inspected, and ensure that the factory facilities and equipment to be inspected are within the planned inspection area.

[0048] The cleaning robot is controlled to patrol the area to be inspected, and during the patrol process, the visual acquisition device configured on the cleaning robot obtains the environmental image of the area to be inspected, and the microphone array group configured on the cleaning robot collects real-time audio data to analyze whether there are abnormal equipment parts. Obstacle information is obtained in real time during the patrol process, and the patrol route is adjusted in real time according to the obstacle information.

[0049] The inspection includes the first inspection and several regular inspections. It should be noted that the factory environment is generally complex. When building its three-dimensional map, it usually contains millions of 3D points and enhances each point through high-dimensional concept embedding, which requires a lot of memory. Therefore, the inspection task selects a reasonable inspection interval Tx according to the server hardware and computing power. Figure 2 The control of the cleaning robot to perform patrol sweeping in the area to be inspected includes:

[0050] Get the map information of the area to be inspected and the corresponding inspection route.

[0051] The cleaning robot is controlled to perform the first inspection in the area to be inspected according to the map information and the inspection route.

[0052] According to the environmental image of the area to be inspected, a 3D fusion feature map is constructed. In this embodiment, the data set that the visual acquisition device needs to collect includes the following information: 1. The real global posture of things and objects in the factory environment at a specific time (Global Poses[R|t]: Global_RT_Trajectory_GT); 2. The images taken in the factory and TUM (Technische PNG images compatible with the München RGB-D dataset.

[0053] The specific preprocessing steps of the dataset are mainly: 1. Convert to PNG format. Save the original RGB and depth images as PNG files. RGB images can be saved directly as standard color PNG files, while depth images require special encoding. The usual optional method is to normalize them to 16-bit grayscale PNG files in the range of 0-65535; 2. Timestamp synchronization. Mark each pair of RGB-D images with the correct timestamp for use in the SLAM process; 3. Generate a global posture file. The global posture of the camera at each moment (rotation matrix [R] and translation vector [t]) is obtained through the SLAM algorithm or other positioning methods and saved as a text file. Each line of the file represents the posture information of a point in time. An optional format is as follows:

[0054] timestamp tx ty tzqxqyqzqw

[0055] Where tx ty tz is the translation component and qxqyqzqw is the rotation expressed as a quaternion.

[0056] The above process can be completed in the ROS system. The above SLAM algorithm can be selected such as ORB-SLAM2, RTAB-Map, etc. to generate global pose estimation. The above dataset can be directly used to build a 3D fusion feature map.

[0057] Among the various optional solutions, you can use the OpenCLIP model for processing, calculate the corresponding CLIP features (CLIP, AudioCLIP, etc.) of text, images, and audio from the GradSLAMDataset, and apply the pixel alignment solution proposed in the ConceptFusion paper.

[0058] Finally, various CLIP features are fused into the 3D point cloud map to complete the construction of the 3D fusion feature map.

[0059] Therefore, the 3D fusion feature map of the factory can be used to perform multimodal queries. Image query can identify the concept of a 3D object in the 3D fusion feature map; text query is realized by combining the built-in 3D object space comparator of ConceptFusion with the large language model LLM, and LLM query can be performed on the specified 3D object, such as location information, spatial position comparison information with other 3D objects, or semantic information of other non-spatial concepts of the 3D object; the main application of audio query is to locate a 3D object in the 3D map, that is, to query the spatial position of its sound source.

[0060] Based on the 3D fusion feature map constructed during the last inspection, the cleaning robot is controlled to perform routine inspections in the area to be inspected.

[0061] The position data of each key component in the environmental image is compared with the position data of each key component in the key component list, and a comparison result is generated; according to the comparison result, each key component is analyzed to see whether there is abnormal displacement, and if so, a position abnormality alarm is generated. The patrol in this embodiment includes the first patrol and the regular patrol, and correspondingly, the position abnormality alarm also includes the first alarm and the second alarm. In the first patrol, if there is abnormal displacement of the key component after the buffer construction time Tw or the key component in the key component list is not detected, the first alarm is issued. The principle is that the cleaning robot cannot perform the patrol warning function during the first patrol. Only after the information of the entire factory or only the patrol route environment (GradSLAMDataset data set) is collected and the 3D fusion feature map of the entire factory (or only the patrol route environment) is built, the patrol warning function can be used normally during the next patrol.

[0062] When the 3D fusion feature map is a complete 3D fusion feature map of the area to be inspected, the location information and integrity of the equipment components will be checked once according to the key component list. If the location check algorithm fails, a secondary alarm will be generated. The main purpose of this approach is to reduce the probability of false positives and warnings caused by the failure to build the 3D object model of the equipment components to be inspected within Tw time due to server hardware performance, network latency or other problems.

[0063] The following table is an example of an alternative key parts list:

[0064] Table 1 Model parameter configuration table

[0065] serial number name Position Description 10001 Bolt A On top of device B; on the surface of device C; 1 meter above device B 10002 Bolt M On top of device J; on the surface of device Q; 2 meters below device X … … …

[0066] Firstly, according to the real-time SLAM posture of the cleaning robot, image query is used in the incomplete 3D fusion feature map to determine whether the key components in the key component list appear in the 3D object model in the image at the viewing angle where the industrial cleaning robot is located.

[0067] If not, the cleaning robot continues to patrol and move forward along the predetermined cleaning route.

[0068] If yes, and the CoceptFusion image query returns two possible 3D object names, Bolt A and Bolt M.

[0069] Furthermore, "name" + "whether" is added before each position description clause for conceptfusion text query, such as: "Is bolt A above device B?", "Is bolt A located on the surface of device C?", "Is bolt A 1 meter above device B?"

[0070] For bolts M with the same appearance style, the spatial query module needs to be added for comparison. That is, when judging bolt A, in addition to the above three judgments, three more judgments need to be added: "Is bolt M on device J?", "Is bolt M on the surface of device Q?", "Is bolt M 2 meters below device X?".

[0071] As long as any position judgment in a position description does not meet the requirements, the key component corresponding to the position description will not meet the safety requirements.

[0072] If a 3D object can be found under the ConceptFusion image query, but none of the position descriptions in the key components of the result meet the safety requirements, then a first alarm or a second alarm will be triggered.

[0073] In addition to the alarm function, real-time audio data will also be collected to help factory technicians locate damaged or abnormally operating equipment parts, that is, real-time audio data will be collected during the cleaning process to analyze whether there are abnormal equipment parts. It should be noted that the microphone group installed on the above cleaning robot is several arrays, so the audio data collected should also be several channels. The above audio analysis specifically includes the following steps:

[0074] During the patrol process, real-time audio data is collected to analyze whether there is abnormal noise; if so, the direction of the sound source corresponding to the abnormal noise is analyzed, if not, the patrol continues. In this embodiment, an array microphone group is used to collect real-time audio data. The collected real-time audio data is compared with each preset abnormal noise in the pre-stored abnormal noise list. If there is a matching preset abnormal noise, it is determined that there is abnormal noise in the area to be inspected. The abnormal noise list stores a number of preset abnormal noises and corresponding noise source types.

[0075] The visual acquisition device on the cleaning robot is controlled to turn to the sound source direction corresponding to the abnormal noise, and collect the environmental image of the sound source direction; in this embodiment, the MUSIC subspace algorithm is specifically used to calculate the sound source direction corresponding to the abnormal noise from the multi-channel audio byte data collected by the array microphone group. Then, the visual acquisition device is adjusted to align with the sound source direction by controlling the automatic adjustment device.

[0076] According to the real-time audio data and the environmental image of the sound source, the abnormal equipment components that emit abnormal noise are analyzed, and an abnormal operation alarm is generated. In this embodiment, according to the abnormal noise list, the type of noise source that emits abnormal noise is analyzed, and according to the environmental image of the sound source, it is confirmed whether there is an equipment component of the noise source type in the environmental image of the sound source. If so, the corresponding equipment component is determined to be an abnormal equipment component, and the position data of the abnormal equipment component is analyzed in combination with the current position of the cleaning robot, and an abnormal operation alarm is generated after integrating the image, name, and position data of the abnormal equipment; if not, the type of noise source, the image of the sound source orientation, and the position data are integrated to generate an abnormal operation alarm.

[0077] Therefore, by adopting this solution, a visual acquisition device and a microphone array group can be configured on the cleaning robot to obtain environmental images and real-time audio data while patrolling, thereby locating abnormal components that are displaced, damaged, or operating abnormally.

[0078] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A safety inspection method based on an industrial cleaning robot, characterized in that: The following steps are involved: Obtaining a key component list, wherein the key component list stores location data of each key component; Control the cleaning robot to patrol the area to be inspected, and during the patrol process, obtain the environmental image of the area to be inspected through the visual acquisition device configured on the cleaning robot; Compare the position data of each key component in the environment image with the position data of each key component in the key component list, and generate a comparison result; According to the comparison results, each key component is analyzed to see whether there is abnormal displacement. If so, a position abnormality alarm is generated.

2. The safety inspection method based on the industrial cleaning robot according to claim 1 is characterized in that: The patrol and sweeping includes the first patrol and sweeping; Control the cleaning robot to patrol the area to be inspected, including: Obtain map information of the area to be inspected and the corresponding inspection route; The cleaning robot is controlled to perform the first inspection in the area to be inspected according to the map information and the inspection route.

3. The safety inspection method based on the industrial cleaning robot according to claim 2 is characterized in that: Also includes: Construct a 3D fusion feature map based on the environmental image of the area to be inspected.

4. The safety inspection method based on the industrial cleaning robot according to claim 3 is characterized in that: The patrol sweep also includes several conventional patrol sweeps; Controlling the cleaning robot to patrol the area to be inspected also includes: Based on the 3D fusion feature map constructed during the last inspection, the cleaning robot is controlled to perform routine inspections in the area to be inspected.

5. The safety inspection method based on the industrial cleaning robot according to claim 1 is characterized in that: Obstacle information is obtained in real time during the patrol and sweeping process, and the patrol and sweeping route is adjusted in real time according to the obstacle information.

6. The safety inspection method based on the industrial cleaning robot according to claim 1 is characterized in that: Also includes: Collect real-time audio data during the inspection process to analyze whether there are any abnormal equipment components.

7. The safety inspection method based on the industrial cleaning robot according to claim 6 is characterized in that: Collect real-time audio data during the inspection process to analyze whether there are abnormal equipment components, including: Collect real-time audio data during the patrol process to analyze whether there is abnormal noise; if so, analyze the direction of the sound source corresponding to the abnormal noise; Controlling the visual collection device provided on the cleaning robot to turn to the direction of the sound source corresponding to the abnormal noise, and collecting the environmental image of the direction of the sound source; Based on real-time audio data and environmental images of the sound source location, analyze abnormal equipment components that emit abnormal noise and generate abnormal operation alarms.

8. The safety inspection method based on the industrial cleaning robot according to claim 7 is characterized in that: The cleaning robot is also provided with a microphone array group; real-time audio data is collected through the microphone array group configured on the cleaning robot.

Images