Intelligent inspection method of wheeled robot, control device, system and storage medium

Abstract

The invention relates to a visual inspection technology, and discloses an intelligent inspection method of a wheeled robot, a control device, a system and a storage medium, and the method comprises the steps: carrying out the image splicing of monitoring images collected by all monitoring devices, and generating an overall monitoring image; analyzing the overall monitoring image, and extracting a workshop map of the production workshop; based on the workshop map, controlling the wheeled robot to perform global scanning on the production workshop to generate a 3D global image; analyzing a missing region part of the overall monitoring image compared with the 3D global image, and discriminating each monitoring dead angle region in combination with a latest workshop map; planning an inspection path of the wheeled robot based on the charging position of the wheeled robot and the position of each monitoring dead angle area; and controlling the wheeled robot to intelligently inspect the production workshop based on the inspection path. The invention aims to realize efficient, intelligent and comprehensive monitoring of the production workshop.

Description

Technical Field

[0001] The present application relates to the field of visual inspection technology, and in particular to an intelligent inspection method, a control device, a control system and a computer-readable storage medium for a wheeled robot. Background Art

[0002] In modern industrial production, efficient, stable and safe operation of production workshops is the core goal pursued by enterprises. The production workshops are equipment-intensive and the process is complex. Various potential risk factors such as equipment failure, illegal operation, and safety hazards always threaten the normal production. Comprehensive, real-time and accurate monitoring of the production workshop can detect and deal with these problems in time, avoid production accidents, reduce equipment damage and downtime, improve product quality, and reduce production costs.

[0003] At present, in order to monitor the production workshop, enterprises usually install multiple monitoring devices in the workshop, such as cameras, sensors, etc. These monitoring devices can collect images, temperature, humidity, pressure and other data in the workshop, providing a certain basis for the management and maintenance of the workshop.

[0004] However, due to the extremely complex environment of the production workshop, there are many large equipment, cargo stacks, and narrow passages. Existing monitoring equipment is usually installed in a fixed position, and its monitoring field of view is limited, making it difficult to cover every corner of the workshop (for example, behind some large mechanical equipment, the narrow gaps between equipment, and the bottom of high-rise shelves are often blind spots for monitoring. These blind spots may hide safety hazards such as equipment failure, material leakage, and illegal operation by personnel, which cannot be discovered and handled in time, posing great risks to production safety), and cannot meet the needs of modern industrial production for efficient, intelligent, and comprehensive monitoring.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide an intelligent inspection method, control device, control system and computer-readable storage medium for a wheeled robot, aiming to achieve efficient, intelligent and comprehensive monitoring of the production workshop.

[0007] To achieve the above objectives, the present application provides an intelligent inspection method for a wheeled robot, comprising the following steps: Based on monitoring devices distributed in multiple locations in the production workshop, monitoring images of the production workshop are collected, and the monitoring images collected by each monitoring device are spliced ​​to generate an overall monitoring image; Analyze the overall monitoring image, extract the workshop map of the production workshop, and mark the obstacles and passages in the workshop map; Based on the workshop map, the wheeled robot is controlled to perform a global scan of the production workshop to generate a 3D global image; wherein, during the global scan, the wheeled robot supplements and updates the workshop map according to the detection data collected by the detection sensor integrated in the body; the detection sensor includes at least an image sensor, a radar sensor and a collision detection sensor; Analyze the missing areas of the overall monitoring image compared with the 3D global image, and identify the missing areas in combination with the latest workshop map to obtain the monitoring blind spots in the production workshop; Based on the charging position of the wheeled robot and the location of each monitoring blind spot area, a multi-objective optimization algorithm is used to plan the inspection path of the wheeled robot; the optimization goals of the multi-objective optimization algorithm planning include the shortest inspection time for a single round and the minimum energy consumption for a single round of inspection; Based on the inspection path, the wheeled robot is controlled to perform intelligent inspection of the production workshop; wherein, based on the detection data collected in real time by the wheeled robot during the inspection process, the wheeled robot is controlled to perform intelligent obstacle avoidance.

[0008] To achieve the above object, the present application also provides a control device, comprising: The monitoring module is used to collect monitoring images of the production workshop based on monitoring devices distributed in multiple different locations in the production workshop, and to stitch the monitoring images collected by each monitoring device to generate an overall monitoring image; An extraction module is used to analyze the overall monitoring image, extract the workshop map of the production workshop, and mark the obstacles and passages in the workshop map; A scanning module is used to control the wheeled robot to perform a global scan of the production workshop based on the workshop map to generate a 3D global image; wherein, during the global scanning process, the wheeled robot supplements and updates the workshop map according to the detection data collected by the detection sensor integrated in the body; the detection sensor includes at least an image sensor, a radar sensor and a collision detection sensor; The analysis module is used to analyze the missing area of ​​the overall monitoring image compared with the 3D global image, and identify the missing area in combination with the latest workshop map to obtain the various monitoring blind spots in the production workshop; The planning module is used to plan the inspection path of the wheeled robot based on the charging position of the wheeled robot and the location of each monitoring blind spot area using a multi-objective optimization algorithm; the optimization goals of the multi-objective optimization algorithm planning include the shortest inspection time for a single round and the minimum energy consumption for a single round of inspection; The inspection module is used to control the wheeled robot to perform intelligent inspection of the production workshop based on the inspection path; wherein, based on the detection data collected in real time by the wheeled robot during the inspection process, the wheeled robot is controlled to perform intelligent obstacle avoidance.

[0009] To achieve the above-mentioned purpose, the present application also provides a control system, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, the steps of the intelligent inspection method of the wheeled robot as described above are implemented.

[0010] To achieve the above objectives, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the intelligent inspection method of the wheeled robot as described above are implemented.

[0011] The intelligent inspection method, control device, control system and computer-readable storage medium of the wheeled robot provided in the present application collect and stitch images based on multi-position monitoring equipment to generate an overall monitoring image, laying the foundation for subsequent analysis, and analyzing the image to extract the workshop map and mark obstacles and channels to provide guidance for the robot's actions. The wheeled robot generates a 3D global image by global scanning, and uses a variety of sensors to update the map in real time to accurately find monitoring blind spots. Then a multi-objective optimization algorithm is used to plan the inspection path, taking into account inspection time and energy consumption, improving inspection efficiency and reducing costs. In this way, the wheeled robot is combined with multi-position monitoring equipment to achieve efficient, intelligent and comprehensive monitoring of the production workshop. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram of the steps of an intelligent inspection method for a wheeled robot in one embodiment of the present application; Figure 2 This is a schematic diagram of a control device in an embodiment of the present application; Figure 3 Schematic diagram of the internal structure of a control system according to an embodiment of the present application.

[0013] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0014] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0015] In addition, if the descriptions of "first", "second", etc. are involved in this application, they are only used for descriptive purposes (such as for distinguishing the same or similar features), and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0016] Reference Figure 1 In one embodiment, the intelligent inspection method of the wheeled robot includes: Step S10: Based on monitoring devices distributed at multiple different locations in the production workshop, monitoring images of the production workshop are collected, and the monitoring images collected by each monitoring device are spliced ​​to generate an overall monitoring image; Step S20, analyzing the overall monitoring image, extracting the workshop map of the production workshop, and marking obstacles and passages in the workshop map; Step S30: Based on the workshop map, control the wheeled robot to perform a global scan of the production workshop to generate a 3D global image; wherein, during the global scan, the wheeled robot supplements and updates the workshop map according to the detection data collected by the detection sensor integrated in the body; the detection sensor includes at least an image sensor, a radar sensor and a collision detection sensor; Step S40: Analyze the missing area of ​​the overall monitoring image compared with the 3D global image, and identify the missing area in combination with the latest workshop map to obtain each monitoring blind spot area in the production workshop; Step S50, based on the charging position of the wheeled robot and the location of each monitoring blind spot area, a multi-objective optimization algorithm is used to plan the inspection path of the wheeled robot; wherein the optimization objectives of the multi-objective optimization algorithm planning include minimizing the single-round inspection time and minimizing the single-round inspection energy consumption; Step S60: Based on the inspection path, the wheeled robot is controlled to perform intelligent inspection of the production workshop; wherein, based on the detection data collected in real time by the wheeled robot during the inspection process, the wheeled robot is controlled to perform intelligent obstacle avoidance.

[0017] In this embodiment, the execution terminal of the embodiment may be a control system, or may be other equipment or devices (such as a control device) that controls the wheeled robot.

[0018] As described in step S10, in order to fully cover the production workshop, the monitoring equipment will be reasonably distributed according to factors such as the layout of the workshop, equipment distribution, and process flow. For example, in a large production workshop, monitoring equipment is usually installed in the four corners of the workshop, on both sides of the passage, above key equipment, and in areas prone to danger, to ensure that there are no large monitoring blind spots in the workshop.

[0019] Different types of monitoring equipment can be selected, such as high-definition cameras, infrared cameras, etc. High-definition cameras can provide clear image details and are suitable for areas with sufficient light; infrared cameras work at night or in low-light environments and can be used to monitor the operation of equipment and personnel activities in the workshop.

[0020] Optionally, the monitoring device will collect images at a set frequency. For some areas with high real-time requirements, such as key processes on a production line, the collection frequency may be set higher, such as collecting 1 to 5 frames of images per second; while for some relatively stable areas, the collection frequency can be appropriately reduced, such as collecting 1 to 2 frames of images per minute.

[0021] Optionally, before image stitching, the collected surveillance images are preprocessed, including removing noise from the images, adjusting the brightness and contrast of the images, etc., to improve the image quality and lay a good foundation for subsequent stitching work.

[0022] Optionally, feature extraction algorithms are used to extract feature points from the preprocessed images. These feature points will be used in subsequent image matching and stitching processes. By comparing feature points in different surveillance images, the corresponding relationship between them can be found. Among them, a matching algorithm based on feature descriptors can be used to quickly and accurately find matching feature point pairs.

[0023] Optionally, based on the matched feature point pairs, a transformation matrix between adjacent images is calculated. This transformation matrix describes how to transform one image into the coordinate system of another image so that they can be accurately stitched together.

[0024] Apply the calculated transformation matrix to adjacent images, perform geometric transformation on the images, and then fuse them. During the fusion process, the overlapping areas between images need to be processed to avoid stitching traces. Methods such as weighted averaging and multi-resolution fusion can be used to achieve smooth image fusion.

[0025] After image stitching and fusion, the individual monitoring images are combined into a complete overall monitoring image. This image can intuitively show the overall picture of the production workshop and provide an important basis for subsequent workshop map extraction and analysis.

[0026] As described in step S20, before conducting an in-depth analysis of the overall surveillance image, a preprocessing operation may be performed to improve the image quality and lay a foundation for subsequent extraction of accurate information.

[0027] Optionally, the color overall monitoring image is converted into a grayscale image. This can reduce the amount of data and the computational complexity while retaining the main feature information of the image for easy subsequent processing.

[0028] Optionally, use Gaussian filtering, median filtering and other methods to remove noise interference in the image. The noise may come from the monitoring device itself, the transmission process, etc. The filtering operation can make the image smoother and enhance the clarity of the image.

[0029] Optionally, edge detection algorithms can be used to highlight the edge information of objects in the image. This helps to more clearly identify the boundaries of different objects in the workshop and provide more accurate data for subsequent feature extraction.

[0030] Optionally, extract key features from the preprocessed images and identify different elements in the workshop. Computer vision techniques, such as feature point-based methods (SIFT, SURF, etc.) or deep learning-based object detection algorithms (such as YOLO, Faster R-CNN, etc.), are used to extract features of various objects in the workshop (such as equipment, shelves, walls, etc.). These features can include the shape, size, color, texture, etc. of the object.

[0031] Based on the extracted features, objects in the workshop are classified and identified. The pre-trained classification model is used to determine the type of each object, such as production equipment, storage shelves, channel signs, etc.

[0032] To establish a suitable coordinate system for the workshop map, a fixed point in the workshop (such as a corner) can be used as the origin to determine the direction and unit length of the X and Y axes. This can convert the position of objects in the image into actual coordinate positions, which is convenient for subsequent navigation and analysis.

[0033] According to the coordinate position and classification information of the object, the corresponding elements are drawn on the map. For example, different graphics and colors are used to represent different types of equipment, shelves and walls, etc., to form an intuitive workshop map.

[0034] Optionally, the identified inaccessible objects (such as large equipment, cargo piles, etc.) are marked as obstacles. These obstacles can be highlighted on the map with a specific color (such as red) or a symbol (such as a cross) so that the wheeled robot can avoid them during the inspection process.

[0035] Optionally, the location and range of the passages are determined based on the actual traffic conditions in the workshop. The passages are marked on the map with another color (such as green) or lines (such as dotted lines) to provide clear guidance for the wheeled robot to plan the inspection path.

[0036] As described in step S30, the workshop map extracted and marked in step S20 is imported into the control system of the wheeled robot, so that the robot can understand the distribution of known obstacles and passages in the workshop.

[0037] Establish the correspondence between the robot position and the map coordinates, and determine the robot's position in the map in real time through the positioning system (such as visual positioning, inertial navigation, etc.).

[0038] Based on this map, the robot will initially plan the path for global scanning to ensure that it can cover all areas of the production workshop. Path planning will take into account factors such as the layout of the workshop, obstacle distribution, and scanning efficiency, and try to avoid repeated scanning and unnecessary detours.

[0039] Optionally, the robot can use landmark features in the workshop map, such as specific equipment and corners, combined with the robot's own positioning system (such as visual positioning, inertial navigation, etc.) to determine the robot's initial position and orientation in the workshop map. This is the basis for subsequent accurate scanning and map updates.

[0040] During the global scanning process, image sensors (such as high-definition cameras) are used to continuously capture images of the environment around the robot. These images contain information such as the appearance, color, and texture of objects in the workshop. The camera usually shoots from multiple angles and directions to obtain more comprehensive visual data. For example, during the movement of the robot, the camera can rotate horizontally and vertically to ensure that the surrounding 360° environment can be captured.

[0041] Optionally, a radar sensor (such as a lidar) can be used to emit a laser beam and measure the time of reflected light to calculate the distance between the robot and surrounding objects. By continuously rotating the radar antenna, distance information in all directions around the robot can be obtained in real time to construct three-dimensional point cloud data of the surrounding environment.

[0042] The image data collected by the image sensor and the point cloud data obtained by the radar sensor are fused. Using the data fusion algorithm, the two different types of data are matched and aligned in time and space, so that the visual information in the image is combined with the distance information in the point cloud data. For example, feature matching methods can be used to associate feature points in the image with corresponding points in the point cloud data.

[0043] Based on the fused data, a 3D global image of the production workshop is constructed using a 3D reconstruction algorithm. The image presents the overall picture of the workshop in the form of a 3D model, including the true shape and positional relationship of various objects such as equipment, shelves, and walls. By integrating and processing data collected at different positions and angles, the accuracy and details of the 3D model are continuously optimized.

[0044] At the same time, as the robot moves, the collision detection sensor continuously detects the surrounding environment. When the sensor detects that there are no objects in the areas marked as obstacles in the original workshop map, or detects that there are passable spaces in areas that were originally considered impassable, these areas are marked as abnormal areas.

[0045] The abnormal area is then further detected and confirmed by collecting sensor data multiple times to determine whether the area is continuously passable, eliminating misjudgments caused by sensor errors or temporary object movement.

[0046] Based on the detection results of the abnormal area, the possible existence of previously unrepresented passages is identified. The spatial size, connectivity and other factors of the area are analyzed to determine whether it meets the characteristics of a passage. The identified passage can be verified by having the robot try to pass through the area while observing the sensor data and the movement of the robot. If the robot can pass through the area smoothly and the sensor does not detect the risk of collision, the area is confirmed to be a previously unrepresented passage and updated in the workshop map.

[0047] According to the results of the collision detection, the original workshop map is updated. If the obstacles marked in the original map are found to be non-existent, they are removed from the map; if a channel that was not originally represented is identified, the corresponding channel information is added to the map, and the navigation data of the map is updated to provide accurate map support for subsequent inspections.

[0048] When the robot completes scanning of the entire production workshop according to the planned path and collects enough data to generate an accurate 3D global image and update the workshop map, the scanning process is determined to be completed.

[0049] As described in step S40, the coordinate systems of the 3D global image, the overall monitoring image and the latest workshop map are unified. They are all mapped to the same coordinate reference system, for example, based on the coordinate system established by the workshop map, to ensure that the location information of different data sources can accurately correspond during the comparison and analysis process.

[0050] Feature points are extracted from the 3D global image and the overall monitoring image respectively. These features are rotation, scale and illumination invariant and can stably represent the features of objects in different images. Using the feature matching algorithm, the feature points of the 3D global image and the overall monitoring image are matched to find the corresponding relationship between them.

[0051] According to the feature matching results, the overall monitoring image is registered using a suitable transformation model (such as affine transformation, perspective transformation) to align it with the 3D global image in space. The registered image is convenient for pixel-by-pixel or region-by-region comparative analysis.

[0052] Based on the image comparison results and the correlation analysis with the workshop map, the blind spots in the production workshop are determined. These areas have obvious information in the 3D global image, but are missing or incomplete in the overall monitoring image.

[0053] Optionally, the registered 3D global image and the overall monitoring image are compared pixel by pixel to calculate the difference in pixel values. If there is pixel information in a certain area in the 3D global image, but the pixel value of the corresponding area in the overall monitoring image is zero or close to zero, the area is marked as a possible missing area.

[0054] Optionally, the image is divided into different regions, and the features and statistical information (such as color, texture, shape, etc.) of each region are compared. If a region has obvious features in the 3D global image, but the features are not obvious or missing in the overall monitoring image, the region is considered as a possible missing region.

[0055] Map the location information of the detected missing areas onto the latest workshop map to determine the specific locations of these areas in the actual space of the workshop.

[0056] Optionally, check the distribution of obstacles around the missing area in the workshop map. If the missing area is blocked by obstacles such as large equipment, shelves, walls, etc., so that the surveillance camera cannot cover the area, then the area is likely to be a monitoring blind spot.

[0057] Optionally, consider the relationship between the missing area and the workshop passage and the accessibility of personnel and equipment. If the missing area is located near an important passage or an area where personnel and equipment frequently move, but cannot be covered by monitoring, then the area should be focused on as a monitoring blind spot.

[0058] Optionally, the importance of the functional area where the missing area is located can be determined in combination with the functional zoning of the workshop. For example, if the missing area is located in an area involving key production processes, storage of dangerous goods, or densely populated areas, then the area is more likely to be a monitoring blind spot that needs to be resolved.

[0059] According to the above-mentioned identification results of missing areas, comprehensive consideration is given to factors such as obstacle obstruction, channel accessibility, and importance of functional areas to determine the various monitoring blind spots in the production workshop, including determining the center point coordinates or boundary coordinates of each monitoring blind spot.

[0060] Optionally, each blind spot area is precisely demarcated to clarify its size. The boundaries of the blind spot area can be represented by drawing polygons or other geometric figures on the workshop map.

[0061] As described in step S50, the precise coordinates of the wheeled robot's charging position in the workshop map are determined. The charging position is usually fixed and has a specific mark in the workshop layout. And the position information of each monitoring blind spot area in the workshop map is collected, and this position information has been determined in step S40. The position information of the charging position and the monitoring blind spot area is unified and integrated into a data structure to facilitate the processing of subsequent algorithms.

[0062] Optionally, consider the distribution of obstacles in the workshop map. Obstacle information can be obtained from the latest workshop map, including inaccessible areas such as equipment and shelves.

[0063] Optionally, analyze the aisle conditions in the workshop to understand factors such as aisle width and connectivity, which will affect the robot's movement speed and path selection.

[0064] Choose a suitable multi-objective optimization algorithm, such as genetic algorithm, particle swarm algorithm, etc. These algorithms can find a balanced solution between multiple optimization objectives. Take genetic algorithm as an example, it simulates the biological evolution process and continuously iterates to search for the optimal solution.

[0065] Optimize the shortest inspection time per round: Define a time evaluation function that takes into account factors such as the robot's moving speed and turning time in different sections of the road. The robot's moving speed in the channel may be affected by the channel width, road conditions, etc., and the turning time is related to the turning angle and the robot's steering performance. For example, suppose the inspection path is P={p1,p2,⋯,p n}, where p i represents the i-th point on the path, t ij From point p i Move to point p j time, then the single round inspection time .

[0066] The goal of optimizing the shortest inspection time per round is to define an energy consumption evaluation function, taking into account the robot's movement energy consumption, turning energy consumption, etc. The robot's movement energy consumption is related to the movement distance, load weight, etc., while the turning energy consumption is related to the turning angle and the robot's power system. For example, let e ij From point p i Move to point p j The energy consumption of a single round of inspection is .

[0067] Constraint setting: The path must avoid obstacles in the workshop map to ensure the safe movement of the robot. This constraint can be achieved by setting the path points not to fall in the area where the obstacles are located. The robot's movement trajectory must conform to the aisle layout of the workshop and cannot pass through inaccessible areas.

[0068] In the path planning process, a set of initial inspection paths is randomly generated as a population based on the genetic algorithm. Each path is represented as a visit sequence of a monitoring blind spot area, and the starting point and end point of the path are both charging locations. The feasibility of each initial path is checked to ensure that it meets the constraints. If there are obstacles or inaccessible areas in the path, the path is adjusted or regenerated.

[0069] For each path in the population, calculate its single-round inspection time T and single-round inspection energy consumption E. According to the objective function, these two target values ​​are combined into a fitness value. The weighted summation method can be used to set F=w1T+w2E, where w1 and w2 are weight coefficients, and w1+w2=1. The weight can be adjusted according to actual needs to balance the importance of time and energy consumption. Optionally, set w1 greater than w2.

[0070] The paths in the population are selected according to the fitness value F, and the paths with higher fitness (such as fitness greater than the preset threshold) are selected as the parent generation to generate the next generation path. The preset threshold can be the average fitness of the current round, that is, the paths with fitness above the average value are selected as the parent generation.

[0071] Perform a crossover operation on the selected parent path to generate a new child path. The crossover operation simulates the gene exchange process in biological inheritance, exchanges some fragments of the parent path, and generates child paths with different access orders. Perform a mutation operation on the child path to randomly change some access orders in the path, increase the diversity of the population, and avoid the algorithm from falling into a local optimal solution.

[0072] Repeat the fitness evaluation, selection, crossover and mutation operations until the termination condition is met. The termination condition can be that the fitness value no longer increases significantly, that is, the average fitness difference between the previous and next rounds is less than the preset difference.

[0073] After the iteration, the path with the best fitness value is selected from the final population as the inspection path of the wheeled robot. This path achieves a good balance between the inspection time and energy consumption of a single round of inspection.

[0074] The planned inspection path is output to the control system of the wheeled robot. The robot conducts inspections according to the path, and monitors its own power and position in real time during the inspection process to ensure that the inspection task can be completed safely and efficiently.

[0075] As described in step S60, the wheeled robot reads the planned inspection path data, which includes a series of coordinate points or path nodes, and clarifies the movement trajectory of the robot in the production workshop.

[0076] When the inspection is started, the various sensors and actuators of the wheeled robot, including the laser radar, camera, motor, etc., are started to perform self-inspection to ensure the normal operation of the equipment. The robot is positioned at the charging position (i.e. the starting point of the inspection path), and the initial position information of the robot is obtained through the positioning system, and matched and calibrated with the starting point of the inspection path.

[0077] The robot's motion control module calculates the direction and distance from the current position to the next path node based on the inspection path. Then the robot's wheels are controlled by the motor drive system to move toward the target node. During the movement, the robot's posture and direction are constantly adjusted to ensure that it moves along the planned path.

[0078] Various sensors installed on the robot are used to collect detection data from the production workshop in real time. These sensors include but are not limited to: LiDAR: It is used to obtain three-dimensional point cloud data of the robot's surrounding environment in real time. By analyzing the point cloud data, the position, shape and distance of obstacles can be detected.

[0079] Camera: Collects images and video information of the workshop to identify equipment status, personnel activities, etc.

[0080] Temperature sensor: monitors temperature changes at different locations in the workshop and promptly detects possible equipment overheating problems.

[0081] Gas sensor: detects the concentration of harmful gases in the workshop to ensure the safety of the production environment.

[0082] It should be noted that although attempts to avoid obstacles have been made during the path planning stage, obstacle avoidance operations are still required after the inspection begins, because the environment of the production workshop may change and new obstacles may appear during the period between the completion of path planning and the start of inspection. For example, workers temporarily move large equipment and place it on the originally planned path, or new goods are piled in the passage. These new obstacles were not taken into account during path planning.

[0083] Moreover, there are a large number of moving objects in the production workshop, such as other robots, forklifts, workers, etc. Path planning is usually based on static environment information, and it is impossible to accurately predict the real-time position and movement trajectory of these moving objects. When the inspection robot is performing a task, moving objects may enter its planned path, so real-time obstacle avoidance is required.

[0084] Alternatively, due to some special reasons, such as equipment maintenance, emergency construction, etc., the layout of some areas in the workshop may be temporarily adjusted, and the originally unobstructed paths may be temporarily blocked or temporary obstacles may be set up. This requires the inspection robot to have real-time obstacle avoidance capabilities during the inspection process.

[0085] Optionally, the point cloud data collected by the lidar is pre-processed by filtering, segmenting, etc. to remove noise and irrelevant information. Then, the point cloud data is divided into different object clusters through a clustering algorithm to identify obstacles.

[0086] Optionally, the type and location of obstacles can be further confirmed by using target detection and recognition algorithms in combination with the image information collected by the camera, for example, different types of obstacles such as equipment, shelves or personnel can be distinguished.

[0087] Optionally, when an obstacle suddenly appears in front of the robot, the threat level of the obstacle to the robot inspection task is evaluated based on factors such as the distance between the obstacle and the robot, the relative speed, etc. For example, if the obstacle is close to the robot and is approaching quickly, the threat level is high; if the obstacle is stationary and far away, the threat level is low.

[0088] Optionally, select an appropriate obstacle avoidance strategy based on the threat level of the obstacle and the characteristics of the workshop environment. Common obstacle avoidance strategies include: Bypass strategy: When the obstacle is small and there is enough space around it, the robot can bypass the obstacle and continue along the inspection path. It can avoid the obstacle area by replanning the local path; Waiting strategy: If the obstacle is moving (such as a person or other equipment) and it is expected that the obstacle will leave the robot's path in a short time, the robot can pause and wait for the obstacle to leave before continuing the inspection.

[0089] According to the selected obstacle avoidance strategy, the robot's motion control module adjusts the robot's motion parameters, such as speed and direction, to achieve obstacle avoidance. During the obstacle avoidance process, it continuously monitors the state of obstacles and changes in the surrounding environment, and adjusts the obstacle avoidance strategy in a timely manner to ensure the robot's safety.

[0090] Optionally, the running status of the wheeled robot is monitored in real time, including parameters such as power, speed, and posture. When the power is lower than the set threshold, the robot automatically adjusts the inspection task and returns to the charging position for charging first.

[0091] The detection data and operation status information collected by the robot during the inspection process are uploaded to the host computer or workshop management system in real time. These data can be used for subsequent analysis and processing, such as equipment status evaluation, production environment monitoring, etc. At the same time, the robot receives instructions sent by the host computer and adjusts the inspection tasks or performs other operations according to the instructions.

[0092] When the robot reaches the end of the inspection path (i.e., the charging position), it determines whether the inspection task is completed. If all path nodes have been visited and there are no unhandled exceptions, the inspection task is considered to be completed.

[0093] All data collected during the inspection process are stored, including sensor data, operation logs, etc., for subsequent query and analysis. An inspection report is generated to summarize the results of the inspection, including problems found, obstacles, and other information, to provide a reference for workshop management and maintenance.

[0094] In one embodiment, images are collected and spliced ​​based on multi-position monitoring equipment to generate an overall monitoring image, laying the foundation for subsequent analysis, and analyzing the image to extract the workshop map and mark obstacles and channels to provide guidance for the robot's actions. The wheeled robot generates a 3D global image by global scanning, and uses a variety of sensors to update the map in real time to accurately find the monitoring blind spots. Then a multi-objective optimization algorithm is used to plan the inspection path, taking into account the inspection time and energy consumption, improving the inspection efficiency and reducing costs. In this way, the wheeled robot is combined with multi-position monitoring equipment to achieve efficient, intelligent and comprehensive monitoring of the production workshop.

[0095] During the inspection, the robot uses the real-time collected data to intelligently avoid obstacles and ensure its own safety. In addition, by integrating multiple data, comprehensive monitoring can be achieved, potential risks can be discovered in a timely manner, and labor costs can be reduced, avoiding errors in manual inspections. In addition, through in-depth data analysis, it provides strong support for production management decisions and helps production workshops operate efficiently, stably and safely.

[0096] In one embodiment, based on the above embodiment, after the step of analyzing the missing area of ​​the overall monitoring image compared with the 3D global image and identifying the missing area in combination with the latest workshop map to obtain each monitoring blind spot area in the production workshop, the following steps are further included: Perform feature recognition on the 3D global image and use the feature recognition results to generate semantic information of devices and objects in each monitoring blind spot area; When planning the inspection route, the corresponding inspection priority is determined based on the semantic information of the equipment and objects in the monitoring blind spot area.

[0097] In this embodiment, computer vision technology and deep learning algorithms are used to extract features from the 3D global image. For example, for devices and objects in the image, visual features such as shape, color, texture, and edge can be extracted. If it is a mechanical part, its specific shape contour and surface texture are important features; for devices with color markings, color can also provide key information.

[0098] It is also possible to combine 3D information to extract the three-dimensional size, spatial position, posture and other features of the object. For example, by analyzing the coordinates and angles of the object in 3D space, its placement status can be determined.

[0099] The extracted features are matched with the pre-built feature library. The feature library contains standard feature templates for various common equipment and objects in the production workshop. By calculating the similarity between the features, it is determined which category the object in the image belongs to.

[0100] Classification algorithms, such as support vector machines and convolutional neural networks, are used to accurately classify objects. For example, equipment can be classified into categories such as production equipment, transportation equipment, and storage equipment, and objects can be classified into categories such as raw materials, finished products, and waste materials.

[0101] Based on the results of feature recognition, the classification information of each device and object is associated with their specific location in the monitoring blind spot area. For example, determine the specific coordinate position of a production equipment in the monitoring blind spot area.

[0102] Optionally, add additional attribute information to each equipment and object in combination with the business information of the workshop. For example, for production equipment, add its production process, operating parameters, maintenance records and other information; for raw materials and finished products, add their names, specifications, uses and other information.

[0103] Using the associated information, a detailed semantic description is generated for each device and object in the blind spot area of ​​the monitoring. For example, "an injection molding machine located at a certain coordinate in the blind spot area of ​​the monitoring is currently producing plastic shells."

[0104] According to the semantic information of equipment and objects, their importance in the production workshop is evaluated. Critical production equipment that directly affects the production process is given a higher priority. For example, a large machining center that undertakes the core production process may cause the entire production line to stop working due to its failure, so it should be inspected first.

[0105] Areas storing important raw materials or finished products should also be given higher priority. For example, the safety and inventory of warehouses storing high-value raw materials need special attention.

[0106] Optionally, analyze the possible risks of equipment and objects. For equipment that is prone to failure and has a high degree of aging, increase its inspection priority. For example, a motor with a long service life has a relatively high probability of failure and needs more frequent inspections.

[0107] Consider the impact of environmental factors on equipment and objects. If a certain blind spot in a surveillance area has harsh environmental conditions such as high temperature and humidity, the equipment and objects in that area may be more easily damaged, and the inspection priority should be increased.

[0108] Taking into account the results of importance and risk assessment, a comprehensive inspection priority is determined for each blind spot area. The weighted sum method can be used to weight the importance and risk assessment scores to obtain the final priority score.

[0109] The monitoring blind spots are sorted according to the priority scores so that the inspection sequence can be properly arranged when planning the inspection route.

[0110] Based on the original multi-objective optimization algorithm, the inspection priority is taken into consideration as an important constraint. The optimization objectives still include the shortest inspection time and the lowest energy consumption per inspection, but at the same time, it is necessary to ensure that high-priority monitoring blind spots can be inspected first.

[0111] During the algorithm's search process, the path selection strategy is adjusted according to the priority of the blind spot area. For example, the path branch that can reach the high priority area as quickly as possible is selected.

[0112] Using the adjusted multi-objective optimization algorithm, combined with the charging position of the wheeled robot and the location of each monitoring blind spot area, an inspection path that meets the priority requirements is planned.

[0113] Evaluate and optimize the planned path to check whether there are any unreasonable places, such as too long path, frequent return, etc. If problems are found, adjust the path until a path that meets the inspection priority requirements and optimizes the inspection time and energy consumption is obtained.

[0114] In one embodiment, the information in the 3D global image is fully utilized to provide more targeted guidance for the inspection path planning of the wheeled robot, improve the inspection efficiency and quality, and promptly discover potential problems in the production workshop.

[0115] During the inspection process, by focusing on different equipment and areas according to priority, more targeted data can be collected. These data can reflect the actual operating status and potential problems of the equipment, and provide strong support for the formulation of reasonable maintenance plans. For example, by frequently inspecting high-priority equipment, you can understand the laws and trends of its failures, so as to formulate a more scientific preventive maintenance plan.

[0116] In one embodiment, based on the above embodiment, the step of controlling the wheeled robot to perform intelligent obstacle avoidance based on the detection data collected in real time during the inspection process of the wheeled robot includes: Based on the detection data collected in real time by the wheeled robot during the inspection process, analyze whether there are obstacles ahead of the wheeled robot's forward path; If so, estimate the duration of the obstacle based on visual detection technology; The obstacle avoidance strategy is selected based on the duration of the obstacle; the obstacle avoidance strategies include detour obstacle avoidance and wait obstacle avoidance.

[0117] In this embodiment, the wheeled robot is equipped with a variety of detection sensors, such as radar sensors, ultrasonic sensors, image sensors, etc. During the inspection process, these sensors will work in real time to collect relevant data of the surrounding environment.

[0118] Among them, the radar sensor obtains the distance and position information of surrounding objects by emitting laser beams and measuring the time of reflected light. It can generate high-precision three-dimensional point cloud data and clearly depict the outline of the robot's surrounding environment; the ultrasonic sensor uses the propagation characteristics of ultrasonic waves in the air to measure the distance to the object in front. It has the characteristics of low cost and fast response speed, and is suitable for close-range obstacle detection; the image sensor can capture the visual image in front of the robot and provide data support for subsequent visual inspections.

[0119] The control system processes the collected detection data in real time. For example, for the point cloud data of the radar sensor, it will perform filtering, segmentation and other operations to remove noise and irrelevant information and extract point cloud clusters that may represent obstacles; for the data of the ultrasonic sensor, it will compare the measured distance with the preset safety distance.

[0120] Based on the processed data, it is determined whether there is an obstacle ahead of the wheeled robot's forward path. If the radar sensor detects a dense point cloud cluster within a certain range ahead, or the distance measured by the ultrasonic sensor is less than the safe distance, or a possible obstacle is identified in the image captured by the image sensor, it is determined that there is an obstacle ahead of the forward path.

[0121] When it is determined that there is an obstacle ahead of the forward path, the robot will focus on analyzing the visual images collected by the image sensor. First, the image is preprocessed, including adjusting the brightness, contrast, removing noise and other operations to improve the image quality and facilitate subsequent feature extraction and analysis.

[0122] The features of obstacles are extracted from the preprocessed images using computer vision algorithms, such as edge detection, corner detection, and color feature extraction. For example, for a rectangular obstacle, its edge and corner information can be extracted; for an obstacle with a specific color, its color features can be extracted.

[0123] The extracted features are matched with the pre-established obstacle feature library to identify the type of obstacles, such as people, vehicles, cargo piles, etc. At the same time, the movement trajectory of the obstacles is tracked by analyzing continuous multi-frame images.

[0124] If the obstacle is identified as a static object, such as a fixed pile of goods, the robot will estimate the time required to bypass the obstacle based on the current inspection task and path planning, and use this as the duration of the obstacle.

[0125] For dynamic obstacles, such as moving people or vehicles, the robot will predict the time it will take to leave the forward path based on its speed, direction, and current position. Algorithms such as Kalman filtering can be used to estimate and predict the motion state of the obstacle, thereby obtaining a more accurate duration of the obstacle.

[0126] When the estimated obstacle duration is long and there is a bypass path around the robot, the bypass obstacle avoidance strategy is selected. For example, the obstacle in front is a vehicle that has been parked for a long time, and the robot can bypass the vehicle through the passage next to it to continue inspection. The control system will re-plan a path to bypass the obstacle based on the workshop map and current location information. When planning the path, factors such as the length of the path, the difficulty of passing, and whether other obstacles will be encountered will be considered to select the optimal bypass path. The wheeled robot moves along the re-planned path while continuing to collect detection data in real time to ensure that no new obstacles are encountered during the bypass process.

[0127] When the estimated duration of the obstacle is short and the detour is complex or infeasible, the waiting obstacle avoidance strategy is selected. For example, if the obstacle ahead is a person passing by briefly, the robot can continue to move forward after waiting for the person to pass. The robot will stop moving in place while keeping monitoring the obstacle. During the waiting process, the estimated duration of the obstacle will be continuously updated. If the estimated time is found to have changed, the obstacle avoidance strategy will be re-evaluated based on the new situation. When the obstacle leaves the forward path, the robot will continue to inspect along the originally planned inspection path.

[0128] During the inspection, by estimating the duration of the obstacle, the wheeled robot can flexibly choose the obstacle avoidance strategy according to the actual situation. When the obstacle lasts for a short time, the robot chooses to wait for the obstacle to avoid, avoiding the time spent on planning and executing the detour path, allowing the robot to continue to inspect along the original path in a short time, reducing unnecessary time waste. On the contrary, if the obstacle lasts for a long time, the robot uses detour obstacle avoidance, which allows the robot to quickly bypass the obstacle and continue the inspection task, avoiding the inefficiency caused by long waiting times.

[0129] In one embodiment, based on the above embodiment, the step of selecting an obstacle avoidance strategy based on the duration of the obstacle includes: Estimate the detour time required to avoid obstacles; If the detour duration is less than the obstacle avoidance duration, detour is selected as the obstacle avoidance strategy; otherwise, waiting for obstacle avoidance is selected as the obstacle avoidance strategy.

[0130] In this embodiment, the workshop map records the information of various areas in the environment, including passages, obstacle locations, etc. When it is necessary to estimate the detour time, the robot will first obtain the current location and the surrounding passable path information from the map.

[0131] Optionally, the robot uses a path planning algorithm, such as the A* algorithm, the Dijkstra algorithm, etc., to plan the best path to bypass obstacles based on the map information. By calculating the total length of the bypass path and combining it with the robot's own moving speed, a preliminary estimate of the time required to complete the bypass is obtained.

[0132] Then, an obstacle avoidance strategy is selected based on the detour duration and obstacle duration.

[0133] When the estimated detour duration is less than the duration of the obstacle, choose detour as the obstacle avoidance strategy. Because in this case, the time spent on detour is shorter than the time spent waiting for the obstacle to be removed. For example, if the estimated duration of the obstacle is 15 seconds, and the detour duration is only 10 seconds, choosing detour can resume normal inspection faster and improve the overall inspection efficiency.

[0134] Once the robot decides to avoid obstacles by detouring, it will immediately start moving along the previously planned detour path while continuing to monitor the surrounding environment in real time to ensure that no new problems are encountered during the detour.

[0135] If the detour duration is greater than or equal to the obstacle duration, wait for obstacle avoidance is selected as the obstacle avoidance strategy. In this case, the time spent waiting for the obstacle to remove itself is shorter, avoiding more time and energy consumed by detour. For example, if the estimated obstacle duration is 8 seconds and the detour duration is 12 seconds, choosing to wait can reduce unnecessary movement and path planning operations. During the waiting process, the robot will keep monitoring the obstacle and continuously update the estimate of the obstacle duration. If the estimated obstacle duration is found to have changed during the waiting process and the detour conditions are met, the robot will re-evaluate and may switch to the detour obstacle avoidance strategy.

[0136] In one embodiment, when a wheeled robot encounters an obstacle, it can quickly evaluate the time required for detouring and waiting, thereby selecting a strategy that takes less time, allowing the robot to complete the entire inspection task more quickly. This strategy based on time comparison allows the robot to plan energy use more reasonably. During the inspection process, it avoids excessive energy waste due to unreasonable obstacle avoidance decisions, ensuring that the robot has enough energy to complete subsequent inspection tasks. Especially in some large and complex inspection environments, the rational use of energy is particularly important.

[0137] In one embodiment, based on the above embodiment, the intelligent inspection method of the wheeled robot further includes: When the wheeled robot starts to conduct intelligent inspections of the production workshop, the density of people in each target area along the inspection route is obtained based on the monitoring equipment; Before the wheeled robot arrives at the corresponding target area, the moving speed of the wheeled robot is pre-adjusted according to the density of people; wherein, the higher the density of people, the lower the adjusted moving speed.

[0138] In this embodiment, when the wheeled robot starts the intelligent inspection of the production workshop, the monitoring equipment installed in the workshop is used to obtain the density of people in each target area passed by the inspection path. The monitoring equipment can be a camera, which uses computer vision technology, such as target detection, counting algorithm, etc. to identify and count people in the monitoring screen, so as to obtain the number of people in each target area. The density of people is further calculated based on factors such as the number of people and the size of the area.

[0139] When the wheeled robot moves toward the corresponding target area, before arriving, it will pre-adjust its moving speed based on the acquired population density. The principle it follows is that the higher the population density, the lower the adjusted moving speed.

[0140] For example, when it is detected that the population density in a target area is low, the wheeled robot can move at a relatively fast speed, which can improve the efficiency of the inspection; when it is detected that the population density in a target area is high, such as a large number of workers are performing concentrated work, in order to ensure the safety of personnel and avoid accidents such as collisions, the wheeled robot will reduce its movement speed and pass through the area slowly.

[0141] In one embodiment, by pre-adjusting the speed, it is possible to better adapt to environments with different population densities, reduce safety accidents caused by excessive speed, and create a safer working environment for the production workshop. The distribution of personnel in different areas of the production workshop will change with time and changes in production tasks. This method of adjusting the speed according to the density of personnel enables the wheeled robot to flexibly adapt to various scenarios and improves its ability to work in complex dynamic environments.

[0142] Moreover, it can reduce the interference of robots on the normal work of personnel, so that personnel can focus more on their production tasks. At the same time, it can also make it easier for personnel to accept the operation of robots in the workshop and promote friendly cooperation between humans and machines. Under the premise of ensuring safety, reasonable adjustment of speed will neither lead to low inspection efficiency due to too slow speed nor affect personnel work due to too fast speed, thereby improving the work efficiency of the entire production workshop.

[0143] In one embodiment, based on the above embodiment, after the step of analyzing the missing area of ​​the overall monitoring image compared with the 3D global image and identifying the missing area in combination with the latest workshop map to obtain each monitoring blind spot area in the production workshop, the step further includes: According to the layout and field of vision of monitoring equipment in the production workshop, computer simulation technology is used to conduct virtual simulation of the monitoring situation in the production workshop to simulate the monitoring blind spots in different situations; Based on the simulation results, the analysis results of the monitoring blind spot area are optimized.

[0144] In this embodiment, computer simulation technology is used to construct a virtual model of the production workshop based on the known layout of the monitoring equipment in the production workshop (i.e., the specific installation location of each monitoring equipment) and its field of view (the spatial area that each monitoring equipment can cover).

[0145] In this virtual model, various scenarios are simulated, such as the flow of personnel in different time periods, changes in the operating status of equipment, etc. Because in actual production, these factors may affect the monitoring effect of the monitoring equipment, causing areas that were not originally monitoring blind spots to become blind spots in some cases. By simulating different situations, potential monitoring blind spots can be discovered more comprehensively.

[0146] The blind spot analysis results obtained by analyzing the overall surveillance image and the 3D global image are compared with the blind spots in different situations obtained by computer simulation. If both methods identify the area as a blind spot, it is marked as confirmed; if the two methods identify different areas (i.e. one method considers it a blind spot, while the other method considers it not), it is marked as pending confirmation (this difference may be due to the limitations of image analysis (such as image resolution, shooting angle, etc.) or inaccurate parameter settings in computer simulation).

[0147] Then, for areas whose status needs to be confirmed, the wheeled robot can be controlled to re-inspect, or manual intervention can be requested for confirmation, thereby optimizing the analysis results of the monitoring blind spot areas.

[0148] Optionally, when the wheeled robot reaches the area where the status is to be confirmed, the robot can collect information about the area from multiple angles and directions to determine whether there are really blind spots in the area. For example, the robot can adjust the angle of the camera and shoot the area from different directions to see if there are any places that cannot be covered by the monitoring equipment.

[0149] Optionally, when the detection results of the wheeled robot still cannot clearly determine whether the area is a monitoring blind spot, or in some special cases (such as the wheeled robot cannot reach the area), manual intervention is requested. Manual personnel can go to the area where the status is to be confirmed for on-site inspection.

[0150] According to the results of the wheeled robot re-inspection or manual intervention confirmation, the previous monitoring blind spot area analysis results are updated. The status of the unconfirmed area is clearly divided into monitoring blind spots or non-monitoring blind spots, and updated in the final analysis report.

[0151] In one embodiment, by comparing the analysis results of different methods and further testing and confirming the area whose status is to be confirmed, it is possible to effectively reduce misjudgments and missed judgments and improve the accuracy of the analysis results of the monitoring blind spot area.

[0152] In one embodiment, based on the above embodiment, the intelligent inspection method of the wheeled robot further includes: Based on the inspection images of the wheeled robot and the monitoring images of the monitoring equipment, risk analysis and early warning are carried out on the production workshop.

[0153] In this embodiment, the wheeled robot moves in the production workshop according to the pre-planned inspection path, and the image sensor it carries collects images at certain time intervals or at specific locations. The planning of the inspection path needs to fully cover all key areas of the production workshop, including equipment-intensive areas, passages, storage areas, etc.

[0154] Fixed-installed monitoring devices in the workshop continuously capture images of the areas they are responsible for. These monitoring devices are distributed in different locations in the workshop and have different viewing angles to achieve all-round monitoring of the workshop.

[0155] The collected wheeled robot inspection images and monitoring equipment monitoring images are preprocessed, including image denoising, grayscale, histogram equalization, etc. Denoising can use methods such as Gaussian filtering to remove noise interference in the image; grayscale converts color images into grayscale images to reduce the amount of data; histogram equalization enhances the contrast of the image and improves the accuracy of subsequent analysis.

[0156] Optionally, identify various production equipment from the image and analyze their appearance and operating status. For example, check whether the shell of the equipment is damaged or deformed, whether the indicator light of the equipment is normal, whether the moving parts of the equipment are abnormally shaking, etc. Through feature extraction algorithms such as edge detection and contour extraction, the key features of the equipment can be accurately extracted.

[0157] Optionally, detect and track people in the image to analyze whether their behavior complies with safety regulations. For example, detect whether the person is wearing necessary safety protection equipment (safety helmets, goggles, etc.), whether he is illegally staying or operating in dangerous areas, and whether he is running, pushing, and other dangerous behaviors. Human posture estimation, target tracking and other technologies are used to extract the characteristics of human behavior.

[0158] Optionally, observe the environmental information in the image, such as whether there is water or oil on the ground, whether the passage is unobstructed, and whether there are abnormal conditions such as smoke or flames. Extract the characteristics of the environmental conditions through color analysis, texture analysis and other methods.

[0159] Optionally, a large amount of image data is collected in advance and risk features are annotated, such as equipment failure, personnel violation, environmental hazard, etc. The annotated data is divided into training set, validation set and test set for building and evaluating risk analysis models.

[0160] Optionally, select a suitable machine learning or deep learning model, such as a convolutional neural network, a recurrent neural network, etc. The model is trained with the training set data, and the parameters of the model are continuously adjusted so that the model can accurately identify the risk features in the image and classify the risk level.

[0161] Risks are classified into different levels, such as low risk, medium risk and high risk, according to the severity of the risk and the possible impact. For example, slight wear and tear of equipment can be classified as low risk, situations where personnel's illegal operation may lead to accidents can be classified as medium risk, and the potential risk of serious accidents such as fire and explosion can be classified as high risk.

[0162] Set corresponding warning rules for different levels of risk. When the model detects risk features in the image and determines the risk level, it triggers the corresponding warning signal according to the warning rules. For example, low risk can be warned through prompt information on the monitoring system interface, medium risk can send SMS or email to notify relevant personnel, and high risk will immediately trigger an audible and visual alarm and notify the emergency rescue department.

[0163] The inspection images of the wheeled robot and the monitoring images of the monitoring equipment are analyzed in real time. Once risk characteristics are found, an early warning is immediately issued according to the early warning rules. At the same time, the early warning information is recorded for subsequent query and analysis.

[0164] In one embodiment, the wheeled robot can flexibly move to every corner of the production workshop for inspection, and collect images of areas that are difficult to cover due to the fixed position of the monitoring equipment. The monitoring equipment can continuously and stably monitor the key areas of the workshop. The combination of the two realizes all-round and no-dead-angle monitoring of the production workshop, greatly increasing the probability of risk discovery.

[0165] Wheeled robots can collect images from different angles and distances, complementing the fixed-view images provided by monitoring equipment. This enables richer information to be obtained for the same area or object, helping to more accurately identify potential risks and reduce misjudgments and missed judgments.

[0166] The wheeled robot collects images in real time during the inspection process, and the monitoring equipment also works continuously, which can capture the dynamic changes in the production workshop in a timely manner. Whether it is a sudden failure of equipment, illegal operation of personnel or abnormal environment, it can be discovered and analyzed and warned in the first time.

[0167] In addition, refer to Figure 2 In an embodiment of the present application, a control device Z10 is further provided, comprising: The monitoring module Z11 is used to collect monitoring images of the production workshop based on monitoring devices distributed in multiple different locations in the production workshop, and to stitch the monitoring images collected by each monitoring device to generate an overall monitoring image; The extraction module Z12 is used to analyze the overall monitoring image, extract the workshop map of the production workshop, and mark the obstacles and passages in the workshop map; The scanning module Z13 is used to control the wheeled robot to perform a global scan of the production workshop based on the workshop map to generate a 3D global image; wherein, during the global scanning process, the wheeled robot supplements and updates the workshop map according to the detection data collected by the detection sensor integrated in the body; the detection sensor includes at least an image sensor, a radar sensor and a collision detection sensor; The analysis module Z14 is used to analyze the missing area of ​​the overall monitoring image compared with the 3D global image, and identify the missing area in combination with the latest workshop map to obtain the monitoring blind spots in the production workshop; The planning module Z15 is used to plan the inspection path of the wheeled robot using a multi-objective optimization algorithm based on the charging position of the wheeled robot and the location of each monitoring blind spot area; the optimization goals of the multi-objective optimization algorithm planning include the shortest inspection time for a single round and the minimum energy consumption for a single round of inspection; The inspection module Z16 is used to control the wheeled robot to perform intelligent inspection of the production workshop based on the inspection path; wherein, based on the detection data collected in real time by the wheeled robot during the inspection process, the wheeled robot is controlled to perform intelligent obstacle avoidance.

[0168] Optionally, the control device Z10 may be a virtual control device (such as a virtual machine) or a physical device (such as a physical device other than a control system that can execute a corresponding method).

[0169] In addition, a control system is also provided in an embodiment of the present application. The internal architecture of the control system can be as follows: Figure 3 As shown, it includes a processor, a memory, a communication interface and an input interface connected by a system bus. Among them, the processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database is used to store data called by the computer program. The communication interface is used to communicate data with an external terminal. The input interface is used to receive a signal input by an external device. When the computer program is executed by the processor, an intelligent inspection method of a wheeled robot as described in the above embodiment is implemented.

[0170] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the control system to which the present application scheme is applied. For example, in some optional embodiments, the control system may also include an output interface (not shown in the figure), and the output interface is also connected to the system bus and is used to output corresponding signals to the peripheral device.

[0171] In addition, the present application also proposes a computer-readable storage medium, the computer-readable storage medium includes a computer program, and when the computer program is executed by a processor, the steps of the intelligent inspection method of the wheeled robot as described in the above embodiment are implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0172] In summary, the intelligent inspection method, control device, control system and computer-readable storage medium of the wheeled robot provided in the embodiments of the present application collect and stitch images based on multi-position monitoring equipment to generate an overall monitoring image, laying the foundation for subsequent analysis, and analyzing the image to extract the workshop map and mark obstacles and channels to provide guidance for the robot's actions. The wheeled robot performs a global scan to generate a 3D global image, and uses a variety of sensors to update the map in real time to accurately identify monitoring blind spots. Then a multi-objective optimization algorithm is used to plan the inspection path, taking into account inspection time and energy consumption, improving inspection efficiency and reducing costs. In this way, a wheeled robot combined with multi-position monitoring equipment can achieve efficient, intelligent and comprehensive monitoring of the production workshop.

[0173] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0174] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0175] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An intelligent inspection method for a wheeled robot, characterized in that: include: Based on monitoring devices distributed in multiple locations in the production workshop, monitoring images of the production workshop are collected, and the monitoring images collected by each monitoring device are spliced ​​to generate an overall monitoring image; Analyze the overall monitoring image, extract the workshop map of the production workshop, and mark the obstacles and passages in the workshop map; Based on the workshop map, the wheeled robot is controlled to perform a global scan of the production workshop to generate a 3D global image; wherein, during the global scan, the wheeled robot supplements and updates the workshop map according to the detection data collected by the detection sensor integrated in the body; the detection sensor includes at least an image sensor, a radar sensor and a collision detection sensor; Analyze the missing areas of the overall monitoring image compared with the 3D global image, and identify the missing areas in combination with the latest workshop map to obtain the monitoring blind spots in the production workshop; Based on the charging position of the wheeled robot and the location of each monitoring blind spot area, a multi-objective optimization algorithm is used to plan the inspection path of the wheeled robot; the optimization goals of the multi-objective optimization algorithm planning include the shortest inspection time for a single round and the minimum energy consumption for a single round of inspection; Based on the inspection path, the wheeled robot is controlled to perform intelligent inspection of the production workshop; wherein, based on the detection data collected in real time by the wheeled robot during the inspection process, the wheeled robot is controlled to perform intelligent obstacle avoidance.

2. The intelligent inspection method of a wheeled robot according to claim 1, characterized in that: After the step of analyzing the missing area of ​​the overall monitoring image and comparing it with the 3D global image, and identifying the missing area in combination with the latest workshop map to obtain each monitoring blind spot area in the production workshop, the method further includes: Perform feature recognition on the 3D global image and use the feature recognition results to generate semantic information of devices and objects in each monitoring blind spot area; When planning the inspection route, the corresponding inspection priority is determined based on the semantic information of the equipment and objects in the monitoring blind spot area.

3. The intelligent inspection method of a wheeled robot according to claim 1, characterized in that: The step of controlling the wheeled robot to perform intelligent obstacle avoidance based on the detection data collected in real time during the inspection process of the wheeled robot comprises: Based on the detection data collected in real time by the wheeled robot during the inspection process, analyze whether there are obstacles ahead of the wheeled robot's forward path; If so, estimate the duration of the obstacle based on visual detection technology; The obstacle avoidance strategy is selected based on the duration of the obstacle; the obstacle avoidance strategies include detour obstacle avoidance and wait obstacle avoidance.

4. The intelligent inspection method of a wheeled robot as claimed in claim 3, characterized in that: The step of selecting an obstacle avoidance strategy based on the duration of the obstacle comprises: Estimate the detour time required to avoid obstacles; If the detour duration is less than the obstacle avoidance duration, detour is selected as the obstacle avoidance strategy; otherwise, waiting for obstacle avoidance is selected as the obstacle avoidance strategy.

5. The intelligent inspection method for a wheeled robot according to claim 1, characterized in that: The intelligent inspection method of the wheeled robot also includes: When the wheeled robot starts to conduct intelligent inspections of the production workshop, the density of people in each target area along the inspection route is obtained based on the monitoring equipment; Before the wheeled robot arrives at the corresponding target area, the moving speed of the wheeled robot is pre-adjusted according to the density of people; wherein, the higher the density of people, the lower the adjusted moving speed.

6. The intelligent inspection method of a wheeled robot according to claim 1, characterized in that: After the step of analyzing the missing area of ​​the overall monitoring image and comparing it with the 3D global image, and identifying the missing area in combination with the latest workshop map to obtain each monitoring blind spot area in the production workshop, the method further includes: According to the layout and field of vision of monitoring equipment in the production workshop, computer simulation technology is used to conduct virtual simulation of the monitoring situation in the production workshop to simulate the monitoring blind spots in different situations; Based on the simulation results, the analysis results of the monitoring blind spot area are optimized.

7. The intelligent inspection method for a wheeled robot according to any one of claims 1 to 6, characterized in that: The intelligent inspection method of the wheeled robot also includes: Based on the inspection images of the wheeled robot and the monitoring images of the monitoring equipment, risk analysis and early warning are carried out on the production workshop.

8. A control device, characterized in that: include: The monitoring module is used to collect monitoring images of the production workshop based on monitoring devices distributed in multiple different locations in the production workshop, and to stitch the monitoring images collected by each monitoring device to generate an overall monitoring image; An extraction module is used to analyze the overall monitoring image, extract the workshop map of the production workshop, and mark the obstacles and passages in the workshop map; A scanning module is used to control the wheeled robot to perform a global scan of the production workshop based on the workshop map to generate a 3D global image; wherein, during the global scanning process, the wheeled robot supplements and updates the workshop map according to the detection data collected by the detection sensor integrated in the body; the detection sensor includes at least an image sensor, a radar sensor and a collision detection sensor; The analysis module is used to analyze the missing area of ​​the overall monitoring image compared with the 3D global image, and identify the missing area in combination with the latest workshop map to obtain the various monitoring blind spots in the production workshop; The planning module is used to plan the inspection path of the wheeled robot based on the charging position of the wheeled robot and the location of each monitoring blind spot area using a multi-objective optimization algorithm; the optimization goals of the multi-objective optimization algorithm planning include the shortest inspection time for a single round and the minimum energy consumption for a single round of inspection; The inspection module is used to control the wheeled robot to perform intelligent inspection of the production workshop based on the inspection path; wherein, based on the detection data collected in real time by the wheeled robot during the inspection process, the wheeled robot is controlled to perform intelligent obstacle avoidance.

9. A control system, characterized in that: The control system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the intelligent inspection method of the wheeled robot as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the intelligent inspection method for a wheeled robot as claimed in any one of claims 1 to 7.

Images