Robot inspection method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a robot inspection method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: collecting environment information of a target inspection space through information collection equipment, and constructing an environment map of the target inspection space through the environment information; determining the position information of the target inspection object and the position information of the obstacle in an environment map; determining an optimal shooting point location related to the target inspection object according to the position information of the target inspection object and the position information of the obstacle, and generating an inspection path related to the target inspection object; moving to the optimal shooting point position according to the inspection path to shoot the target inspection object to obtain an inspection image about the target inspection object; and sending the inspection image to an upper system for inspection analysis to obtain a robot inspection result. By applying the technical scheme provided by the invention, more efficient and intelligent article inspection and management are realized, the article safety can be effectively guaranteed, and the labor cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a robot inspection method, and also to a robot inspection device, an electronic device, and a computer-readable storage medium. Background Art

[0002] At present, the management of goods in factories or warehouses still relies on manual inspection and management. However, with the increase in the types and quantities of items and the number of people using them, the disadvantages of manual operation are becoming increasingly apparent, the management efficiency of items is becoming increasingly low, the safety performance of items is difficult to guarantee, and labor costs are becoming increasingly high.

[0003] Therefore, how to achieve more efficient and intelligent inspection and management of items to effectively ensure the safety of items and reduce labor costs is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0004] The purpose of this application is to provide a robot inspection method, which realizes more efficient and intelligent inspection and management of items, can effectively ensure the safety of items and reduce labor costs; another purpose of this application is to provide a robot inspection device, electronic equipment and computer-readable storage medium, all of which have the above-mentioned beneficial effects.

[0005] In a first aspect, the present application discloses a robot inspection method, comprising:

[0006] Using information collection equipment to collect environmental information of the target inspection space, and using the environmental information to construct an environmental map of the target inspection space;

[0007] Determining the location information of the target inspection object and the location information of the obstacle in the environmental map;

[0008] Determine the optimal shooting point for the target inspection object according to the position information of the target inspection object and the position information of the obstacle, and generate an inspection path for the optimal shooting point;

[0009] Move to the optimal shooting point according to the inspection path to shoot the target inspection object, and obtain an inspection image of the target inspection object;

[0010] The inspection image is sent to the upper-level system for inspection analysis to obtain the robot inspection result.

[0011] Optionally, determining the location information of the target inspection object and the location information of the obstacle in the environment map includes:

[0012] Determine a first quadrilateral region where the target inspection object is located in the environment map, and use the coordinates of four vertices of the first quadrilateral region as position information of the target inspection object;

[0013] A second quadrilateral area where the obstacle is located is determined in the environment map, and the second quadrilateral area is expanded outwardly in combination with a preset outward expansion distance to generate an obstacle avoidance area, and the coordinates of four vertices of the obstacle avoidance area are used as the position information of the obstacle.

[0014] Optionally, determining an optimal shooting point position for the target inspection object according to the position information of the target inspection object and the position information of the obstacle includes:

[0015] Acquire shooting configuration information; the shooting configuration information includes a combination of one or more of shooting distance, shooting angle, and shooting field of view;

[0016] Determine an initial shooting point position of the target inspection object according to the position information of the target inspection object and the shooting configuration information;

[0017] When the initial shooting point does not hit the obstacle avoidance area, determining the initial shooting point as the optimal shooting point;

[0018] When the shooting field of view of the optimal shooting point cannot cover the target inspection object, return to the step of determining the initial shooting point of the target inspection object according to the position information of the target inspection object and the shooting configuration information to obtain multiple optimal shooting points until the total shooting field of view of all the optimal shooting points covers the target inspection object.

[0019] Optionally, generating an inspection path for the optimal shooting point includes:

[0020] Determining the coordinate information of each of the optimal shooting points and the current coordinate information of the robot;

[0021] Processing the coordinate information of each of the optimal shooting points and the current coordinate information of the robot using a shortest path algorithm to obtain an initial inspection path for the optimal shooting points;

[0022] The initial inspection path is optimized by using a path compression algorithm to obtain the inspection path about the optimal shooting point.

[0023] Optionally, the inspection image is sent to an upper-level system for inspection analysis to obtain a robot inspection result, including:

[0024] Performing feature analysis on the inspection image to obtain target image features;

[0025] Sending the inspection image and the target image features to the upper system for inspection analysis to obtain the robot inspection result;

[0026] A management operation on the target inspection object is performed according to the robot inspection result.

[0027] Optionally, the target inspection object is a commodity shelf; and performing feature analysis on the inspection image to obtain target image features includes:

[0028] Using the inspection image to perform out-of-stock detection on the commodity shelf to obtain out-of-stock features; the out-of-stock features include out-of-stock commodity features and out-of-stock location features;

[0029] Accordingly, performing management operations on the target inspection object according to the robot inspection result includes:

[0030] Replenishment operations are performed on the commodity shelves according to the out-of-stock features in the robot inspection results.

[0031] Optionally, the target inspection object is a commodity shelf; and performing feature analysis on the inspection image to obtain target image features includes:

[0032] Using the inspection image to perform label detection on the commodity shelf to obtain label features; the label features include features of the mis-signed commodity and features of the mis-signed location;

[0033] Accordingly, performing management operations on the target inspection object according to the robot inspection result includes:

[0034] A label update operation is performed on the commodity shelf according to the label features in the robot inspection result.

[0035] In a second aspect, the present application also discloses a robot inspection device, comprising:

[0036] A construction module, used to collect environmental information of a target inspection space using an information collection device, and construct an environmental map of the target inspection space using the environmental information;

[0037] A determination module, used to determine the location information of the target inspection object and the location information of the obstacle in the environment map;

[0038] A generation module, used to determine the optimal shooting point position of the target inspection object according to the position information of the target inspection object and the position information of the obstacle, and generate an inspection path for the target inspection object;

[0039] A shooting module is used to move to the optimal shooting point according to the inspection path to shoot the target inspection object and obtain an inspection image of the target inspection object;

[0040] The sending module is used to send the inspection image to the upper system for inspection analysis to obtain the robot inspection result.

[0041] In a third aspect, the present application further discloses an electronic device, comprising:

[0042] Memory for storing computer programs;

[0043] A processor is used to implement the steps of any one of the robot inspection methods described above when executing the computer program.

[0044] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the robot inspection methods described above are implemented.

[0045] The present application provides a robot inspection method, including: using information collection equipment to collect environmental information of a target inspection space, and using the environmental information to construct an environmental map of the target inspection space; determining the position information of the target inspection object and the position information of obstacles in the environmental map; determining the optimal shooting point position for the target inspection object based on the position information of the target inspection object and the position information of the obstacles, and generating an inspection path for the optimal shooting point position; moving to the optimal shooting point position according to the inspection path to shoot the target inspection object, and obtaining an inspection image of the target inspection object; sending the inspection image to an upper-level system for inspection analysis to obtain a robot inspection result.

[0046] By applying the technical solution provided by this application, an automated intelligent inspection is realized by using a robot. First, the environmental information in the target inspection space is collected to construct an environmental map of the target inspection space, and the target inspection objects and obstacles therein are determined. Thus, the optimal shooting point for the target inspection object can be determined according to the position information of the target inspection object and the obstacles in the environmental map, that is, the position most suitable for shooting the target inspection object, thereby generating an inspection path for the robot to reach the optimal shooting point to complete the shooting of the target inspection object. Finally, the inspection image of the target inspection object obtained by shooting is uploaded to the upper-level system for inspection analysis, and the robot inspection result can be obtained. Obviously, based on this technical solution, automated intelligent inspection is realized, which completely gets rid of manual dependence, greatly improves the management efficiency of objects (target inspection objects), effectively ensures the safety of objects, and reduces labor costs.

[0047] The robot inspection device, electronic device and computer-readable storage medium provided in this application also have the above-mentioned technical effects, which will not be elaborated in this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.

[0049] Figure 1 A schematic diagram of a process flow of a robot inspection method provided in this application;

[0050] Figure 2 A functional schematic diagram of a robot inspection method provided in this application;

[0051] Figure 3 A flowchart of another robot inspection method provided by this application;

[0052] Figure 4 A schematic diagram of the structure of a robot inspection device provided in this application;

[0053] Figure 5 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0054] The core of this application is to provide a robot inspection method, which realizes more efficient and intelligent inspection and management of items, can effectively ensure the safety of items and reduce labor costs; another core of this application is to provide a robot inspection device, electronic equipment and computer-readable storage medium, all of which have the above-mentioned beneficial effects.

[0055] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0056] An embodiment of the present application provides a robot inspection method.

[0057] Please refer to Figure 1 , Figure 1This is a flow chart of a robot inspection method provided in the present application. The robot inspection method may include the following S101 to S105.

[0058] S11: Collect environmental information of the target inspection space using information collection equipment, and construct an environmental map of the target inspection space using the environmental information.

[0059] This step aims to realize the collection of environmental information in the target inspection space and the generation of an environmental map, wherein the target inspection space is the space that currently needs to be inspected by the robot, such as a factory or a warehouse. Specifically, the environmental information in the target inspection space can be first collected using information collection equipment, which can be a camera, sensor or other equipment, which can be directly installed on the robot or installed in a dispersed manner at a specific location in the target inspection space. This application does not limit this; thus, the collected environmental information can be used to construct an environmental map in the target inspection space. In one possible implementation, SLAM (Simultaneous Localization and Mapping) technology can be used to build and update the environmental map in real time.

[0060] S12: Determine the location information of the target inspection object and the location information of the obstacle in the environment map.

[0061] This step aims to determine the positions of the target inspection objects and obstacles in the environment map, where the target inspection objects are the target objects that need to be inspected, such as various types of goods stored in the warehouse; obstacles are the target objects that affect the movement of the robot, such as the walls in the warehouse, other items other than the target inspection objects, etc. It can be understood that by determining the positions of the target inspection objects and obstacles in the environment map, obstacle avoidance and target inspection object positioning can be achieved, which helps to achieve more accurate and safe robot inspections.

[0062] In one embodiment of the present application, determining the location information of the target inspection object and the location information of the obstacle in the environment map may include:

[0063] Determine a first quadrilateral region where the target inspection object is located in the environment map, and use the coordinates of four vertices of the first quadrilateral region as position information of the target inspection object;

[0064] A second quadrilateral region where the obstacle is located is determined in the environment map, and the second quadrilateral region is expanded outwardly according to a preset outward expansion distance to generate an obstacle avoidance region, and the coordinates of four vertices of the obstacle avoidance region are used as the position information of the obstacle.

[0065] The embodiment of the present application provides a method for determining the position of a target inspection object and an obstacle. First, with respect to the target inspection object, the four vertex coordinates of the area where the target inspection object is located (specifically, it can be a quadrilateral, i.e., the first quadrilateral area mentioned above) can be directly used to mark the position of the target inspection object; secondly, with respect to obstacles, since the robot needs to avoid obstacles during the inspection process, in order to ensure the normal movement of the robot, the area where the obstacle is located (which can also be a quadrilateral, i.e., the second quadrilateral area mentioned above) can be expanded by a certain distance (i.e., the preset expansion distance) to form an obstacle avoidance area, and then the four vertex coordinates of the obstacle avoidance area can be used to mark the position of the obstacle. Of course, the specific value of the preset expansion distance does not affect the implementation of the present technical solution, and can be set by technical personnel according to actual conditions, and the present application does not limit this.

[0066] S13: determining an optimal shooting point for the target inspection object according to the position information of the target inspection object and the position information of the obstacle, and generating an inspection path for the optimal shooting point.

[0067] This step aims to determine the optimal shooting point of the target inspection object and generate the inspection path of the optimal shooting point. The optimal shooting point is the position that is most suitable for shooting the target inspection object, that is, the position that can obtain the best shooting effect of the target inspection object (such as full coverage of the shooting field of view, optimal shooting light, etc.), and the inspection path is the path that the robot moves to the optimal shooting point.

[0068] In one embodiment of the present application, determining the optimal shooting point position of the target inspection object according to the position information of the target inspection object and the position information of the obstacle may include:

[0069] Acquire shooting configuration information; the shooting configuration information includes one or more combinations of shooting distance, shooting angle, and shooting field of view;

[0070] Determine the initial shooting point of the target inspection object according to the location information and shooting configuration information of the target inspection object;

[0071] When the initial shooting point does not hit the obstacle avoidance area, the initial shooting point is determined to be the optimal shooting point;

[0072] When the shooting field of view of the optimal shooting point cannot cover the target inspection object, return to the step of determining the initial shooting point of the target inspection object according to the location information and shooting configuration information of the target inspection object to obtain multiple optimal shooting points until the total shooting field of view of all optimal shooting points covers the target inspection object.

[0073] The embodiment of the present application provides a method for determining the optimal shooting point of the target inspection object. It is understandable that there may be different shooting requirements for different target inspection objects, including but not limited to shooting distance, shooting angle, shooting field of view, etc. In the specific implementation process, these shooting requirements can be pre-set in the form of configuration information, so that the shooting configuration information and the location information of the target inspection object can be combined to determine the appropriate shooting position (initial shooting point); further, after determining the initial shooting point, it is also necessary to avoid obstacles, that is, avoid the shooting point being located in the obstacle avoidance area, so as to obtain the optimal shooting point. However, according to the field of view of the shooting device, there may be a situation where an optimal shooting point cannot completely cover the target inspection object. At this time, the optimal shooting point can continue to be found, so that the total field of view of multiple optimal shooting points can achieve full coverage of the target inspection object, ensuring that the most comprehensive inspection image of the target inspection object can be obtained, thereby ensuring the accuracy of the robot inspection results.

[0074] In one embodiment of the present application, generating an inspection path for an optimal shooting point may include:

[0075] Determine the coordinate information of each optimal shooting point and the current coordinate information of the robot;

[0076] The coordinate information of each optimal shooting point and the current coordinate information of the robot are processed using the shortest path algorithm to obtain the initial inspection path for the optimal shooting point;

[0077] The path compression algorithm is used to optimize the initial inspection path to obtain the inspection path with the optimal shooting point.

[0078] The embodiment of the present application provides a method for generating an inspection path of an optimal shooting point, and this process needs to be combined with the current position of the robot. It should be noted that in the process of generating the inspection path, in order to ensure the efficiency of the robot inspection, the arrival order of the robots at each optimal shooting point can be reasonably arranged to avoid repeated inspection paths, and a shortest robot inspection path can be obtained while avoiding obstacles. Based on this, this process can be implemented in combination with the shortest path algorithm and the path compression algorithm; at the same time, obstacle avoidance can be implemented using a collision detection algorithm. Furthermore, under the requirements of cyclic inspections, dynamic path planning based on a time window can also be set so that the robot can perform inspections using the optimal path in different inspection cycles.

[0079] S14: moving to the optimal shooting point according to the inspection path to shoot the target inspection object to obtain an inspection image of the target inspection object.

[0080] This step is intended to achieve the capture of inspection images of the target inspection object. Specifically, after determining the inspection path of the optimal shooting point, the robot can be controlled to move to the optimal shooting point along the inspection path to capture the target inspection object, thereby obtaining one or more inspection images of the target inspection object. It is understandable that the shooting device is directly mounted on the robot.

[0081] S15: Send the inspection image to the upper system for inspection analysis to obtain the robot inspection result.

[0082] This step aims to realize inspection image processing based on the upper-level system to obtain the robot inspection results about the target inspection object and complete the robot inspection.

[0083] In one embodiment of the present application, sending the inspection image to the upper system for inspection analysis to obtain the robot inspection result may include:

[0084] Perform feature analysis on the inspection image to obtain target image features;

[0085] Send the inspection image and target image features to the upper-level system for inspection analysis to obtain the robot inspection results;

[0086] Execute management operations on the target inspection object based on the robot inspection results.

[0087] The embodiment of the present application provides a method for analyzing inspection images. Specifically, after capturing the inspection image of the target inspection object, the robot can first perform local analysis to extract inspection image features, and then upload the inspection image and its target image features to the upper-level system for further analysis to obtain the robot inspection results. Thus, the corresponding management operations can be performed on the target inspection object according to the robot inspection results, thereby realizing inspection object management based on automatic robot inspection. Among them, the extraction process of the target image features can be implemented through the corresponding neural network model, and the extracted feature content can be determined by the inspection requirements, the type of target inspection object, etc., which will not be repeated in this application.

[0088] Of course, in order to avoid the problem of unstable system operation caused by frequent data (inspection images and their target image features) uploading operations, after completing the robot's local analysis, the inspection images and their target image features can also be saved locally until the upload command is received, and then the data upload operation can be performed.

[0089] In a possible implementation, the target inspection object is a commodity shelf; performing feature analysis on the inspection image to obtain the target image feature may include: using the inspection image to perform out-of-stock detection on the commodity shelf to obtain out-of-stock features; the out-of-stock features include out-of-stock commodity features and out-of-stock location features;

[0090] Accordingly, performing management operations on the target inspection object according to the robot inspection results may include: performing a replenishment operation on the commodity shelf according to the out-of-stock characteristics in the robot inspection results.

[0091] The embodiment of the present application takes the inspection of commodity shelves as an example and provides an implementation method for extracting target image features. Specifically, for the inspection image of the commodity shelf, out-of-stock detection can be performed to obtain out-of-stock features. It can be understood that the out-of-stock feature is used to characterize whether the commodity shelf is out of stock. Therefore, based on the out-of-stock feature, it can be determined whether the commodity shelf currently needs to be replenished. Among them, the out-of-stock feature can specifically include out-of-stock commodity features and out-of-stock location features. The out-of-stock commodity features may include but are not limited to the out-of-stock commodity type, the out-of-stock commodity quantity, etc., and the out-of-stock location features may include information such as the spatial coordinates of the out-of-stock commodity. Therefore, the replenishment operation can be conveniently implemented based on the out-of-stock feature.

[0092] In another possible implementation, the target inspection object is a commodity shelf; performing feature analysis on the inspection image to obtain target image features may include: using the inspection image to perform label detection on the commodity shelf to obtain label features; the label features include features of the wrongly signed commodity and features of the wrongly signed location;

[0093] Accordingly, performing management operations on the target inspection object according to the robot inspection result may include: performing a label update operation on the commodity shelf according to the label features in the robot inspection result.

[0094] The embodiment of the present application takes the inspection of commodity shelves as an example and provides another implementation method for extracting target image features. Specifically, for the inspection image of the commodity shelf, label detection can be performed on it to obtain label features. It can be understood that the label feature can specifically be a commodity price tag feature, which is used to indicate whether the commodity price tag is missing, whether it is in the wrong position, whether the content is wrong, etc. Therefore, based on the commodity price tag feature, it can be determined whether the commodity shelf currently needs to be updated (corrected) by the label. Similarly, the label feature can include the wrongly signed commodity feature and the wrongly signed position feature. The wrongly signed commodity feature can specifically be the wrongly signed commodity type, and the wrongly signed position feature can specifically be the label space coordinates of the wrongly signed commodity. Therefore, based on the label feature, the label of the incorrectly labeled commodity can be easily updated.

[0095] In addition, before performing feature analysis on the inspection image and obtaining the target image features, it may also include: performing preprocessing operations on the inspection image; wherein the preprocessing operations include a combination of one or more of denoising, cropping, and light balance. In other words, in order to effectively ensure the accuracy of the image analysis results, after obtaining the inspection image, before performing feature analysis on the inspection image, an image preprocessing operation may be performed on the inspection image, such as denoising, cropping, and light balance as described above, to obtain a high-quality inspection image to achieve high-accuracy image analysis, which helps to achieve more accurate robot inspections.

[0096] It can be seen that the robot inspection method provided in the embodiment of the present application realizes automated intelligent inspection by using a robot. First, the environmental information in the target inspection space is collected to construct an environmental map of the target inspection space, and the target inspection objects and obstacles therein are determined. Thus, the optimal shooting point for the target inspection object can be determined according to the position information of the target inspection object and the obstacles in the environmental map, that is, the position most suitable for shooting the target inspection object, thereby generating an inspection path for the robot to reach the optimal shooting point to complete the shooting of the target inspection object. Finally, the inspection image of the target inspection object obtained by shooting is uploaded to the upper-level system for inspection analysis, and the robot inspection result can be obtained. Obviously, based on this technical solution, automated intelligent inspection is realized, which completely gets rid of manual dependence, greatly improves the management efficiency of objects (target inspection objects), effectively ensures the safety of objects, and reduces labor costs.

[0097] Based on the above-mentioned application embodiments, the present application provides another robot inspection method.

[0098] Please refer to Figure 2 and Figure 3 , Figure 2 This is a functional schematic diagram of a robot inspection method provided in this application. Figure 3 This is a flow chart of another robot inspection method provided by the present application. The implementation process of the robot inspection method provided by the embodiment of the present application is as follows:

[0099] Step S101, environmental perception and map construction:

[0100] After the robot is started, it can collect environmental information through cameras and sensors, and use SLAM technology to build and update the environmental map in real time, while marking the locations of inspection objects and obstacles.

[0101] Step S102: Path planning and shooting point generation for inspection tasks:

[0102] According to the coordinate position of the inspection object, combined with the coordinate position and expansion distance of the obstacle, as well as the shooting requirements, the optimal shooting point coordinates are automatically generated, and the inspection path is planned based on this to ensure that the robot can cover all inspection objects and avoid obstacles. It can be understood that the key to realizing the path planning and shooting point generation of the inspection task lies in how to generate appropriate shooting points based on the inspection object position, obstacle position and shooting requirements, and plan the optimal path accordingly. The following is the specific implementation method and principle.

[0103] 1. Data preparation:

[0104] (1) Inspection object coordinates: Locate the coordinates of all objects that need to be inspected (such as shelves, goods, etc.). Each object, which is a quadrilateral by default, is represented by the (x, y) coordinates of its four vertices.

[0105] (2) Obstacle position and expansion distance: According to the actual boundary of the obstacle, set a safe expansion distance to form an obstacle avoidance zone. The expansion boundary can still be a quadrilateral, simplified to a set of several (x, y) coordinates.

[0106] (3) Shooting requirements: Shooting requirements define the distance between the shooting point and the inspection object, the shooting angle, and the field of view. Based on these requirements, shooting points are generated to ensure full coverage.

[0107] 2. Shooting point generation

[0108] (1) Generation method: For each inspection object, according to the predetermined shooting distance and angle, a certain number of pixels are expanded outward at the front end and the distance is evenly divided to generate the shooting point. For example, the front of the inspection object is used as the reference direction, and the shooting point can be set by offsetting the corresponding angle based on this reference. In addition, if there are obstacles around the inspection object, the generation of the shooting point can be adjusted to avoid the obstacles and ensure that the robot can reach the shooting point smoothly; this process can be combined with the collision detection algorithm in the specific implementation to determine whether the shooting point is located in the obstacle area.

[0109] (2) Field of view adjustment: Based on the field of view of the camera, if a single shooting point cannot completely cover the inspection object, multiple shooting points can be generated. For example, an image coverage model can be used to measure the field of view coverage between a given shooting point and the inspection object, and additional shooting points can be added if necessary.

[0110] 3. Path planning:

[0111] (1) Path planning based on shooting points: Use the shortest path algorithm to plan the path from the starting position to each shooting point to ensure the optimal path and avoid obstacles. In this process, the order of each shooting point needs to be considered to avoid repeated paths and determine the optimal path order for visiting each shooting point.

[0112] (2) Path optimization: To reduce path duplication, a path compression algorithm can be used to merge adjacent shooting points as much as possible and arrange the visit order of the inspection objects in a reasonable manner. In addition, if there is a need for cyclic inspection, dynamic path planning based on a time window can be set up so that the robot can inspect all objects using the optimal path within different inspection cycles.

[0113] Based on this, please refer to the following examples:

[0114] Suppose that a group of shelves in a convenience store needs to be inspected, marked as A, with corner coordinates of (1, 1), (1, 2), (2, 1), and (2, 2). The shooting requirements are: a distance of 0.5 m from the shelves, and shooting points spaced 0.5 m apart.

[0115] (1) Shooting point generation: Based on the shooting requirements, shooting points are generated along the direction of travel with the coordinates of each shelf as the center, namely (0.5, 1.5), (1.5, 2.5), (2.5, 1.5), and (1.5, 0.5).

[0116] (2) Path planning: The optimal inspection sequence is determined as follows: the robot starts from the starting point (0, 0), goes to the shooting points (0.5, 1.5) → (1.5, 2.5) → (2.5, 1.5) → (1.5, 0.5), and then returns to the starting point (0, 0).

[0117] This method realizes path planning based on the location of the inspection object and shooting requirements, ensuring the effective inspection and safe obstacle avoidance of the inspection robot.

[0118] Step S103, visual recognition and data collection:

[0119] After the robot reaches the designated shooting point, it can use visual recognition technology to collect shelf image information and detect the status of goods on the shelf through the board-side AI model, including whether the goods are out of stock, whether the labels are missing, etc. Among them, in the out-of-stock detection or price tag detection tasks, the robot's visual recognition and data collection process can include the following steps:

[0120] 1. Arrival at the shooting point: The robot follows the inspection path and uses navigation and positioning technology to accurately reach the designated shelf point. During this process, it is necessary to ensure the optimal shooting distance and angle between itself and the shelf to obtain a clear and complete shelf image.

[0121] 2. Image acquisition: The robot uses a high-definition camera or a multi-camera system to capture images of the shelves and collect image data of all SKUs (Stock Keeping Units) on the shelves. These images will become the input for subsequent detection.

[0122] 3. Image preprocessing: The collected images are preprocessed on the board-side device, including denoising, cropping, and light balancing, to ensure the quality and consistency of the image and improve the accuracy of subsequent detection.

[0123] 4. Out-of-stock detection / price tag detection: The pre-processed image is analyzed by the board-side AI model. The model can identify the status of goods on the shelf based on the features learned from the training data, for example:

[0124] (1) Out-of-stock detection: The model identifies whether there are empty spots for products on the shelf and determines the out-of-stock situation based on the SKU information;

[0125] (2) Price tag detection: The model detects the location and status of the price tag of each product to determine whether there are any price tags that are missing, in the wrong location, or have incorrect content.

[0126] 5. Data recording and reporting: The robot stores the detection results (such as out-of-stock SKU numbers, missing price tag locations, etc.) in the local database and synchronously uploads the data to the central system at the appropriate communication window for further analysis or generation of task reports.

[0127] 6. Exception marking and task update: Based on the detection results, the robot will mark the exceptions as content that requires manual inspection or subsequent task processing, thereby supporting store managers to respond quickly.

[0128] Through this process, the robot can efficiently and accurately identify abnormal conditions of shelves and provide retail stores with timely operational data and decision support.

[0129] Step S104, data upload and result notification:

[0130] The test results are uploaded to the management system in real time, reminder information is generated for identified abnormal situations, and store staff are notified so that shelf problems can be dealt with in a timely manner.

[0131] It can be seen that the robot inspection method provided in the embodiment of the present application has the following technical advantages:

[0132] (1) Improve inspection efficiency: Through autonomous robot navigation, intelligent path planning and shooting points, the time and labor costs of manual inspections can be greatly reduced, and efficient execution of inspection tasks can be achieved through automation.

[0133] (2) Optimize product management: Use visual recognition and AI detection models to identify problems such as out-of-stock products and missing price tags in real time, ensure that products on the shelves are neatly arranged, and reduce sales losses caused by out-of-stock products or incorrect price tags.

[0134] (3) Real-time data feedback and intelligent statistics: Through real-time data uploading and intelligent analysis functions, managers can understand the status of shelves in real time, identify problem areas, optimize replenishment and display strategies, and improve overall warehouse utilization.

[0135] (4) Reduce manual intervention and improve the degree of automation: The system supports the automatic management of scheduled and cyclic inspection tasks, significantly reducing the frequency of manual operations during the inspection process, realizing the periodic automatic execution of inspection work, and further improving the convenience and stability of management.

[0136] (5) Adapt to dynamic environments and improve environmental adaptability: Based on SLAM technology, the robot can realize autonomous positioning and path planning in complex environments. It has good environmental adaptability, can avoid obstacles in dynamic scenes, and effectively complete inspection tasks.

[0137] It can be seen that this technical solution has significantly improved the intelligence level of inspection and shelf management, provided a convenient and efficient solution for retail management, and has significant economic and operational benefits.

[0138] An embodiment of the present application provides a robot inspection device.

[0139] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of a robot inspection device provided in this application. The robot inspection device may include:

[0140] Construction module 1 is used to collect environmental information of the target inspection space using information collection equipment, and construct an environmental map of the target inspection space using the environmental information;

[0141] Determination module 2, used to determine the location information of the target inspection object and the location information of the obstacle in the environment map;

[0142] Generating module 3, used to determine the optimal shooting point position of the target inspection object according to the position information of the target inspection object and the position information of the obstacle, and generate the inspection path of the optimal shooting point position;

[0143] The shooting module 4 is used to move to the optimal shooting point according to the inspection path to shoot the target inspection object and obtain the inspection image of the target inspection object;

[0144] The sending module 5 is used to send the inspection image to the upper system for inspection analysis to obtain the robot inspection result.

[0145] It can be seen that the robot inspection device provided in the embodiment of the present application realizes automated intelligent inspection by using a robot. First, the environmental information in the target inspection space is collected to construct an environmental map of the target inspection space, and the target inspection objects and obstacles therein are determined. Thus, the optimal shooting point for the target inspection object can be determined according to the position information of the target inspection object and the obstacles in the environmental map, that is, the position most suitable for shooting the target inspection object, thereby generating an inspection path for the robot to reach the optimal shooting point to complete the shooting of the target inspection object. Finally, the inspection image of the target inspection object obtained by shooting is uploaded to the upper-level system for inspection analysis, and the robot inspection result can be obtained. Obviously, based on this technical solution, automated intelligent inspection is realized, which completely gets rid of manual dependence, greatly improves the management efficiency of objects (target inspection objects), effectively ensures the safety of objects, and reduces labor costs.

[0146] In one embodiment of the present application, the above-mentioned determination module 2 can be specifically used to determine a first quadrilateral area where the target inspection object is located in the environmental map, and use the four vertex coordinates of the first quadrilateral area as the location information of the target inspection object; determine a second quadrilateral area where the obstacle is located in the environmental map, and expand the second quadrilateral area in combination with a preset expansion distance to generate an obstacle avoidance area, and use the four vertex coordinates of the obstacle avoidance area as the location information of the obstacle.

[0147] In one embodiment of the present application, the above-mentioned generation module 3 can be specifically used to obtain shooting configuration information; the shooting configuration information includes a combination of one or more of shooting distance, shooting angle, and shooting field of view; determining the initial shooting point of the target inspection object according to the position information of the target inspection object and the shooting configuration information; when the initial shooting point does not hit the obstacle avoidance area, determining the initial shooting point as the optimal shooting point; when the shooting field of view of the optimal shooting point cannot cover the target inspection object, returning to the step of determining the initial shooting point of the target inspection object according to the position information and shooting configuration information of the target inspection object, so as to obtain multiple optimal shooting points, until the total shooting field of view of all optimal shooting points covers the target inspection object.

[0148] In one embodiment of the present application, the above-mentioned generation module 3 can be specifically used to determine the coordinate information of each optimal shooting point and the current coordinate information of the robot; use the shortest path algorithm to process the coordinate information of each optimal shooting point and the current coordinate information of the robot to obtain an initial inspection path for the optimal shooting point; use the path compression algorithm to optimize the initial inspection path to obtain an inspection path for the optimal shooting point.

[0149] In one embodiment of the present application, the sending module 5 may include:

[0150] An analysis unit, used for performing feature analysis on the inspection image to obtain target image features;

[0151] A sending unit is used to send the inspection image and target image features to the upper system for inspection analysis to obtain the robot inspection result;

[0152] The execution unit is used to execute management operations on the target inspection object according to the robot inspection result.

[0153] In one embodiment of the present application, the target inspection object is a commodity shelf; the above-mentioned analysis unit can be specifically used to use the inspection image to perform out-of-stock detection on the commodity shelf to obtain out-of-stock features; the out-of-stock features include out-of-stock commodity features and out-of-stock location features;

[0154] Accordingly, the above-mentioned execution unit can be specifically used to perform replenishment operations on commodity shelves according to the out-of-stock characteristics in the robot inspection results.

[0155] In one embodiment of the present application, the inspection target object is a commodity shelf; the above-mentioned analysis unit can be specifically used to detect labels on the commodity shelf using the inspection image to obtain label features; the label features include wrongly signed commodity features and wrongly signed location features;

[0156] Accordingly, the above-mentioned execution unit can be specifically used to perform label update operations on commodity shelves according to label features in the robot inspection results.

[0157] For an introduction to the device provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0158] An embodiment of the present application provides an electronic device.

[0159] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application, and the electronic device may include:

[0160] Memory for storing computer programs;

[0161] The processor is used to implement the steps of any one of the above-mentioned robot inspection methods when executing a computer program.

[0162] like Figure 5 , which is a schematic diagram of the composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0163] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0164] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the robot inspection method.

[0165] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0166] Use information collection equipment to collect environmental information of the target inspection space, and use the environmental information to construct an environmental map of the target inspection space;

[0167] Determine the location information of the target inspection object and the location information of obstacles in the environmental map;

[0168] Determine the optimal shooting point for the target inspection object according to the location information of the target inspection object and the location information of the obstacle, and generate an inspection path for the optimal shooting point;

[0169] Move to the optimal shooting point according to the inspection path to shoot the target inspection object, and obtain the inspection image of the target inspection object;

[0170] The inspection images are sent to the upper-level system for inspection analysis to obtain the robot inspection results.

[0171] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0172] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0173] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0174] Of course, it should be noted that Figure 5 The structure shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 5 More or fewer components than shown, or combinations of certain components.

[0175] An embodiment of the present application provides a computer-readable storage medium.

[0176] The computer-readable storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned robot inspection methods can be implemented.

[0177] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0178] For an introduction to the computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0179] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0180] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0181] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0182] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A robot inspection method, characterized in that: include: Using information collection equipment to collect environmental information of the target inspection space, and using the environmental information to construct an environmental map of the target inspection space; Determining the location information of the target inspection object and the location information of the obstacle in the environmental map; Determine the optimal shooting point for the target inspection object according to the position information of the target inspection object and the position information of the obstacle, and generate an inspection path for the optimal shooting point; Move to the optimal shooting point according to the inspection path to shoot the target inspection object, and obtain an inspection image of the target inspection object; The inspection image is sent to the upper-level system for inspection analysis to obtain the robot inspection result.

2. The robot inspection method according to claim 1, characterized in that: Determining the location information of the target inspection object and the location information of the obstacle in the environment map includes: Determine a first quadrilateral region where the target inspection object is located in the environment map, and use the coordinates of four vertices of the first quadrilateral region as position information of the target inspection object; A second quadrilateral area where the obstacle is located is determined in the environment map, and the second quadrilateral area is expanded outwardly in combination with a preset outward expansion distance to generate an obstacle avoidance area, and the coordinates of four vertices of the obstacle avoidance area are used as the position information of the obstacle.

3. The robot inspection method according to claim 2, characterized in that: Determining an optimal shooting point position for the target inspection object according to the position information of the target inspection object and the position information of the obstacle includes: Acquire shooting configuration information; the shooting configuration information includes a combination of one or more of shooting distance, shooting angle, and shooting field of view; Determine an initial shooting point position of the target inspection object according to the position information of the target inspection object and the shooting configuration information; When the initial shooting point does not hit the obstacle avoidance area, determining the initial shooting point as the optimal shooting point; When the shooting field of view of the optimal shooting point cannot cover the target inspection object, return to the step of determining the initial shooting point of the target inspection object according to the position information of the target inspection object and the shooting configuration information to obtain multiple optimal shooting points until the total shooting field of view of all the optimal shooting points covers the target inspection object.

4. The robot inspection method according to claim 3, characterized in that: Generating an inspection path for the optimal shooting point includes: Determining the coordinate information of each of the optimal shooting points and the current coordinate information of the robot; Processing the coordinate information of each of the optimal shooting points and the current coordinate information of the robot using a shortest path algorithm to obtain an initial inspection path for the optimal shooting points; The initial inspection path is optimized by using a path compression algorithm to obtain the inspection path about the optimal shooting point.

5. The robot inspection method according to claim 1, characterized in that: The inspection image is sent to the upper system for inspection analysis to obtain the robot inspection result, including: Performing feature analysis on the inspection image to obtain target image features; Sending the inspection image and the target image features to the upper system for inspection analysis to obtain the robot inspection result; A management operation on the target inspection object is performed according to the robot inspection result.

6. The robot inspection method according to claim 5, characterized in that: The target inspection object is a commodity shelf; performing feature analysis on the inspection image to obtain target image features includes: Using the inspection image to perform out-of-stock detection on the commodity shelf to obtain out-of-stock features; the out-of-stock features include out-of-stock commodity features and out-of-stock location features; Accordingly, performing management operations on the target inspection object according to the robot inspection result includes: Replenishment operations are performed on the commodity shelves according to the out-of-stock features in the robot inspection results.

7. The robot inspection method according to claim 5, characterized in that: The target inspection object is a commodity shelf; performing feature analysis on the inspection image to obtain target image features includes: Using the inspection image to perform label detection on the commodity shelf to obtain label features; the label features include features of the mis-signed commodity and features of the mis-signed location; Accordingly, performing management operations on the target inspection object according to the robot inspection result includes: A label update operation is performed on the commodity shelf according to the label features in the robot inspection result.

8. A robot inspection device, characterized in that: include: A construction module, used to collect environmental information of a target inspection space using an information collection device, and construct an environmental map of the target inspection space using the environmental information; A determination module, used to determine the location information of the target inspection object and the location information of the obstacle in the environment map; A generation module, used to determine the optimal shooting point position for the target inspection object according to the position information of the target inspection object and the position information of the obstacle, and generate an inspection path for the optimal shooting point position; A shooting module is used to move to the optimal shooting point according to the inspection path to shoot the target inspection object and obtain an inspection image of the target inspection object; The sending module is used to send the inspection image to the upper system for inspection analysis to obtain the robot inspection result.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the robot inspection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot inspection method according to any one of claims 1 to 7 are implemented.

Images