Drug delivery method and robot based on openmv and autonomous path planning

Abstract

The application relates to a medicine delivery method and robot based on OpenMV and autonomous path planning. The method comprises the following steps: acquiring a medicine delivery map and the number of a target room; determining a room point corresponding to the number of the target room as an end point, and performing path planning based on the medicine delivery map and the end point by using a path planning algorithm to obtain a target path; when the medicine is in the robot, generating a marching instruction according to the target path, and the marching instruction is used for instructing the robot to march on a corresponding road surface in reality according to the target path; in the marching process of the robot, a guide line preset on the road surface is recognized based on an OpenMV image recognition technology, and the marching instruction is adjusted based on the guide line. Through the preset medicine delivery map and the number of the target room, the actual route to the target room is quickly obtained by using the path planning algorithm, and the robot is prevented from deviating from an ideal line due to marching errors in the medicine delivery process, so that the medicine delivery efficiency of the medicine delivery robot is improved.

Description

Technical Field

[0001] This invention belongs to the field of medical robot technology, and in particular relates to a drug delivery method and robot based on OpenMV and autonomous path planning. Background Technology

[0002] With the aging population and increasing demand for healthcare, hospitals and other medical facilities urgently need to improve service efficiency and quality. In terms of drug delivery, traditional methods rely heavily on manual operation, which is inefficient, error-prone, and poses a risk of infection in certain environments. While technological advancements have brought some improvements, existing drug delivery robots still have shortcomings in recognition accuracy and pathfinding efficiency.

[0003] Current medicine delivery robots are not intelligent and efficient enough in finding their way. They often cannot make quick and dynamic adjustments according to the complex actual environment, which may lead to detours and jams during the delivery process, greatly affecting the efficiency of medicine delivery. Summary of the Invention

[0004] Therefore, it is necessary to address the aforementioned technical problems that may occur during the drug delivery process, such as detours and blockages, which greatly affect the efficiency of drug delivery. A drug delivery method and robot based on OpenMV and autonomous path planning that can improve drug delivery efficiency is needed.

[0005] Firstly, this application provides a drug delivery method based on OpenMV and autonomous path planning, applied to robots, including:

[0006] Obtain the medicine delivery map and the target room number; the medicine delivery map includes nodes and road segments connecting the nodes. The nodes include the starting point, the intersections of each road segment, and each room location. Each road segment connects only two nodes, and each room location corresponds to an independent number, which corresponds to the room number in reality.

[0007] The room location corresponding to the target room number is determined as the destination. Based on the medicine delivery map and the destination, a path planning algorithm is used to plan the path to obtain the target path. The target path represents the path from the starting point to the destination in the medicine delivery map.

[0008] When the medicine is inside the robot, a movement command is generated based on the target path. The movement command is used to instruct the robot to move along the corresponding road surface in reality according to the target path.

[0009] During the robot's movement, based on OpenMV image recognition technology, the robot identifies the guide lines preset on the road surface and adjusts its movement commands based on the guide lines so that the robot can reach the target room along the guide lines.

[0010] Secondly, this application also provides a robot, including: a drive module, a vision recognition module, a gravity detection module, and a core control module;

[0011] The core control module is electrically connected to the vision execution module. After the robot acquires the medicine delivery map and the target room number, it determines the corresponding room location on the medicine delivery map as the destination. Based on the medicine delivery map and the target room number, it uses a path planning algorithm to plan the path and obtain the target path. The medicine delivery map includes nodes and road segments connecting the nodes. Nodes include the starting point, the intersections of each road segment, and each room location. Each road segment connects only two nodes, and each room location corresponds to an independent number, which corresponds to the room number in reality. The target path represents the path from the starting point to the destination on the medicine delivery map.

[0012] The gravity detection module is electrically connected to the core control module and is used to detect whether the medicine is in the robot. If the medicine is detected to be in the robot, a start command is sent to the core control module. The start command is used to instruct the core control module to generate a travel command according to the target path and send the travel command to the drive module. The travel command is used to instruct the drive module to control the robot to travel on the corresponding road surface in reality according to the target path.

[0013] The visual recognition module uses OpenMV image recognition technology to identify the guide lines preset on the road surface during the robot's movement. Based on the recognition results, it generates path adjustment instructions and sends them to the core control module. The path adjustment instructions are used to instruct the core control module to adjust the movement instructions so that the robot can reach the target room along the guide lines.

[0014] The aforementioned medicine delivery method and robot based on OpenMV and autonomous path planning utilize a path planning algorithm to obtain a target path on the medicine delivery map based on the map and the target room number. Then, in reality, the robot delivers medicine according to the actual route corresponding to this target path. Because the medicine delivery map and the actual route are pre-set to correspond, the robot can quickly obtain the actual route to the target room. During the robot's movement, OpenMV image recognition technology is used to identify the pre-set guide lines on the actual route, enabling the robot to reach the target room along the ideal route by following the guide lines, preventing the robot from deviating from the ideal route due to its own movement errors.

[0015] By quickly obtaining the actual route to the target room and preventing the robot from deviating from the ideal route due to its own travel errors during the delivery process, the efficiency of the medicine delivery robot is improved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a drug delivery method based on OpenMV and autonomous path planning according to the present invention.

[0018] Figure 2 This is a flowchart illustrating the process of adjusting the movement command in an optional embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of the obstacle avoidance process in one optional embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram illustrating the process of obtaining the target room number in an optional embodiment of the present invention;

[0021] Figure 5 This is a schematic diagram of the structure of a robot provided by the present invention.

[0022] In the picture:

[0023] 1. Body; 2. Communication module; 3. Drive module; 4. Vision recognition module; 5. Gravity detection module; 6. Core control module. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] Before providing a detailed description of this application, the nouns and terms used in the embodiments of the application will be explained. The nouns and terms used in the embodiments of this application shall be interpreted as follows.

[0026] (1) OpenMV Image Recognition Technology: OpenMV is a programmable microcontroller camera with image processing capabilities. It mainly consists of an STM32 chip and a camera, featuring simple structure, stable performance, and low cost. OpenMV has built-in image processing algorithms, making it easy to use, and its STM32 reference materials are abundant, facilitating secondary development. For example, it can train neural networks for mask recognition, TensorFlow Lite human detection, recognize multiple complex QR codes, high-definition barcodes, and perform color recognition, circle recognition, LCD screen edge detection, AprilTag tracking, template matching, feature point detection, neural network recognition of different objects, and number recognition. Furthermore, it supports WiFi wireless data transmission and can even be used for microscopic observation of paramecia, face recognition, and eye tracking. In practical applications, users only need to install the OpenMV IDE software on their computer, connect the OpenMV hardware to the computer via USB, click the link and start running, and OpenMV will begin running the relevant code, displaying the camera image in the IDE. The IDE also includes many examples, allowing users to choose different examples to run and learn from.

[0027] (2) ToF Principle: Also known as Time of Flight, this is a distance measurement technique. Its basic principle is to calculate distance by measuring the time it takes for a light pulse to travel from emission to return. Specifically, it emits a light pulse (usually infrared light), which is reflected back after encountering an object. The sensor records the time t1 when the light pulse is emitted and the time t2 when the reflected light is received. Then, based on the speed of light c (approximately 3 × 10⁻⁶ Hz in air), the distance is calculated. 8 The distance d is calculated using m / s. The formula is as follows:

[0028] The present application will now be described in detail with reference to the accompanying drawings:

[0029] refer to Figure 1 The document presents a flowchart illustrating a drug delivery method based on OpenMV and autonomous path planning, as provided in an embodiment of this application, comprising the following steps:

[0030] S101. Obtain the medicine delivery map and the target room number; wherein, the medicine delivery map includes nodes and road segments connecting the nodes. The nodes include the starting point, the intersections of each road segment and each room location. Each road segment connects only two nodes. Each room location corresponds to an independent number, and the independent number corresponds to the room number in reality.

[0031] Specifically, the medicine delivery map is designed based on the real-world room layout and can be received externally or obtained from a storage system. The map contains numerous nodes and connecting routes, including the starting point, intersections, and room locations. Each route strictly connects two nodes. In the medicine delivery map data, each room location corresponds to a unique number, which corresponds one-to-one with the corresponding room number in reality. This correspondence can be either identical numbers or a mapping relationship.

[0032] S102. Determine the room location corresponding to the target room number as the destination, and based on the medicine delivery map and the destination, use a path planning algorithm to perform path planning to obtain the target path; the target path represents the path from the starting point to the destination in the medicine delivery map.

[0033] Specifically, first, the target room number is matched with room locations on the medicine delivery map to determine the corresponding room location as the destination. Next, a path planning algorithm is used, based on the medicine delivery map, to analyze each node and road segment in the map. Starting from the starting point, the algorithm gradually explores connecting road segments and nodes, evaluating the probability and efficiency of different paths to the destination. For each node, the algorithm considers all road segments connecting that node and selects the optimal next node based on preset evaluation criteria (such as path length, travel time, etc.). This process is repeated until a path to the destination is found. Finally, the algorithm determines a target path from the starting point to the destination and outputs it for subsequent medicine delivery operations. The path planning algorithm can be Dijkstra's algorithm, A* (heuristic) algorithm, RRT (Rapid Expanding Random Tree) algorithm, ant colony optimization, etc.

[0034] S103. When the medicine is in the robot, a travel instruction is generated according to the target path. The travel instruction is used to instruct the robot to travel on the corresponding road surface in reality according to the target path.

[0035] Specifically, once the robot detects the medicine within its casing, it analyzes the real-world locations of the target path's start and end points to determine the robot's initial and target positions. The target path is then divided into multiple continuous segments, and specific movement commands are generated for each segment. For example, if the segment is straight, the commands include the direction and distance traveled; if the segment is curved, the commands include a combination of turning angle and distance traveled. Considering the robot's motion characteristics and speed limitations, the parameters in the movement commands are adjusted to ensure stable and safe movement. For nodes and intersections in the path, the robot's decision-making commands at these locations are defined, such as selecting the correct direction of travel at intersections based on specific markers or sensor information. The generated movement commands are encoded in a robot-recognizable format, such as specific numerical sequences or code instruction sets. The movement commands are then transmitted to the robot's drive module, initiating the robot's movement on the corresponding real-world surface according to the commands.

[0036] S104. During the robot's movement, based on OpenMV image recognition technology, the robot identifies the guide lines preset on the road surface and adjusts the movement commands based on the guide lines so that the robot can reach the target room along the guide lines.

[0037] Specifically, during its movement, the robot continuously captures images of the road surface using an OpenMV camera. This image data is transmitted to the robot's core control module for real-time processing. Image recognition algorithms analyze the road surface images to locate pre-defined guide lines. Guide lines can be identified through various methods, such as color and shape features. For example, if the guide line is a line of a specific color, it can be segmented based on a color threshold to extract the guide line region. Once the guide line is identified, its position and direction in the image are calculated. The robot's positional and angular deviations relative to the guide line are determined by calculating parameters such as the guide line's centerline and slope. Based on these deviations, the robot's movement commands are adjusted. If the robot deviates significantly from the guide line, the steering force can be increased or the forward speed adjusted to bring the robot back to the guide line as quickly as possible. If the angular deviation between the robot and the guide line is large, the steering angle can be adjusted appropriately to align the robot's direction of travel with the guide line. The robot continuously monitors changes in the guide line and adjusts its movement commands accordingly to ensure it moves stably along the guide line. By combining steps S103 and S104, we can avoid the robot's delivery route from being an undesirable route due to its own travel errors during the process.

[0038] In one optional embodiment, based on the drug delivery map and the destination, a path planning algorithm is used to plan the path to obtain the target path, including the following steps:

[0039] Step 1: Record the starting point as the current point P, set its g value to 0, and create a first open list and a first closed list; where g is the actual cost, representing the actual distance from the starting point to the current node.

[0040] Step 2: Add the current point P to the first closed list.

[0041] Step 3: Find all neighboring points of the current point P according to the medicine delivery map. If a neighboring point is neither in the first open list nor in the first closed list, calculate the g value of the neighboring point, calculate the f value of the neighboring point according to the formula f = g + h, set the current point P as the parent node of the neighboring point, and put the current point P into the first open list.

[0042] Wherein, for a given node, its neighboring nodes are other nodes directly connected to the given node through line segments with Euclidean distance values; h is the heuristic cost, representing the estimated distance from the current point to the destination.

[0043] Step 4: Determine if the first open list is empty. If it is, the pathfinding fails; otherwise, continue with the following steps.

[0044] Step 5: Take the node with the smallest f value from the first open list.

[0045] Step 6: Determine whether the node with the smallest f value is the endpoint. If yes, determine the node with the smallest f value as the endpoint. Based on the parent-child relationship of each node between the starting point and the endpoint, obtain the planned path and use the planned path as the target path. If no, continue with the following steps.

[0046] Step 7: Update the current point P to the node with the smallest f value, and jump back to step 3.

[0047] Specifically, this path planning method based on g, h, and f values ​​is called the A* algorithm. It is very effective in many practical applications and can quickly find a relatively optimized path.

[0048] For step 1, the starting point is recorded as the current point P, meaning that at the beginning of path planning, we use the starting point as the node currently under consideration. Its g value is set to 0 because the actual distance from the starting point to itself is 0. A first open list and a first closed list are created. The first open list stores nodes to be checked, and the first closed list stores nodes that have already been checked.

[0049] For step 2, the current point P (i.e. the starting point) is added to the first closed list, indicating that this node has been processed and does not need to be checked again.

[0050] For step 3, find all neighboring points of the current point P according to the drug delivery map. Neighboring points are other nodes directly connected to the current node via line segments with Euclidean distance values, meaning they are directly adjacent on the map. If a neighboring point is neither in the first open list nor the first closed list, it means that the node has not yet been processed. Calculate the g value of the neighboring point, which represents the actual distance from the starting point to the current node. This distance can be calculated by summing the lengths of the line segments traversed from the starting point to the current node. Calculate the f value of the neighboring point using the formula f = g + h, where h is the heuristic cost, representing the estimated distance from the current point to the destination. The heuristic cost can be calculated using various methods, such as Euclidean distance, Manhattan distance, etc. Set the current point P as the parent node of the neighboring point; this is to allow backtracking to the starting point when determining the subsequent path. The parent node records which node the current node originated from. Finally, add the neighboring point to the first open list for later inspection.

[0051] For step 4, check if the first open list is empty. If the open list is empty, it means there are no more nodes to check, i.e., the pathfinding has failed and a path from the starting point to the destination cannot be found.

[0052] For step 5, if the first open list is not empty, take the node with the smallest f value from the open list. This node is considered to be the most promising node to reach the destination.

[0053] For step 6, determine if the node with the smallest f-value is the endpoint. If so, it means a path from the starting point to the endpoint has been found. Based on the parent-child relationships of each node between the starting and ending points, the planned path is obtained. Starting from the endpoint, by tracing back the parent node of each node all the way back to the starting point, the complete planned path can be obtained. This planned path is used as the target path for the robot's movement.

[0054] For step 7, if the node with the smallest f value is not the destination, update the current point P to the node with the smallest f value, and then jump back to step 3 to continue searching for a path.

[0055] By repeatedly performing these steps, the algorithm can gradually expand the path from the starting point until it finds a path to the destination or determines that no path exists.

[0056] refer to Figure 2 In one optional embodiment, identifying guide lines pre-set on the road surface and adjusting travel instructions based on the guide lines includes the following steps:

[0057] S201. Acquire each frame of the road surface image.

[0058] Specifically, as the robot moves, it continuously acquires images of the road surface through its onboard cameras. The cameras can acquire images at certain time intervals or be triggered based on the robot's movement.

[0059] S202. Based on OpenMV image recognition technology, determine whether the guide line deviates from the guide line based on the position of the color block of the guide line.

[0060] Specifically, OpenMV image recognition technology uses color features to identify guide lines. First, the color characteristics of the guide lines are determined, such as a specific range of RGB or HSV values. Then, each acquired image frame is analyzed, and a color segmentation algorithm is used to find color patches that match the guide line color. If the color patch is located in the center of the image or within a preset range, the robot is considered to be on the same path as the guide line; if the color patch deviates from the center or exceeds the preset range, the robot is considered to have deviated from the guide line.

[0061] S203. If the robot deviates from the guide line, the target path is corrected, and the travel command is adjusted based on the corrected target path so that the robot travels along the guide line.

[0062] Specifically, when a robot is determined to have deviated from the guide line, the target path needs to be corrected. The correction method can be determined based on the degree and direction of the deviation. If the deviation is minor, the robot can be gradually brought back to the guide line by fine-tuning the steering angle and forward speed in the travel commands. If the deviation is significant, it may be necessary to replan a local path to return to the guide line as quickly as possible.

[0063] refer to Figure 3 In an optional embodiment, this drug delivery method based on OpenMV and autonomous path planning further includes the following steps:

[0064] S301. Based on the Time-of-Flight (ToF) principle, it measures the distance between the robot and surrounding objects.

[0065] Specifically, the Time-of-Flight (ToF) sensors on the robot emit light signals of specific wavelengths, such as infrared light. When the light signal encounters surrounding objects, it is reflected back, and the sensor receives the reflected light. By calculating the round-trip time of the light signal, the distance between the robot and surrounding objects can be accurately determined.

[0066] S302. If the distance is less than the preset safety value, stop and plan obstacles to obtain an obstacle avoidance path.

[0067] Specifically, when the robot detects that the distance to surrounding objects is less than a preset safety value using the ToF sensor, it will stop immediately to avoid a collision. Then, the obstacle planning algorithm will be activated.

[0068] Obstacle planning algorithms can be based on map information, the robot's current position, and the distribution of surrounding obstacles. The locations of detected obstacles can be marked on the drug delivery map. Then, a path planning algorithm, such as the A* algorithm, is used to replan a path from the current position to the target position that avoids the obstacles—that is, an obstacle avoidance path.

[0069] S303. Generate obstacle avoidance instructions based on the obstacle avoidance path; the obstacle avoidance instructions are used to instruct the robot to avoid obstacles during its movement.

[0070] Specifically, after determining the obstacle avoidance path, the robot generates corresponding obstacle avoidance commands based on this path. These commands include parameters such as the robot's direction of movement, speed, and turning angle.

[0071] If the obstacle avoidance path is a straight section, the obstacle avoidance command can be set to proceed along the straight line at an appropriate speed. If the obstacle avoidance path includes turns, the appropriate steering angle and speed change need to be calculated based on the angle and radius of the turns. For example, gradually reduce speed before the turn, maintain a stable steering angle during the turn, and gradually accelerate back to normal speed after the turn.

[0072] After generating obstacle avoidance commands, the robot's core control module controls its movement accordingly, ensuring the robot avoids obstacles and safely reaches the target room. Simultaneously, the robot continuously monitors changes in the surrounding environment to adjust obstacle avoidance commands promptly when encountering new obstacles.

[0073] In one alternative embodiment, obstacle planning is performed to obtain an obstacle avoidance path, including the following steps:

[0074] Step 1: Construct an obstacle grid map, represented by a two-dimensional array MAP[i][j], using the robot's current position coordinates (x, y, j). s ,y s ) and target position coordinates (x e ,y e The starting and ending grid points are determined by mapping the coordinates of the starting and ending grid points to the coordinates in the obstacle grid map, thus obtaining the coordinates of the starting grid point (i). s ,j s ), Obtain the grid coordinates of the end point (i) e ,j e ), Where Δx and Δy are the side lengths of each grid cell in the obstacle grid map.

[0075] Step 2: Construct a formula for calculating the comprehensive cost F-value of graticules in the barrier-free area of ​​the obstacle grid map. The formula for calculating the comprehensive cost F-value is as follows: The grid coordinates of the grid cells in the barrier-free area are (i,j). Where g(i,j)=|ii s |+|jj s | represents the cost from the starting grid to a grid in the barrier-free area; This represents the cost from a grid cell in the accessible area to the endpoint grid cell; ω× d(i,j) represents the obstacle avoidance cost of a grid cell in the barrier-free area, d(i,j) is the distance from a grid cell in the barrier-free area to the nearest obstacle grid cell, and ω is the weighting coefficient.

[0076] Step 3: Based on A with obstacle avoidance weights * The algorithm performs obstacle avoidance planning and progressively plans obstacle avoidance paths based on the comprehensive cost F value, including the following steps:

[0077] 1) Create a second open list and a second closed list. Place the starting point grid into the second open list, according to the formula. and the coordinates corresponding to the starting grid (i s ,j s Set the F value of the starting point grid to f(i). s ,j s );

[0078] 2) Take the raster with the smallest F value from the second open list as the current raster, remove the current raster from the second open list, and put it into the second closed list;

[0079] 3) For all adjacent unobstructed grids of the current grid, if the adjacent grid is neither in the second open list nor the second closed list, calculate the F value of the adjacent grid based on the grid coordinates of the adjacent grid, set the current grid as the parent node of the adjacent grid, and put the current grid into the second open list; if the adjacent grid is already in the second open list, determine whether the g(i,j) value of reaching the adjacent grid through the current path is less than the original g(i,j) value of the adjacent grid. If so, update the g(i,j) value and F value of the adjacent grid, and update the parent node of the adjacent grid to the current grid.

[0080] 4) Determine if the second open list is empty. If it is empty, the obstacle avoidance planning fails. If it is not empty, continue to execute the above steps. When the grid cell taken from the second open list is the end point grid cell, the obstacle avoidance planning is successful. Based on the parent node relationship of each grid cell, the planned path is obtained and used as the obstacle avoidance path.

[0081] Specifically, firstly, the robot's operating space is divided into uniform grids. If a grid contains an obstacle, it is marked as an obstacle grid; otherwise, it is an obstacle-free grid. Assume the entire space is divided into M×N grids, and a two-dimensional array MAP[i][j] represents the grid map, where i = 1, 2, ..., M, j = 1, 2, ..., N. If MAP[i][j] = 1, it indicates that the grid contains an obstacle; if MAP[i][j] = 0, it indicates that the grid is obstacle-free. The starting and ending grids can be determined by mapping the robot's current position and target position to coordinates in the grid map. Assume the starting point coordinates are (x... s ,y s The coordinates of the endpoint are (x, y). e ,y e If the starting grid is (i), then the starting grid is (i). s ,j s ),in, The end point grid is (i e ,j e ),in, Here, Δx and Δy are the side lengths of each grid cell.

[0082] Then, the formula for calculating the cost value is constructed. The cost value includes: the cost from the starting grid to the barrier-free area, the cost from the barrier-free area to the ending grid, and the obstacle avoidance cost of the barrier-free area.

[0083] The cost from the starting grid to the unobstructed area can be represented by the actual distance from the starting grid to the current grid, similar to the g value in the traditional A* algorithm. It can be calculated using Manhattan distance or Euclidean distance; here, we take Manhattan distance as an example. Let the current grid coordinates be (i,j), then the cost from the starting grid to this grid is g(i,j)=|ii s |+|jj s |

[0084] The cost from the accessible area to the endpoint grid is similar to the heuristic cost h in the traditional A* algorithm; it's an estimated distance from the current grid to the endpoint. It can also be calculated using Manhattan distance or Euclidean distance. Taking Euclidean distance as an example, the cost h(i,j) is...

[0085] The obstacle avoidance cost in an accessible area mainly considers factors such as the proximity to obstacles within the accessible area. It can be determined based on the distance between the current grid cell and the nearest obstacle grid cell. Let d(i,j) be the distance from the current grid cell to the nearest obstacle grid cell, then the obstacle avoidance cost can be expressed as... Where ω is a weighting coefficient used to adjust the proportion of obstacle avoidance cost in the total cost.

[0086] Combining the above-mentioned cost values ​​yields the comprehensive cost value F.

[0087] Finally, obstacle avoidance planning is performed based on the A* algorithm with obstacle avoidance weights, including the following steps:

[0088] 1) Add the starting point grid to the open list and set its F value to f(i s ,j s ),

[0089] 2) Take the grid cell with the smallest F value from the open list as the current grid cell, remove it from the open list, and put it into the closed list.

[0090] 3) Perform the following processing on all adjacent unobstructed grid cells of the current grid:

[0091] a. If an adjacent grid cell is neither in the open list nor the closed list, calculate its g value, h value, and obstacle avoidance cost to obtain the F value, set the current grid cell as its parent node, and then add it to the open list.

[0092] b. If an adjacent raster is already in the open list, check if the g-value of the adjacent raster reached via the current path is less than its original g-value. If so, update the g-value, F-value, and parent node of the adjacent raster.

[0093] 4) Check if the open list is empty:

[0094] a. If so, it means that all possible path points have been found before reaching the end point, and obstacle avoidance planning has failed.

[0095] b. If not, continue with step 2).

[0096] 5) When the grid cell retrieved from the open list is the end grid cell, obstacle avoidance planning is successful. At this point, the planned path can be obtained by tracing back from the end grid cell to the starting grid cell and considering the parent node relationships of each grid cell. This path can then be used as the obstacle avoidance path.

[0097] In one optional embodiment, obtaining the target room number includes the following steps:

[0098] Acquire an image to be recognized, which contains handwritten characters;

[0099] Based on OpenMV image recognition technology, handwritten characters in the image to be recognized are identified to obtain the number of the target room.

[0100] Specifically, images containing handwritten characters related to the target room number are acquired from relevant devices (such as robot cameras). The images undergo preprocessing, such as grayscale conversion and filtering. Using OpenMV's character recognition algorithm, characters can be identified through template matching or feature extraction combined with a classifier. The identified characters are then combined to obtain the target room number.

[0101] refer to Figure 4 In one optional embodiment, based on OpenMV image recognition technology, handwritten characters in the image to be recognized are identified to obtain the target room number, including the following steps:

[0102] S401. Using the OpenMV shape detection algorithm, select regions of specific shapes from the image to be identified to obtain the target region.

[0103] Specifically, it's necessary to determine the target shape features, which can be related to the region where the handwritten character is located, such as common shapes like rectangles and circles. The algorithm iterates through every pixel of the image to be recognized, judging whether it matches the target shape based on the geometric definition of the shape and related parameters (such as side length, angle, curvature, etc.). Once a region matching the shape features is detected, it is marked as the target region. This process can filter out most areas in the image that are unrelated to the handwritten character, thereby reducing the complexity of subsequent processing and improving the accuracy of character recognition.

[0104] S402. Apply a character segmentation algorithm to the target area to separate handwritten characters, and use a character recognition algorithm library to recognize each separated handwritten character to obtain multiple recognition results.

[0105] Specifically, the purpose of character segmentation algorithms is to separate connected handwritten characters for subsequent individual recognition. Commonly used character segmentation algorithms include projection-based methods and connected component-based methods.

[0106] For projection-based methods, the projections of the target region in the horizontal and vertical directions are calculated. The projection values ​​reflect the distribution of character strokes in the corresponding directions. By analyzing the changes in projection values, the gaps between characters can be found, thus separating the characters.

[0107] For methods based on connected regions, the target region is regarded as a connected graph composed of pixels. Different connected regions are divided according to the connectivity between pixels (such as 4-connectivity or 8-connectivity). Each connected region may correspond to a handwritten character.

[0108] For each isolated handwritten character, a character recognition algorithm library is used for identification. This library contains various algorithms and models for character recognition. These algorithms may be based on feature extraction and classification methods, such as extracting stroke features, structural features, etc., and comparing these features with predefined character models to determine the character's category.

[0109] Alternatively, deep learning methods can be used, employing pre-trained neural network models to recognize characters. By learning from a large number of character samples, neural network models can automatically extract character features and perform accurate classification.

[0110] S403. Combine multiple recognition results to form a number, which is used as the number of the target room.

[0111] Specifically, if the recognition result is a number, it can be directly combined to form a number. If some uncertain characters or misrecognitions occur during the recognition process, corrections can be made according to certain rules. For example, if the recognition result contains characters that do not conform to the common format of room numbers, they can be inferred and replaced based on context or other auxiliary information. The final number formed is the target room number.

[0112] In practice, when the robot needs to obtain the target room number from the outside, but the terminal malfunctions or it is inconvenient to send the target room number to the robot using the terminal, having the robot recognize handwritten characters to obtain the target room number is also a solution, thus not affecting the efficiency of the robot's medicine delivery.

[0113] In an optional embodiment, the method further includes the following steps:

[0114] Upon reaching the target room, if the medicine is not in the robot, return along the same path in reality.

[0115] In an embodiment of this application, a drug delivery method based on OpenMV and autonomous path planning is provided, applied to a robot. The method includes: obtaining a drug delivery map and the number of a target room; determining the room location corresponding to the target room number as the destination, and performing path planning based on the drug delivery map and the destination using a path planning algorithm to obtain a target path; when the drug is in the robot, generating a travel command according to the target path; during the robot's travel, identifying a guide line preset on the road surface based on OpenMV image recognition technology, and adjusting the travel command based on the guide line so that the robot reaches the target room along the guide line.

[0116] This method is based on a path planning algorithm. It obtains a target path on the delivery map based on the delivery map and the target room number. Then, in reality, it delivers the medicine according to the actual route corresponding to this target path. Because the delivery map and the real-world route are pre-set to correspond, the robot can quickly obtain the actual route to the target room. During the robot's movement, OpenMV image recognition technology is used to identify pre-set guide lines on the actual route, ensuring the robot reaches the target room along the ideal route by following the guide lines. This prevents the robot from deviating from the ideal route due to its own movement errors, thereby improving the efficiency of robot medicine delivery.

[0117] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0118] Based on the same inventive concept, this application also provides a robot for implementing the aforementioned drug delivery method based on OpenMV and autonomous path planning. The robot includes: a drive module, a vision recognition module, a gravity detection module, and a core control module.

[0119] The core control module is electrically connected to the vision execution module. After the robot acquires the medicine delivery map and the target room number, it determines the corresponding room location on the medicine delivery map as the destination. Based on the medicine delivery map and the target room number, it uses a path planning algorithm to plan the path and obtain the target path. The medicine delivery map includes nodes and road segments connecting the nodes. Nodes include the starting point, the intersections of each road segment, and each room location. Each road segment connects only two nodes, and each room location corresponds to an independent number, which corresponds to the room number in reality. The target path represents the path from the starting point to the destination on the medicine delivery map.

[0120] The gravity detection module is electrically connected to the core control module and is used to detect whether the medicine is in the robot. If the medicine is detected to be in the robot, a start command is sent to the core control module. The start command is used to instruct the core control module to generate a travel command according to the target path and send the travel command to the drive module. The travel command is used to instruct the drive module to control the robot to travel on the corresponding road surface in reality according to the target path.

[0121] The visual recognition module uses OpenMV image recognition technology to identify the guide lines preset on the road surface during the robot's movement. Based on the recognition results, it generates path adjustment instructions and sends them to the core control module. The path adjustment instructions are used to instruct the core control module to adjust the movement instructions so that the robot can reach the target room along the guide lines.

[0122] Specifically, the robot can be an exoskeleton robot, a drone, a wheeled robot, and so on. The following explanation uses a wheeled robot as an example to illustrate this application.

[0123] refer to Figure 5 The wheeled robot may include: a body 1, a communication module 2, a drive module 3, a vision recognition module 4, a gravity detection module 5, and a core control module 6.

[0124] The body can be made of metal or plastic, and each module has a pre-set mounting position.

[0125] The drive module 3 can be installed at the bottom of the housing 1, and the drive module 3 may include:

[0126] 1) Motor: It can be a DC brushless encoder motor to drive the robot's wheels, enabling the robot to move on the ground, and measure the motor speed and wheel rotation angle, feeding back to the core control module 6 to achieve precise control of the robot's motion state.

[0127] 2) Wheels: These can be ordinary rubber wheels or tires made of other materials.

[0128] Communication module 2 may include:

[0129] 1) Wireless communication chips: such as Wi-Fi chips (e.g., ESP8266, ESP32, etc.) or Bluetooth chips. Wi-Fi chips can achieve long-distance, high-speed data transmission, suitable for communication with terminals within a local area network; Bluetooth chips are suitable for short-range, low-power connections, such as pairing with nearby handheld devices.

[0130] 2) Communication interfaces: including UART (Universal Asynchronous Receiver / Transmitter), SPI (Serial Peripheral Interface), etc., used for data transmission with the core control module 6.

[0131] The communication module 2 can be placed at a high position inside the robot to avoid signal interference from other metal parts or electronic devices inside the robot.

[0132] The visual recognition module 4 (using OpenMV image recognition technology) may include:

[0133] 1) Camera: This can be a small CMOS (Complementary Metal-Oxide-Semiconductor) camera with suitable resolution to capture images in front of the robot. For example, some OpenMV modules are equipped with cameras with a resolution of 320×240 pixels or higher to meet basic needs such as identifying guide lines.

[0134] 2) Image sensor: Converts optical images into digital signals to provide data for subsequent image recognition.

[0135] 3) Microcontroller: Runs OpenMV firmware, processes image data, and performs image recognition algorithm calculations. This microcontroller typically has a certain amount of memory and processing power to store image data and execute the recognition program.

[0136] The visual recognition module 4 is typically mounted high on the front of the robot to provide a good field of view and facilitate the identification of guide lines on the road ahead. The camera lens needs to be directly facing the robot's direction of travel, and care must be taken to avoid obstructing the lens.

[0137] Gravity detection module 5 may include:

[0138] 1) Pressure sensor: This can be a strain gauge pressure sensor or a capacitive pressure sensor. A strain gauge pressure sensor detects pressure changes by measuring the strain of a material under pressure; a capacitive pressure sensor senses pressure by utilizing changes in capacitance. When a medicine is placed on the robot, the pressure sensor will detect the pressure change.

[0139] 2) Signal conditioning circuit: amplifies and filters the weak signal output by the pressure sensor so that it can be accurately identified by the core control module 6.

[0140] The gravity detection module 5 can be installed at the bottom of the area where the robot places medicines.

[0141] The core control module 6 can use the STM32F103C8T6 chip, a microcontroller based on the ARM Cortex-M3 architecture. This chip has 256KB of onboard FLASH memory, which can be used to store program code and some fixed parameters, such as map data and A* algorithm code. It has a 72MHz CPU clock frequency, enabling it to quickly process various control tasks. It features CAN bus functionality, allowing communication with other CAN bus-enabled devices, enhancing the system's scalability and communication capabilities. In addition, it has five USART serial ports and two I2C buses for connecting various external devices.

[0142] The core control module 6 can be placed at the center of the robot or near the main functional modules. This reduces signal transmission delay and facilitates wiring, enabling it to be effectively connected to other modules.

[0143] In one alternative embodiment, the robot further includes a ToF laser ranging module;

[0144] The ToF laser ranging module is electrically connected to the core control module and is used to measure the distance between the robot and surrounding objects, and send the measured distance data to the core control module.

[0145] The core control module is also used to process distance data. If the distance data is less than a preset safety value, it sends a stop command to the drive module and an obstacle avoidance planning command to the core control module. The stop command is used to instruct the drive module to control the robot to stop moving.

[0146] The core control module is also used to: respond to the received obstacle avoidance planning instructions, perform obstacle planning, obtain the obstacle avoidance path, generate obstacle avoidance instructions based on the obstacle avoidance path, and send the obstacle avoidance instructions to the drive module; the obstacle avoidance instructions are used to instruct the drive module to avoid obstacles.

[0147] Specifically, a ToF laser ranging module may include:

[0148] 1) Laser emitter: emits a laser beam of a specific wavelength. Semiconductor lasers can be used, which have advantages such as small size, low power consumption, and high reliability.

[0149] 2) Photodetector: Receives laser signals reflected from objects. It can be an avalanche photodiode (APD) or a photomultiplier tube (PMT), etc., capable of converting weak light signals into electrical signals.

[0150] 3) Time measurement circuit: Accurately measures the time interval from laser emission to reception. High-precision clock chips and timing circuits can be used to achieve nanosecond-level time measurement accuracy.

[0151] 4) Signal processing circuit: This circuit amplifies, filters, and digitizes the electrical signal output from the photodetector to extract valid distance information. Simultaneously, it communicates with the core control module to transmit the measured distance data.

[0152] ToF laser ranging modules can be installed on the front or top of the robot to provide the widest possible coverage of the robot's surroundings. The installation location must take into account the laser's emission and reception angles to avoid obstruction by the robot's own structure.

[0153] In an optional embodiment, the robot's visual recognition module 4 is further configured to: acquire an image to be recognized, the image containing handwritten characters; recognize the handwritten characters in the image to be recognized based on OpenMV image recognition technology, obtain the target room number, and send the target room number to the core control module.

[0154] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A drug delivery method based on OpenMV and autonomous path planning, applied to robots, characterized in that, The method includes: Obtain the medicine delivery map and the target room number; wherein, the medicine delivery map includes nodes and road segments connecting the nodes, the nodes include a starting point, various road segment intersections and various room locations, each road segment connects only two nodes, each room location corresponds to an independent number, and the independent number corresponds to the room number corresponding to the room location in reality; The location of the room corresponding to the number of the target room is determined as the destination. Based on the medicine delivery map and the destination, a path planning algorithm is used to plan the path to obtain the target path. The target path represents the path from the starting point to the destination in the medicine delivery map. When the medicine is inside the robot, a travel instruction is generated based on the target path. The travel instruction is used to instruct the robot to travel on the corresponding road surface in reality according to the target path. During the robot's movement, based on OpenMV image recognition technology, a guide line preset on the road surface is identified, and the movement command is adjusted based on the guide line so that the robot can reach the target room along the guide line.

2. The method according to claim 1, characterized in that, The step of using the path planning algorithm based on the drug delivery map and the destination to plan the target path includes: Step 1: Record the starting point as the current point P, set its g value to 0, and create a first open list and a first closed list; where g is the actual cost, representing the actual distance from the starting point to the current node; Step 2: Add the current point P to the first closed list; Step 3: Find all the neighboring points of the current point P according to the medicine delivery map. If a neighboring point is neither in the first open list nor in the first closed list, calculate the g value of the neighboring point, calculate the f value of the neighboring point according to the formula f = g + h, set the current point P as the parent node of the neighboring point, and put the current point P into the first open list. Wherein, for a given node, its neighboring points are other nodes that are directly connected to the given node through line segments with Euclidean distance values; h is the heuristic cost, representing the estimated distance from the current point to the destination; Step 4: Determine if the first open list is empty. If it is, the pathfinding fails; otherwise, continue with the following steps. Step 5: Take the node with the smallest f value from the first open list; Step 6: Determine whether the node with the smallest f value is the endpoint. If yes, determine the node with the smallest f value as the endpoint. Based on the parent-child relationship of each node between the starting point and the endpoint, obtain the planned path and use the planned path as the target path. If no, continue with the following steps. Step 7: Update the current point P to the node with the smallest f value, and jump back to step 3.

3. The method according to claim 1, characterized in that, The process of identifying guide lines pre-set on the road surface and adjusting the travel command based on the guide lines includes: Acquire each frame of the road surface image; Based on OpenMV image recognition technology, it is determined whether the image deviates from the guide line according to the position of the color block of the guide line. If the robot deviates from the guide line, the target path is corrected, and the travel command is adjusted based on the corrected target path so that the robot travels along the guide line.

4. The method according to claim 1, characterized in that, The method further includes: Based on the Time-of-Flight (ToF) principle, the distance between the robot and surrounding objects is measured; If the distance is less than the preset safety value, stop and perform obstacle planning to obtain an obstacle avoidance path; Based on the obstacle avoidance path, an obstacle avoidance command is generated; the obstacle avoidance command is used to instruct the robot to avoid obstacles during its movement.

5. The method according to claim 4, characterized in that, The obstacle planning process to obtain obstacle avoidance paths includes: Step 1: Construct an obstacle grid map, represented by a two-dimensional array MAP[i][j], using the robot's current position coordinates (x, y, j). s ,y s ) and target position coordinates (x e ,y e The starting and ending grid points are determined by mapping the coordinates of the starting grid point to the coordinates in the obstacle grid map, thus obtaining the coordinates of the starting grid point (i). s ,j s ), Obtain the grid coordinates of the end point (i) e ,j e ), Where Δx and Δy are the side lengths of each grid cell in the obstacle grid map; Step 2: Construct a formula for calculating the comprehensive cost F value of grid cells in the barrier-free area of ​​the barrier grid map. The formula for calculating the comprehensive cost F value is as follows: Wherein, the grid coordinates of the grid in the barrier-free area are (i,j); Where g(i,j)=|ii s |+|jj s | represents the cost from the starting grid to a grid in the barrier-free area; This represents the cost from a grid cell in the accessible area to the endpoint grid cell; d(i,j) represents the obstacle avoidance cost of the grid in the barrier-free area, d(i,j) is the distance from the grid in the barrier-free area to the nearest obstacle grid, and ω is the weighting coefficient; Step 3: Based on A with obstacle avoidance weights * The algorithm performs obstacle avoidance planning and progressively plans obstacle avoidance paths based on the comprehensive cost F value, including: Create a second open list and a second closed list, and put the starting point grid into the second open list according to the formula. and the coordinates (i) corresponding to the starting point grid s ,j s The F value of the starting point grid is set to f(j). s ,j s ); Take the grid cell with the smallest F value from the second open list as the current grid cell, remove the current grid cell from the second open list, and put it into the second closed list; For all adjacent unobstructed grids of the current grid, if the adjacent grid is neither in the second open list nor in the second closed list, then calculate the F value of the adjacent grid based on its grid coordinates, set the current grid as the parent node of the adjacent grid, and add the current grid to the second open list; if the adjacent grid is already in the second open list, then determine whether the g(i,j) value of reaching the adjacent grid through the current path is less than the original g(i,j) value of the adjacent grid. If so, update the g(i,j) value and F value of the adjacent grid, and update the parent node of the adjacent grid to the current grid. Determine if the second open list is empty. If it is empty, the obstacle avoidance planning fails. If it is not empty, continue to execute the above steps. When the grid cell taken from the second open list is the end point grid cell, the obstacle avoidance planning is successful. Based on the parent node relationship of each grid cell, the planned path is obtained and used as the obstacle avoidance path.

6. The method according to claim 1, characterized in that, Obtaining the target room number includes: Acquire an image to be recognized, wherein the image contains handwritten characters; Based on OpenMV image recognition technology, the handwritten characters in the image to be recognized are identified to obtain the number of the target room.

7. The method according to claim 6, characterized in that, The method of identifying the handwritten characters in the image to be identified based on OpenMV image recognition technology to obtain the number of the target room includes: The OpenMV shape detection algorithm is used to filter regions of specific shapes from the image to be identified, thereby obtaining the target region; The handwritten characters are separated by applying a character segmentation algorithm to the target region, and each of the separated handwritten characters is recognized using a character recognition algorithm library to obtain multiple recognition results; The multiple identification results are combined to form a number, which serves as the number of the target room.

8. A robot, characterized in that, include: The module consists of a drive module, a vision recognition module, a gravity detection module, and a core control module. The core control module is electrically connected to the vision execution module. After the robot acquires the medicine delivery map and the target room number, it determines the corresponding room location in the medicine delivery map as the destination. Based on the medicine delivery map and the target room number, it uses a path planning algorithm to plan a path to obtain the target path. The medicine delivery map includes nodes and road segments connecting the nodes. Each node includes a starting point, road segment intersections, and each room location. Each road segment connects only two nodes, and each room location corresponds to an independent number, which in turn corresponds to the room number in reality. The target path represents the path from the starting point to the destination in the medicine delivery map. The gravity detection module is electrically connected to the core control module and is used to detect whether the medicine is in the robot. If the medicine is detected to be in the robot, a start command is sent to the core control module. The start command is used to instruct the core control module to generate a travel command according to the target path and send the travel command to the drive module. The travel command is used to instruct the drive module to control the robot to travel on the corresponding road surface in reality according to the target path. The visual recognition module uses OpenMV image recognition technology to identify guide lines preset on the road surface during the robot's movement. Based on the recognition results, it generates path adjustment instructions and sends the path adjustment instructions to the core control module. The path adjustment instructions are used to instruct the core control module to adjust the movement instructions so that the robot can reach the target room along the guide lines.

9. The robot according to claim 8, characterized in that, The robot also includes a ToF laser ranging module; The ToF laser ranging module is electrically connected to the core control module and is used to measure the distance between the robot and surrounding objects, and send the measured distance data to the core control module. The core control module is also used to process the distance data. If the distance data is less than a preset safety value, a stop command is sent to the drive module and an obstacle avoidance planning command is sent to the core control module. The stop command is used to instruct the drive module to control the robot to stop moving. The core control module is also used to: respond to the received obstacle avoidance planning instruction, perform obstacle planning, obtain an obstacle avoidance path, generate an obstacle avoidance instruction based on the obstacle avoidance path, and send the obstacle avoidance instruction to the drive module; the obstacle avoidance instruction is used to instruct the drive module to avoid obstacles.

10. The robot according to claim 8, characterized in that, The visual recognition module is further configured to: acquire an image to be recognized, the image containing handwritten characters; based on OpenMV image recognition technology, recognize the handwritten characters in the image to be recognized, obtain the number of the target room, and send the number of the target room to the core control module.

Images