Robot path identification method and system

Abstract

The invention discloses a robot path recognition method and system, and belongs to the technical field of robots, and the system comprises an acquisition module, a recognition module, a coffee cup shape recognition result acquisition module, a coffee cup position recognition result acquisition module, a determination module, a position posture acquisition module and a path recognition module. The method comprises the following steps: acquiring an environment image including a coffee making area, and performing multi-stage identification and processing on the image, including determining candidate block images of suspected coffee cups, obtaining coffee cup structure standard parameters for shape matching, and identifying a coffee making area range for position judgment. And finally, fusing the shape and position identification results to determine a target coffee cup so as to confirm the moving path of the coffee cup. The method can effectively solve the problem that in the prior art, due to the fact that coffee powder is attached to a camera lens, an image is blurred, coffee cup recognition is inaccurate, and path planning deviation is caused.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a robot path recognition method and system. In modern coffee making environments, automated coffee making robot systems are widely deployed to improve efficiency. Such systems usually rely on visual recognition modules integrated on the robot or above the coffee work area to identify the position, type of coffee cup and obstacles on the workbench, and then plan the motion trajectory of the robot's gripper from the coffee bean bin, the coffee grinder to the brewing head and other key operation points.

[0002] However, in actual operation, in the working environment of the coffee making robot, the coffee grinder will inevitably produce extremely fine coffee powder during the continuous grinding of coffee beans. These powders will adhere to the camera lens surface used by the robot visual recognition system with the flow of air, causing slight overall blurring of the original image obtained by the camera, and thus affecting the identification of the position of the coffee cup, and ultimately affecting the coffee making of the automated coffee making robot.

[0003] Therefore, a solution is needed to solve the above problems. SUMMARY

[0004] The purpose of the present application is to provide a robot path recognition method and system, which aims to solve the technical problems mentioned in the background art, i.e. the decline in image quality due to the attachment of coffee powder on the camera lens surface for obtaining images, which in turn affects the accuracy of coffee cup recognition and the deviation of path planning, ultimately leading to the failure of the robot application.

[0005] In order to achieve the above purpose, the technical solution of the present application is: As an aspect of the present application, a robot path recognition method is provided, which is used to confirm the moving path of a coffee cup when a robot is making coffee, comprising: collecting an environment image including a coffee making area; identifying the environment image and determining at least one candidate block image of a suspected coffee cup from the environment image; obtaining the structural standard parameters of the coffee cup, matching the structural standard parameters of the coffee cup with each candidate block image of the suspected coffee cup and obtaining a coffee cup shape recognition result; identifying the range of the coffee making area from the environment image and determining whether each candidate block image of the suspected coffee cup is within the range to obtain a coffee cup position recognition result; fusing the coffee cup shape recognition result and the coffee cup position recognition result to determine a target coffee cup; Acquire position information and posture information of the target coffee cup in the coffee making area; Acquire position information and posture information of the target coffee cup in the coffee making area;

[0006] Further, the step of identifying the environment image and determining at least one candidate block image of the suspected coffee cup from the environment image specifically comprises: Perform gray processing on the environment image to obtain a processed environment image; Perform binary processing on the processed environment image according to a preset threshold judgment reference to obtain a binary image corresponding to the processed environment image; Perform grid segmentation on the binary image corresponding to the processed environment image to obtain a plurality of segmented binary block images; Screen the plurality of binary block images according to a preset basic shape of the coffee cup to obtain a plurality of screened binary block images; Perform noise reduction on the plurality of screened binary block images to obtain a candidate block image of the suspected coffee cup.

[0007] Further, the step of screening the plurality of binary block images according to the preset basic shape of the coffee cup to obtain a plurality of screened binary block images specifically comprises: Obtain a basic shape contour of the coffee cup by acquiring prior knowledge of the basic shape of the coffee cup; Match the basic shape contour of the coffee cup with the plurality of segmented binary block images to confirm whether there is an image area matching the basic shape contour of the coffee cup in the binary block image, and obtain a confirmed binary block image.

[0008] Further, the step of matching the structural standard parameters of the coffee cup with each candidate block image of the suspected coffee cup and obtaining a coffee cup shape recognition result specifically comprises: Identify the candidate block image of the suspected coffee cup and extract a graphic contour from the candidate block image of the suspected coffee cup; Match the extracted graphic contour with the standard parameters of the coffee cup to screen a block image containing the coffee cup from the plurality of candidate block images of the suspected coffee cup, and take the screened block image containing the coffee cup as the coffee cup shape recognition result.

[0009] Further, the standard parameters of the coffee cup include the shape of the cup mouth, the length-width ratio of the cup mouth, the shape of the cup body, the length-width ratio of the cup body, the shape of the cup handle, and the length-width ratio of the cup handle.

[0010] Further, the step of matching the extracted graphic contour with the standard parameters of the coffee cup to screen out the block image containing the coffee cup from the plurality of candidate block images of the suspected coffee cup, and taking the screened block image containing the coffee cup as the coffee cup shape recognition result, specifically comprises: adopting a shape detection algorithm model to detect and obtain a first feature contour conforming to a preset circular or elliptical shape from the graphic contour, and marking the first feature contour as a potential cup mouth contour; taking the position of the potential cup mouth contour as a detection reference position, detecting and obtaining a second feature contour conforming to a preset U shape from the graphic contour below the position of the potential cup mouth contour, and marking the second feature contour as a potential cup body contour; taking the position of the potential cup body contour as a detection reference position, detecting and obtaining a third feature contour conforming to a preset arc shape from the graphic contour around the position of the potential cup body contour, and marking the third feature contour as a potential cup handle contour; combining the potential cup handle contour, the potential cup body contour corresponding to the potential cup handle contour, and the image of the potential cup handle contour, and matching the standard parameters of the coffee cup to screen out the block image containing the coffee cup, and taking the screened block image containing the coffee cup as the coffee cup shape recognition result.

[0011] Further, the step of identifying the range of the coffee making area from the environment image, and determining whether each candidate block image of the suspected coffee cup is within the range to obtain the coffee cup position recognition result, specifically comprises: determining the positions of each boundary point of the coffee making area in the environment image; determining the area range of the coffee making area in the environment image according to the positions of each boundary point in the environment image; determining whether the proportion of the overlapping area between the candidate block image of the suspected coffee cup and the area range of the coffee making area in the environment image satisfies the shape proportion threshold value; determining the candidate block image of the suspected coffee cup satisfying the shape proportion threshold value requirement, and taking the candidate block image of the suspected coffee cup as the coffee cup position recognition result.

[0012] Further, the step of fusing the coffee cup shape recognition result and the coffee cup position recognition result to determine the target coffee cup, specifically comprises: matching the structural standard parameters of the coffee cup with each candidate block image of the suspected coffee cup, and giving a first confidence score to each candidate block image of the suspected coffee cup completing the matching; The scope of the coffee-making area is identified from the environmental image, and it is determined whether each candidate block image of a suspected coffee cup is within this scope. The proportion of each candidate block image of a suspected coffee cup within the scope of the coffee-making area is given a second confidence score. The shape judgment score is calculated by combining the first confidence score corresponding to each candidate block image of a suspected coffee cup that has been matched and the second confidence score corresponding to the proportion of each candidate block image of a suspected coffee cup in the coffee making area. The target coffee cup is determined by judging its shape and score.

[0013] Furthermore, acquiring the position and orientation information of the target coffee cup within the coffee-making area specifically includes: Key point detection is performed on the environmental image to determine the location of each key point contained in the target coffee cup in the environmental image; Based on the positions of each key point in the environmental image, the position and orientation of the target coffee cup in the environmental image are determined.

[0014] This application discloses a robot path recognition method that acquires environmental images, including the coffee-making area, and performs multi-stage recognition and processing on the images. This includes identifying candidate image blocks resembling a coffee cup, obtaining standard structural parameters of the coffee cup for shape matching, identifying the coffee-making area for location determination, and finally fusing the shape and location recognition results to identify the target coffee cup, thereby confirming the coffee cup's movement path. This method effectively solves the problem in existing technologies where coffee powder adhering to the camera lens causes image blurring, leading to inaccurate coffee cup recognition and path planning deviations.

[0015] As a second aspect of this application, a robot path recognition system is provided, comprising: The acquisition module is used to acquire environmental images, including the coffee-making area. The recognition module is used to recognize the environmental image and determine at least one candidate block image of a suspected coffee cup from the environmental image; A coffee cup shape recognition result acquisition module is used to acquire the structural standard parameters of a coffee cup, match the structural standard parameters of the coffee cup with candidate block images of each suspected coffee cup, and obtain the coffee cup shape recognition result. The coffee cup location recognition result acquisition module is used to identify the range of the coffee making area from the environmental image and determine whether each candidate block image of a suspected coffee cup is within the range, so as to obtain the coffee cup location recognition result. The determining module is used to fuse the coffee cup shape recognition result and the coffee cup position recognition result to determine the target coffee cup; A position and attitude acquisition module is used to acquire the position and attitude information of the target coffee cup within the coffee making area. The path recognition module is used to respond to the operation command of the coffee cup and obtain the position information of the robot, and to identify the movement path of the coffee cup based on the target coffee cup in the coffee making area and the position and posture information of the target coffee cup.

[0016] This application discloses a robot path recognition system, which includes an acquisition module, a recognition module, a coffee cup shape recognition result acquisition module, a coffee cup position recognition result acquisition module, a determination module, a position and pose acquisition module, and a path recognition module. The system acquires environmental images, including the coffee-making area, and performs multi-stage recognition and processing on these images. This includes identifying candidate regions that resemble coffee cups, acquiring standard structural parameters of the coffee cup for shape matching, identifying the coffee-making area for position determination, and finally fusing the shape and position recognition results to determine the target coffee cup, thereby confirming the coffee cup's movement path. This method effectively solves the problem in existing technologies where coffee powder adhering to the camera lens causes image blurring, leading to inaccurate coffee cup recognition and path planning deviations.

[0017] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a robot path recognition method in this embodiment; Figure 2 This is a flowchart illustrating step S2 in a robot path recognition method according to this embodiment; Figure 3 This is a flowchart illustrating step S3 in a robot path recognition method according to this embodiment; Figure 4 This is a flowchart illustrating step S4 in a robot path recognition method according to this embodiment; Figure 5 This is a system structure block diagram of a robot path recognition system in this embodiment.

[0019] Figure label: 101. Acquisition Module; 102. Recognition Module; 103. Coffee Cup Shape Recognition Result Acquisition Module; 104. Coffee Cup Position Recognition Result Acquisition Module; 105. Determination Module; 106. Position and Attitude Acquisition Module; 107. Path Recognition Module; 100. Robot Path Recognition System. Detailed Implementation

[0020] To better illustrate the present invention, the invention will now be described in further detail with reference to the accompanying drawings.

[0021] It should be understood that, in order to make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0022] In the current technology, during the continuous grinding of coffee beans, the coffee grinder inevitably produces extremely fine coffee powder in the working environment of coffee-making robots. This powder adheres to the surface of the camera lens used by the robot's vision recognition system as it is carried by airflow, causing a slight overall blurring of the original image captured by the camera. Consequently, the robot cannot identify the position of the coffee cup, thus affecting the coffee-making process of the automated coffee-making robot.

[0023] Therefore, the technical problem that this application actually solves is how to identify a coffee cup with coffee powder adhering to it using a visual recognition system.

[0024] The following is an example illustration. In this example: Firstly, such as Figure 1 As shown, a robot path recognition method is provided, which is used by a robot to determine the movement path of a coffee cup when making coffee, including: Step S1: Acquire environmental images, including the coffee-making area; Step S2: Identify the environmental image and determine at least one candidate block image of a suspected coffee cup from the environmental image; Step S3: Obtain the structural standard parameters of the coffee cup, match the structural standard parameters of the coffee cup with the candidate block images of each suspected coffee cup, and obtain the coffee cup shape recognition result. Step S4: Identify the range of the coffee making area from the environmental image and determine whether each candidate block image of a suspected coffee cup is within this range, so as to obtain the coffee cup location identification result. Step S5: Merge the coffee cup shape recognition result with the coffee cup position recognition result to determine the target coffee cup; Step S6: Obtain the position and orientation information of the target coffee cup within the coffee making area; Step S7: In response to the operation command on the coffee cup, obtain the robot's position information, and identify the coffee cup's movement path based on the target coffee cup in the coffee making area and the target coffee cup's position and posture information.

[0025] First, it is necessary to collect environmental images, including those of the coffee-making area. This can be achieved in various ways. For example, a camera permanently installed in the coffee-making area can be used to take periodic photos.

[0026] After acquiring the environmental image, at least one candidate region image of a possible coffee cup is identified from the environmental image. This step aims to initially filter out areas in the image that may contain a coffee cup, thereby reducing the computational burden of subsequent processing.

[0027] Next, the structural standard parameters of the coffee cup are obtained, and these parameters are matched with candidate block images of various suspected coffee cups to obtain the coffee cup shape recognition result. The structural standard parameters of the coffee cup can be pre-stored in the system's database; for example, it can store CAD model data, key dimension proportions, or feature point coordinates of the coffee cup. During the matching process, the geometric features (such as contour, area, and aspect ratio) of each candidate block image can be compared with these standard parameters.

[0028] Simultaneously, the coffee-making area is identified from the environmental image, and it is determined whether each candidate image block of a suspected coffee cup falls within this area, thus obtaining the coffee cup location identification result. The scope of the coffee-making area can be identified in several ways. For example, by manually calibrating, the pixel coordinates of the four corner points of the coffee-making area in the image are determined, thereby defining an area as the coffee-making area.

[0029] Then, for each candidate block image of a suspected coffee cup, the degree of overlap between its bounding box and the identified coffee-making area is calculated. If the candidate block image is within the coffee-making area, or if its overlap with the coffee-making area exceeds a preset threshold, then the candidate block image is considered to meet the location requirements.

[0030] Subsequently, the coffee cup shape recognition results are fused with the coffee cup position recognition results to determine the target coffee cup. The fusion process aims to comprehensively consider information from both shape and position dimensions to improve recognition accuracy.

[0031] After identifying the target coffee cup, it is necessary to obtain its position and orientation information within the coffee-making area. The coffee cup position recognition results and coffee cup shape recognition results obtained in steps S3 and S4 contain the position and shape of candidate block images of suspected coffee cups, which include the position and shape of the target coffee cup. After identifying the target coffee cup, its position and shape can be retrieved from these results to serve as its position and orientation information.

[0032] Finally, in response to the operation command on the coffee cup, the robot's position information is obtained, and the movement path of the coffee cup is identified based on the target coffee cup within the coffee-making area and its position and posture information. That is, combining the determined target coffee cup and its position and posture information, a path planning algorithm can be used to calculate a motion trajectory from the target coffee cup to the working platform of the target coffee cup, and this trajectory is the movement path of the coffee cup.

[0033] In this embodiment, environmental images are first acquired and candidate image blocks suspected of being coffee cups are preliminarily screened, effectively narrowing down the scope of subsequent processing. Then, by matching the coffee cup's structural standard parameters with the candidate image blocks, and identifying the coffee-making area and determining whether the candidate image blocks are within that area, a dual confirmation of the coffee cup's shape and position is achieved. This dual confirmation mechanism significantly improves the accuracy of coffee cup recognition even in environments with blurry images and low contrast. Furthermore, by fusing shape and position recognition results, the target coffee cup is determined. After determining the target coffee cup, its position and posture information are accurately acquired. Finally, it can respond to operational commands and plan the coffee cup's movement path based on the precise target coffee cup information and its own position. The robot path recognition method in this embodiment, through a multi-stage recognition and confirmation mechanism, effectively solves the problems of coffee cup recognition errors and path planning deviations caused by image quality degradation in complex coffee-making environments, which are common in traditional methods.

[0034] In this embodiment, as Figure 2 As shown, the steps of identifying the environmental image and determining at least one candidate block image of a suspected coffee cup from the environmental image specifically include: Step S21: Perform grayscale processing on the environmental image to obtain the processed environmental image; Step S22: Using a binarization algorithm, the processed environment image is binarized according to a preset threshold judgment benchmark to obtain the binarized image corresponding to the processed environment image. Step S23: Perform gridding on the binarized image corresponding to the processed environment image to obtain multiple binarized block images after segmentation; Step S24: Filter multiple binarized block images according to the preset basic shape of a coffee cup to obtain multiple filtered binarized block images; Step S25: Denoise the multiple filtered binarized block images to obtain candidate block images of the suspected coffee cup.

[0035] Grayscale conversion of environmental images refers to transforming the original color environmental image into a grayscale image. This process aims to simplify image data, reduce the computational complexity of subsequent processing, and preserve brightness information in the image to facilitate subsequent feature extraction.

[0036] The binarization algorithm is used to binarize the processed environmental image according to a preset threshold criterion. This means converting the pixels in the grayscale image into a binary image containing only two pixel values ​​(e.g., black and white) based on the preset threshold criterion. The purpose of this step is to separate the target object (e.g., a coffee cup) in the image from the background, highlighting the object's outline and region.

[0037] Mesh segmentation of the binarized image corresponding to the processed environmental image refers to dividing the entire binarized image into multiple grid regions, each forming an independent binarized block image. This operation helps to decompose large images into smaller, more easily processed units, thereby enabling the extraction and analysis of local features and improving processing efficiency.

[0038] Based on this, the segmented binary block images are filtered according to the preset basic shape of a coffee cup. This means using pre-stored typical geometric shape information of a coffee cup (such as a circular or elliptical rim, a U-shaped body, etc.) to perform preliminary matching and judgment on each binary block image. The purpose is to quickly eliminate those block images that obviously do not conform to the basic shape of a coffee cup, thereby reducing the computational load of subsequent recognition.

[0039] Finally, the selected binarized block images are denoised by applying image denoising algorithms (such as median filtering, Gaussian filtering, etc.) to eliminate noise points that may be generated during image acquisition and processing. This aims to improve the quality of the selected binarized block images and ensure that subsequent coffee cup shape recognition can be based on more accurate image data.

[0040] In this embodiment, the step of filtering multiple binarized block images based on a preset basic shape of a coffee cup to obtain multiple filtered binarized block images specifically includes: Step S221: Obtain prior knowledge of the basic shape of the coffee cup to obtain the basic shape outline of the coffee cup; Step S222: Match the basic shape outline of the coffee cup with the segmented multiple binarized block images to confirm whether there is an image region in the binarized block image that matches the basic shape outline of the coffee cup, and obtain the confirmed binarized block image.

[0041] The prior knowledge of the basic shape of a coffee cup can be understood as pre-defined information about the typical geometric shape or structural features that a coffee cup might present in an image. For example, a coffee cup typically has a round or elliptical rim, a U-shaped or cylindrical body, and an arc-shaped handle. This prior knowledge is used to generate one or more basic shape contours of a coffee cup, such as a standard circle, an ellipse, or a specific combination of arcs. Its purpose is to provide a reference template for subsequent image matching.

[0042] The acquired basic shape outline of the coffee cup is further matched with multiple segmented binarized block images. This matching process aims to evaluate whether there are image regions in each binarized block image that are highly similar to the basic shape outline of the coffee cup. Specifically, various image matching algorithms can be used, such as shape description-based matching. Through this matching process, it can be identified which binarized block images contain image regions that match the basic shape outline of the coffee cup. Therefore, these binarized block images confirmed to have matching image regions are used as confirmed binarized block images, thereby effectively filtering potential coffee cup regions from a large number of binarized block images.

[0043] This step introduces prior knowledge of the basic shape of a coffee cup and transforms it into its basic outline. This allows for more targeted shape feature-based judgment when filtering the segmented binary image blocks. It is precisely this shape matching mechanism based on prior knowledge that enables the system to more effectively identify regions in the image that might represent a coffee cup, thereby improving the accuracy of the initial screening, reducing the computational load of subsequent processing, and laying the foundation for more refined identification.

[0044] In this embodiment, as Figure 3 As shown, the step of matching the structural standard parameters of the coffee cup with candidate block images of various suspected coffee cups to obtain the coffee cup shape recognition result specifically includes: Step S31: Identify the candidate block image of the suspected coffee cup and extract the graphic contour from the candidate block image of the suspected coffee cup; Step S32: Match the extracted graphic contour with the standard parameters of a coffee cup to filter out the block image containing the coffee cup from multiple candidate block images that are suspected to be coffee cups, and use the filtered block image containing the coffee cup as the coffee cup shape recognition result.

[0045] Specifically, the process involves identifying candidate regions of images resembling coffee cups and extracting their boundary information. Here, the graphic contour represents the shape features of the object. Extracting the graphic contour can be achieved using various image processing algorithms, such as edge detection algorithms. These algorithms process the candidate regions of images resembling coffee cups to identify and extract the edge information of the object in the image, thus forming the graphic contour. The goal is to transform the pixel information of the image into more structured shape information, facilitating subsequent shape analysis and matching.

[0046] Matching the extracted graphic contour with the standard parameters of a coffee cup involves comparing the graphic contour obtained from a candidate image of a potential coffee cup with pre-defined geometric features or structural attributes of a coffee cup. The standard parameters of a coffee cup may include, but are not limited to, the shape of the rim, the body, the stem, and their length-to-width ratio. The matching process can employ algorithms such as shape context feature matching. The goal is to quantify the similarity between the extracted graphic contour and the standard coffee cup shape, thereby determining whether the image block definitely contains a coffee cup.

[0047] The coffee cup shape recognition process has been refined and optimized by introducing graphic contour extraction and a matching and filtering mechanism based on standard parameters. Specifically, firstly, by extracting graphic contours from candidate image blocks that resemble coffee cups, the complex image pixel information can be simplified into clear shape boundaries, making subsequent shape analysis and matching more efficient and accurate. Secondly, these extracted graphic contours are precisely compared with preset standard parameters for coffee cups. It is precisely this matching and filtering based on clear shape features that enables the system to accurately identify the image block that truly contains the coffee cup from numerous candidate image blocks.

[0048] In this embodiment, the step of matching the extracted graphic contour with the standard parameters of a coffee cup to filter out the block image containing the coffee cup from multiple candidate block images of suspected coffee cups, and using the filtered block image containing the coffee cup as the coffee cup shape recognition result, specifically includes: Step S321: Using a shape detection algorithm model, detect and find the first feature contour that conforms to the preset circular or elliptical shape from the graphic contour, and mark the first feature contour as a potential cup rim contour. Step S322: Using the location of the potential cup rim outline as the detection reference position, detect and find the second feature outline that conforms to the preset U shape from the graphic outline and below the location of the potential cup rim outline, and mark the second feature outline as the potential cup body outline. Step S323: Using the location of the potential cup body outline as the detection reference position, detect and find the third feature outline that conforms to the preset arc shape from the graphic outline and around the location of the potential cup body outline, and mark the third feature outline as the potential cup handle outline. Step S324: Combine the images of the potential cup handle outline and the potential cup body outline and potential cup handle outline corresponding to the potential cup handle outline, and match the standard parameters of the coffee cup to filter out the block images containing the coffee cup, and use the filtered block images containing the coffee cup as the coffee cup shape recognition result.

[0049] The shape detection algorithm model can employ Hough transform to detect circles or ellipses. Its purpose is to accurately separate the features of each component of a coffee cup from its graphic outline.

[0050] The first feature contour refers to the area identified within the graphic contour that matches the shape of the coffee cup's rim, typically appearing as a circle or ellipse. Marking this as a potential rim contour allows subsequent steps to perform further detection based on it. The second feature contour refers to the area identified below the potential rim contour that matches the shape of the coffee cup's body, typically appearing as a U-shape. Marking this as a potential body contour helps determine the main body of the coffee cup.

[0051] The third feature profile refers to the area identified around the potential cup body profile that matches the shape of the coffee cup handle, typically appearing as an arc. Marking this as the potential handle profile is an auxiliary structure for identifying the coffee cup.

[0052] The detection reference position refers to the reference point or area used for subsequent feature contour detection. For example, when detecting the second feature contour, the location of the potential cup rim contour is used as the reference to ensure that the cup body is below the cup rim; when detecting the third feature contour, the location of the potential cup body contour is used as the reference to ensure that the cup handle is located around the cup body.

[0053] The standard parameters of the coffee cup may include the shape of the rim, the length-to-width ratio of the rim, the shape of the body, the length-to-width ratio of the body, the shape of the handle, and the length-to-width ratio of the handle. These parameters are used to verify and filter the detected potential contours to ensure that the identified feature contours do indeed belong to the corresponding parts of the coffee cup.

[0054] The overall shape recognition of a coffee cup is decomposed into the detection of individual features of its components (rim, body, and stem), and the relative positional relationships between these components are used for verification. First, a shape detection algorithm model identifies the potential rim outline as the starting point for recognition. Then, using this rim outline as a reference, the potential body outline is detected below it. Next, using the body outline as a reference, the potential stem outline is detected around its perimeter. This step-by-step, layer-by-layer detection method makes the coffee cup recognition process more structured and accurate. Finally, by combining these potential feature outlines and their images and matching them with preset standard parameters for coffee cups, the image blocks containing the coffee cup can be accurately selected.

[0055] In this embodiment, as Figure 4 As shown, the step of identifying the range of the coffee-making area from the environmental image and determining whether each candidate block image of a suspected coffee cup falls within this range to obtain the coffee cup location identification result specifically includes: Step S41: Determine the location of each boundary point of the coffee making area in the environmental image; Step S42: Determine the area range of the coffee-making area in the environmental image based on the positions of each boundary point in the environmental image; Step S43: Set a preset shape proportion threshold and determine whether the proportion of the overlapping area between the candidate block image of the suspected coffee cup and the coffee making area in the environmental image meets the shape proportion threshold. Step S44: Determine candidate block images of suspected coffee cups that meet the shape proportion threshold requirements, and use the candidate block images of suspected coffee cups as the coffee cup location recognition results.

[0056] The boundary points of the coffee-making area can be obtained in advance through manual annotation, importing CAD drawings, or recognition by machine learning models.

[0057] Based on the positions of each boundary point in the environmental image, the area of ​​the coffee-making region within the environmental image is determined. For example, a polygonal region can be formed by connecting these boundary points, or a rectangular or elliptical region can be obtained by fitting these boundary points, thereby accurately defining the actual spatial extent of the coffee-making region.

[0058] Furthermore, the shape proportion threshold is used to measure the extent to which a candidate region image of a suspected coffee cup covers the coffee-making area. For example, if a large portion (e.g., more than 80%) of a candidate region image of a suspected coffee cup overlaps with the coffee-making area, then the candidate region image of the suspected coffee cup is considered to be located within the coffee-making area. The calculation of overlapping areas can be achieved through geometric region intersection calculation.

[0059] Finally, candidate image blocks that meet the shape proportion threshold requirements are identified as potential coffee cups, and these candidate image blocks are used as the coffee cup location identification results. That is, only candidate image blocks whose majority of the area is located within the coffee-making area are confirmed as potentially containing a target coffee cup, thus excluding interfering objects located outside the designated area.

[0060] In this step, the boundary points and area of ​​the coffee-making region are first precisely defined, providing a clear reference for subsequent coffee cup location determination. Then, by introducing a shape proportion threshold, the overlap between candidate image blocks of suspected coffee cups and the coffee-making region is evaluated, ensuring that only suspected coffee cups occupying a significant proportion of the coffee-making region are identified. Therefore, this scheme can effectively distinguish between objects located inside and outside the coffee-making region, avoiding misidentification of objects outside the region as the target coffee cup, thereby improving the accuracy and reliability of coffee cup location identification.

[0061] In this embodiment, the step of fusing the coffee cup shape recognition result and the coffee cup position recognition result to determine the target coffee cup specifically includes: Step S51: Match the structural standard parameters of the coffee cup with the candidate block images of each suspected coffee cup, and give a first confidence score to each candidate block image of a suspected coffee cup that has been matched. Step S52: Identify the range of the coffee making area from the environmental image, determine whether each candidate block image of a suspected coffee cup is within this range, and give a second confidence score to the proportion of each candidate block image of a suspected coffee cup in the coffee making area. Step S53: Combine the first confidence score corresponding to each candidate block image of a suspected coffee cup that has been matched and the second confidence score corresponding to the proportion of each candidate block image of a suspected coffee cup in the coffee making area to calculate the shape judgment score. Step S54: Determine the target coffee cup based on the shape score.

[0062] Specifically, the first confidence score can be understood as a quantification of the degree to which a candidate image of a suspected coffee cup matches the structural standard parameters of the coffee cup. For example, based on the output of a matching algorithm (such as feature point matching, contour similarity calculation, etc.), candidate images with high matching scores can be assigned higher first confidence scores, while those with low matching scores can be assigned lower scores. This score can be a number between 0 and 1, and its purpose is to characterize the probability that the candidate image is the target coffee cup in terms of shape.

[0063] Similarly, the second confidence score can be understood as a quantification of the correlation between the candidate image of the suspected coffee cup and the extent of the coffee-making region. Specifically, this score can be calculated based on the proportion of the overlapping area between the candidate image of the suspected coffee cup and the extent of the coffee-making region within the corresponding candidate image of the suspected coffee cup. For example, a higher overlap ratio results in a higher second confidence score, and vice versa. This score can also be a number between 0 and 1, its purpose being to characterize the probability that the candidate image is the target coffee cup in spatial location.

[0064] In practical applications, the shape judgment score is a comprehensive evaluation value calculated by combining the first confidence score and the second confidence score. For example, a weighted average or other fusion algorithm can be used to calculate this score. For instance, the shape judgment score = W1 * (first confidence score) + W2 * (second confidence score), where W1 and W2 are preset weighting coefficients used to adjust the importance of shape matching and position matching in the final judgment. Its purpose is to provide a quantitative indicator reflecting the overall probability that each candidate block image is the target coffee cup.

[0065] Therefore, after obtaining the shape judgment score of each candidate block image of a suspected coffee cup, the target coffee cup can be determined based on the score.

[0066] In this step, by introducing a first confidence score and a second confidence score and fusing them to calculate a shape judgment score, the ambiguity and uncertainty that may exist in traditional simple fusion methods when determining the target coffee cup are resolved. Specifically, the first confidence score quantifies the degree to which the candidate block image matches the standard parameters of the coffee cup structure in terms of shape, ensuring the correctness of the selected target in appearance; the second confidence score quantifies the degree of spatial correlation between the candidate block image and the coffee-making area, ensuring the rationality of the selected target in location. Because of the quantitative evaluation of the two key dimensions of shape and location, the system can more precisely identify the object that best matches the target coffee cup, avoiding misjudgments caused by insufficient or interference-prone information in a single dimension.

[0067] In an exemplary example, suppose three candidate block images A, B, and C that resemble coffee cups are identified in an environmental image.

[0068] For candidate block image A: it has a high degree of matching with the structural standard parameters of the coffee cup and is given a first confidence score of 0.9; its overlap ratio with the coffee making area is 0.8 and it is given a second confidence score of 0.8.

[0069] For candidate block image B: its matching degree with the structural standard parameters of the coffee cup is moderate, and it is assigned a first confidence score of 0.7; its overlap ratio with the coffee making area is 0.95, and it is assigned a second confidence score of 0.95.

[0070] For candidate block image C: its matching degree with the structural standard parameters of the coffee cup is low, and it is assigned a first confidence score of 0.5; its overlap ratio with the coffee making area is 0.2, and it is assigned a second confidence score of 0.2.

[0071] Assuming the preset weighting coefficients W1 is 0.6 and W2 is 0.4, then: The shape judgment score of candidate block image A = 0.6*0.9 + 0.4*0.8 = 0.54 + 0.32 = 0.86; The shape judgment score of candidate block image B = 0.6*0.7 + 0.4*0.95 = 0.42 + 0.38 = 0.80; The shape judgment score of candidate block image C = 0.6*0.5 + 0.4*0.2 = 0.30 + 0.08 = 0.38.

[0072] Based on the shape judgment score, candidate block image A has the highest shape judgment score of 0.86. Therefore, the system identifies candidate block image A as the target coffee cup. Through this quantitative evaluation and fusion mechanism, even in the presence of multiple potential targets, the system can accurately identify the most suitable target coffee cup based on the comprehensive matching degree of their shape and position.

[0073] In this embodiment, obtaining the position and orientation information of the target coffee cup within the coffee-making area specifically includes: Step S61: Perform key point detection on the environmental image to determine the position of each key point contained in the target coffee cup in the environmental image; Step S62: Determine the position and orientation of the target coffee cup in the environmental image based on the positions of each key point in the environmental image.

[0074] Keypoint detection refers to identifying and locating feature points on a target coffee cup in an environmental image through image processing. These keypoints can be edge points, corner points, texture feature points, etc., of the coffee cup. For example, a deep learning-based object detection model can be used to identify these keypoints.

[0075] After determining the positions of the keypoints within the target coffee cup in the environmental image, the overall position and orientation of the target coffee cup can be calculated using the positional information of these keypoints. Specifically, the positional information of the target coffee cup can be represented by the coordinates of its keypoints. Orientation information typically refers to the orientation of the target coffee cup in space, for example, described using Euler angles.

[0076] The proposed solution extracts feature points of a target coffee cup from environmental images through keypoint detection. The location information of these keypoints serves as the foundational data, enabling subsequent calculation of the coffee cup's pose within the coffee-making area. This keypoint-based method improves the accuracy of position and pose information acquisition.

[0077] In summary, the robot path recognition method in this embodiment acquires environmental images, including the coffee-making area, and performs multi-stage recognition and processing on the images. This includes identifying candidate image blocks that resemble coffee cups, obtaining standard structural parameters of the coffee cup for shape matching, identifying the coffee-making area for location determination, and finally fusing the shape and location recognition results to determine the target coffee cup, thereby confirming the coffee cup's movement path. This method effectively solves the problem in existing technologies where coffee powder adhering to the camera lens causes image blurring, leading to inaccurate coffee cup recognition and path planning deviations.

[0078] Secondly, such as Figure 5 As shown, a robot path recognition system 100 is provided, comprising: Acquisition module 101 is used to acquire environmental images, including the coffee making area; The identification module 102, the identification module 101 is used to identify the environmental image and determine at least one candidate block image of a suspected coffee cup from the environmental image; The coffee cup shape recognition result acquisition module 103 is used to acquire the structural standard parameters of the coffee cup, match the structural standard parameters of the coffee cup with the candidate block images of each suspected coffee cup, and obtain the coffee cup shape recognition result. The coffee cup location recognition result acquisition module 104 is used to identify the range of the coffee making area from the environmental image and determine whether each candidate block image of a suspected coffee cup is within the range, so as to obtain the coffee cup location recognition result. The determining module 105 is used to fuse the coffee cup shape recognition result and the coffee cup position recognition result to determine the target coffee cup; Position and attitude acquisition module 106, which is used to acquire the position information and attitude information of the target coffee cup in the coffee making area; The path recognition module 107 is used to respond to the operation command of the coffee cup and obtain the position information of the robot, and to identify the movement path of the coffee cup based on the target coffee cup in the coffee making area and the position and posture information of the target coffee cup.

[0079] This embodiment presents a robot path recognition system, comprising a data acquisition module, a recognition module, a coffee cup shape recognition result acquisition module, a coffee cup position recognition result acquisition module, a determination module, a position and pose acquisition module, and a path recognition module. The system acquires environmental images, including the coffee-making area, and performs multi-stage recognition and processing on these images. This includes identifying candidate regions resembling a coffee cup, acquiring standard structural parameters of the coffee cup for shape matching, identifying the coffee-making area for position determination, and finally fusing the shape and position recognition results to determine the target coffee cup, thereby confirming the coffee cup's movement path. This method effectively solves the problem in existing technologies where coffee powder adhering to the camera lens causes image blurring, leading to inaccurate coffee cup recognition and path planning deviations.

[0080] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit them. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure.

Claims

1. A robot path recognition method, used by a robot to determine the movement path of a coffee cup during coffee making, characterized in that, include: Collect environmental images, including the coffee-making area; The environmental image is identified, and at least one candidate block image of a suspected coffee cup is determined from the environmental image; Obtain the standard structural parameters of the coffee cup, match the standard structural parameters of the coffee cup with the candidate block images of each suspected coffee cup, and obtain the coffee cup shape recognition result; The coffee-making area is identified from the environmental image, and it is determined whether each candidate block image of a suspected coffee cup is within this area, so as to obtain the coffee cup location identification result. The target coffee cup is determined by fusing the coffee cup shape recognition result with the coffee cup position recognition result; Obtain the position and orientation information of the target coffee cup within the coffee-making area; In response to the operation command of the coffee cup, the robot obtains its position information and identifies the movement path of the coffee cup based on the target coffee cup in the coffee making area and the position and posture information of the target coffee cup.

2. The robot path recognition method according to claim 1, characterized in that, The step of identifying the environmental image and determining at least one candidate block image of a suspected coffee cup from the environmental image specifically includes: The environmental image is converted to grayscale to obtain the processed environmental image; A binarization algorithm is used to binarize the processed environmental image according to a preset threshold judgment benchmark, so as to obtain the binarized image corresponding to the processed environmental image. The binarized image corresponding to the processed environmental image is divided into grids to obtain multiple binarized block images. Multiple binarized block images are filtered based on a preset coffee cup basic shape to obtain multiple filtered binarized block images; Denoising is performed on multiple filtered binarized block images to obtain candidate block images that resemble coffee cups.

3. The robot path recognition method according to claim 2, characterized in that, The step of filtering multiple binarized block images based on a preset coffee cup basic shape to obtain multiple filtered binarized block images specifically includes: Obtain prior knowledge of the basic shape of the coffee cup to obtain its basic shape outline; The basic shape outline of the coffee cup is matched with multiple segmented binarized block images to confirm whether there is an image region in the binarized block image that matches the basic shape outline of the coffee cup, thus obtaining the confirmed binarized block image.

4. The robot path recognition method according to claim 1, characterized in that, The step of matching the structural standard parameters of the coffee cup with candidate block images of various suspected coffee cups to obtain the coffee cup shape recognition result specifically includes: Identify candidate block images of suspected coffee cups and extract graphic contours from them; The extracted graphic contour is matched with the standard parameters of a coffee cup to filter out the block images containing the coffee cup from multiple candidate block images that are suspected to be coffee cups, and the filtered block images containing the coffee cup are used as the coffee cup shape recognition result.

5. The robot path recognition method according to claim 4, characterized in that: The standard parameters of the coffee cup include the shape of the rim and the ratio of its length to width, the shape of the body and the ratio of its length to width, and the shape of the handle and the ratio of its length to width.

6. The robot path recognition method according to claim 4, characterized in that, The step of matching the extracted graphic contour with the standard parameters of a coffee cup to filter out the block image containing the coffee cup from multiple candidate block images that are suspected to be coffee cups, and using the filtered block image containing the coffee cup as the coffee cup shape recognition result, specifically includes: A shape detection algorithm model is used to detect and find the first feature contour that conforms to the preset circular or elliptical shape from the graphic contour, and the first feature contour is marked as a potential cup rim contour. Using the location of the potential cup rim outline as the detection reference position, a second feature outline conforming to the preset U shape is detected and found from the graphic outline and below the location of the potential cup rim outline, and the second feature outline is marked as the potential cup body outline. Using the location of the potential cup body outline as the detection reference position, a third feature outline that conforms to the preset arc shape is detected and found from the graphic outline and around the location of the potential cup body outline, and the third feature outline is marked as the potential cup handle outline. By combining images of potential cup handle contours and corresponding potential cup body contours and potential cup handle contours, and matching them with standard parameters of a coffee cup, block images containing coffee cups are obtained, and these filtered block images containing coffee cups are used as the coffee cup shape recognition results.

7. The robot path recognition method according to claim 1, characterized in that, The steps of identifying the coffee-making area from the environmental image and determining whether each candidate block image of a suspected coffee cup falls within this area to obtain the coffee cup location identification result specifically include: Determine the locations of the various boundary points within the coffee-making area in the environmental image; The coffee-making area is determined within the environmental image based on the location of each boundary point. A preset shape proportion threshold is used to determine whether the proportion of the overlapping area between the candidate block image of the suspected coffee cup and the coffee making area in the environmental image meets the shape proportion threshold. Candidate block images of suspected coffee cups that meet the shape proportion threshold requirements are identified, and these candidate block images of suspected coffee cups are used as the coffee cup location recognition results.

8. The robot path recognition method according to claim 1, characterized in that, The step of fusing the coffee cup shape recognition result and the coffee cup position recognition result to determine the target coffee cup specifically includes: The structural standard parameters of the coffee cup are matched with the candidate block images of each suspected coffee cup, and a first confidence score is given to each candidate block image of a suspected coffee cup that has been matched. The scope of the coffee-making area is identified from the environmental image, and it is determined whether each candidate block image of a suspected coffee cup is within this scope. The proportion of each candidate block image of a suspected coffee cup within the scope of the coffee-making area is given a second confidence score. The shape judgment score is calculated by combining the first confidence score corresponding to each candidate block image of a suspected coffee cup that has been matched and the second confidence score corresponding to the proportion of each candidate block image of a suspected coffee cup in the coffee making area. The target coffee cup is determined by judging its shape and score.

9. The robot path recognition method according to claim 1, characterized in that, The acquisition of the target coffee cup's position and orientation information within the coffee-making area specifically includes: Key point detection is performed on the environmental image to determine the location of each key point contained in the target coffee cup in the environmental image; Based on the positions of each key point in the environmental image, the position and orientation of the target coffee cup in the environmental image are determined.

10. A robot path recognition system, characterized in that, include: The acquisition module is used to acquire environmental images, including the coffee-making area. The recognition module is used to recognize the environmental image and determine at least one candidate block image of a suspected coffee cup from the environmental image; A coffee cup shape recognition result acquisition module is used to acquire the structural standard parameters of a coffee cup, match the structural standard parameters of the coffee cup with candidate block images of each suspected coffee cup, and obtain the coffee cup shape recognition result. The coffee cup location recognition result acquisition module is used to identify the range of the coffee making area from the environmental image and determine whether each candidate block image of a suspected coffee cup is within the range, so as to obtain the coffee cup location recognition result. The determining module is used to fuse the coffee cup shape recognition result and the coffee cup position recognition result to determine the target coffee cup; A position and attitude acquisition module is used to acquire the position and attitude information of the target coffee cup within the coffee making area. The path recognition module is used to respond to operation commands to the coffee cup, acquire the robot's position information, and identify the coffee cup's movement path based on the target coffee cup within the coffee-making area, its position information, and its posture information.

Images