A robot vision recognition method, system, device and storage medium

Abstract

The application provides a robot visual recognition method, system, device and storage medium. The visual label corner points in visual recognition are determined according to the path matching area and the dynamic operation characteristics of the robot. The dynamic compensation amount in visual recognition is determined by recognizing the deviation interval and the real-time path trajectory of the robot. The static offset domain of the robot in visual recognition of the operation path is determined according to the operation path edge. Then, the recognition loss value in visual recognition is determined by the static offset domain and the label corner points. The visual fitness of the robot in path recognition is determined by the dynamic compensation amount and the recognition loss value. The path recognized by the robot visual recognition is dynamically adjusted according to the visual fitness. The above scheme is based on visual fitness, and can realize dynamic matching of the recognized path under the interference of the lack of tracing freedom degree of robot visual recognition, thereby improving the recognition ability of the robot to the operation path.

Description

Technical Field

[0001] This application relates to the field of visual recognition technology, and more specifically, to a robot visual recognition method, system, device, and storage medium. Background Technology

[0002] Visual recognition refers to the process of automatically detecting, analyzing, and understanding objects, scenes, or features from images or videos using computer vision and image processing technologies. It acquires environmental information through cameras or other visual sensors. Visual recognition systems use a series of algorithms, including image preprocessing, feature extraction, object detection, and classification, to identify and interpret the content in images. With the support of deep learning, this technology can handle complex visual tasks such as face recognition, object detection, and scene understanding. Visual recognition is widely used in fields such as autonomous driving, robot navigation, security monitoring, and intelligent manufacturing, improving the automation and intelligence levels of the systems.

[0003] In existing technologies, robot visual recognition is typically achieved by integrating high-precision cameras and other visual sensors. These devices are responsible for capturing image or video data of the surrounding environment. Subsequently, computer vision algorithms are used to preprocess the acquired visual information to improve image quality and highlight key features. Next, feature extraction techniques are used to identify key information such as edges, corners, and textures in the image, and pattern recognition and machine learning algorithms are applied to analyze and classify these features to identify and understand objects, paths, and other important elements in the environment. However, in robot visual recognition, due to the influence of factors such as changes in lighting, occlusion, background complexity, and blurred or uncommon path features, the path information recognized by the robot during the visual recognition of its operating path is often compromised. There is a problem of missing tracking degrees of freedom. Tracking degrees of freedom represent a robot's ability to effectively track and identify its running path through sensors and algorithms during visual recognition. During visual recognition, the robot's flexibility and accuracy in following a predetermined path under different environmental conditions (such as changes in lighting, occlusion, background complexity, path blurring, or uncommon features) are crucial. Therefore, the lack of tracking degrees of freedom can lead to the accumulation of errors in visual recognition, weakening the robot's ability to identify running paths and thus reducing its dynamic matching ability during visual recognition. As a result, how to achieve dynamic matching of recognition paths under the interference of missing tracking degrees of freedom, thereby improving the robot's ability to identify running paths, has become a challenge for the industry. Summary of the Invention

[0004] This application provides a robot vision recognition method, system, device, and storage medium, which can realize dynamic matching of recognition paths when robot vision recognition is interfered with by the lack of tracking degrees of freedom, thereby improving the robot's ability to identify the running path.

[0005] In a first aspect, this application provides a robot visual recognition method, comprising the following steps:

[0006] Acquire path recognition information collected by the robot during its operation;

[0007] The path recognition information is matched and filtered to obtain the path matching area during visual recognition. The visual label corner points in visual recognition are determined based on the path matching area and the dynamic operation characteristics of the robot.

[0008] The robot's body zero position is marked, and the body zero position is configured with deviation to obtain the identification deviation range when the robot recognizes the path. The dynamic compensation amount for visual recognition is determined based on the identification deviation range and the robot's real-time path trajectory.

[0009] The robot obtains the edge of the running path in visual recognition, determines the static offset domain when the robot performs visual recognition of the running path based on the edge of the running path, and then determines the recognition loss value when performing visual recognition based on the static offset domain and the label corner point.

[0010] The robot's visual adaptability to path recognition is determined by the dynamic compensation amount and the recognition loss value, and the path visually recognized by the robot is dynamically adjusted based on the visual adaptability.

[0011] In some embodiments, matching and filtering the path identification information to obtain the path matching region for visual recognition specifically includes:

[0012] Extract the path label from the path recognition information when the robot performs visual recognition;

[0013] Determine the path prediction value during robot vision recognition;

[0014] The path labels and path predictions are used for feature filtering to obtain the path matching region for visual recognition.

[0015] In some embodiments, determining the dynamic compensation amount for visual recognition based on the identification deviation range and the robot's real-time path trajectory specifically includes:

[0016] Obtain the robot's real-time path trajectory;

[0017] The dynamic compensation factor for the robot during operation is determined based on the identification deviation range.

[0018] The dynamic compensation amount for visual recognition is determined based on the dynamic compensation factor and the real-time path.

[0019] In some embodiments, determining the static offset domain for the robot to perform visual recognition of the running path based on the running path edge specifically includes:

[0020] The continuous offset baseline is determined based on the edge of the running path when the robot performs visual recognition of the running path.

[0021] Determine the long-tail dataset when the robot visually recognizes its running path;

[0022] The static offset domain is determined based on the continuous offset baseline and the long-tail dataset when the robot performs visual recognition of the running path.

[0023] In some embodiments, determining the recognition loss value for visual recognition using the static offset domain and the label corner points specifically includes:

[0024] The recognition concentration during robot vision recognition is determined based on the static offset domain.

[0025] The recognition discreteness during robot vision recognition is determined based on the corner points of the label.

[0026] The recognition loss value during visual recognition is determined by the recognition concentration and the recognition discreteness.

[0027] In some embodiments, determining the robot's visual adaptability to path recognition using the dynamic compensation amount and the recognition loss value specifically includes:

[0028] The first adaptive component for the robot's path recognition is determined based on the dynamic compensation amount;

[0029] The second adaptive component for the robot's path recognition is determined based on the recognition loss.

[0030] The visual adaptation of the robot during path recognition is determined based on the first adaptation component and the second adaptation component.

[0031] In some embodiments, path recognition information of the robot during operation is obtained by reading the robot's data storage unit.

[0032] Secondly, this application provides a robot vision recognition system, comprising:

[0033] The acquisition module is used to acquire path recognition information collected by the robot during operation;

[0034] The processing module is used to match and filter the path recognition information to obtain the path matching area during visual recognition, and to determine the visual label corner points in visual recognition based on the path matching area and the dynamic operation characteristics of the robot.

[0035] The processing module is also used to mark the zero position of the robot body, configure the deviation of the zero position of the body to obtain the identification deviation range when the robot path is recognized, and determine the dynamic compensation amount when visual recognition based on the identification deviation range and the real-time path trajectory of the robot.

[0036] The processing module is also used to obtain the edge of the running path in robot visual recognition, determine the static offset domain when the robot performs visual recognition of the running path based on the edge of the running path, and then determine the recognition loss value when performing visual recognition based on the static offset domain and the label corner point.

[0037] The execution module is used to determine the robot's visual adaptation to path recognition by the dynamic compensation amount and the recognition loss value, and to dynamically adjust the path visually recognized by the robot based on the visual adaptation.

[0038] Thirdly, this application provides a computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the robot vision recognition method described above.

[0039] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described robot vision recognition method.

[0040] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0041] The robot visual recognition method, system, device, and storage medium provided in this application first acquire path recognition information collected by the robot during operation; second, the path recognition information is matched and filtered to obtain the path matching area during visual recognition, and the visual label corner points in visual recognition are determined based on the path matching area and the robot's dynamic operation characteristics; next, the robot's body zero position is marked, and the body zero position is configured with deviation to obtain the identification deviation range during robot path recognition, and the dynamic compensation amount during visual recognition is determined based on the identification deviation range and the robot's real-time path trajectory; then, the running path edge in robot visual recognition is acquired, and the static offset domain when the robot performs visual recognition of the running path is determined based on the running path edge, and then the recognition loss value during visual recognition is determined by the static offset domain and the label corner points; finally, the visual adaptability of the robot during path recognition is determined by the dynamic compensation amount and the recognition loss value, and the path visually recognized by the robot is dynamically adjusted based on the visual adaptability.

[0042] Therefore, this application enables dynamic path matching for robots when visual recognition is hampered by a lack of tracking degrees of freedom, thereby improving the robot's ability to identify operating paths. Firstly, to correct errors in path recognition and tracking in real time, a dynamic compensation amount is determined based on the identification deviation range and the robot's real-time path trajectory to ensure the robot can accurately identify and move along a predetermined path, optimizing its path recognition and navigation capabilities in complex environments. Secondly, to reflect the degree of errors or deviations occurring during visual recognition, a recognition loss value is determined using the static offset domain and label corner points. This recognition loss value measures the robot's deviation from the ideal path during recognition. The process involves several steps: first, determining the degree of path recognition; then, determining the robot's visual adaptability to path recognition through dynamic compensation and recognition loss values. This aims to adjust the robot's motion or path planning to adapt to environmental changes in visual recognition, thereby improving the robot's ability to adjust its path in real time in dynamic environments and effectively avoiding the loss of tracking freedom caused by errors in path recognition and tracking and mistakes occurring during visual recognition. Finally, the path visually recognized by the robot is dynamically adjusted based on the visual adaptability. In summary, the technical solution provided in this application can achieve dynamic matching of recognition paths when robot visual recognition is interfered with by the loss of tracking freedom, thereby improving the robot's ability to identify operating paths. Attached Figure Description

[0043] Figure 1 This is an exemplary flowchart of a robot vision recognition method according to some embodiments of this application;

[0044] Figure 2 This is an exemplary flowchart illustrating the determination of visual label corner points in visual recognition according to some embodiments of this application;

[0045] Figure 3 This is an exemplary flowchart illustrating the determination of the identification deviation range when identifying a robot path, according to some embodiments of this application;

[0046] Figure 4 These are schematic diagrams of exemplary hardware and / or software of a robot vision recognition system according to some embodiments of this application;

[0047] Figure 5 This is a schematic diagram of the structure of a computer device for implementing a robot vision recognition method according to some embodiments of this application. Detailed Implementation

[0048] The core of this application is as follows: First, acquire path recognition information collected by the robot during operation; second, match and filter the path recognition information to obtain the path matching region for visual recognition, and determine the visual label corner points in visual recognition based on the path matching region and the robot's dynamic operation characteristics; next, mark the robot's body zero position, and configure the body zero position with deviation to obtain the identification deviation range for robot path recognition, and determine the dynamic compensation amount for visual recognition based on the identification deviation range and the robot's real-time path trajectory; then, acquire the running path edge in the robot's visual recognition, determine the static offset domain for the robot's visual recognition of the running path based on the running path edge, and then determine the recognition loss value for visual recognition based on the static offset domain and the label corner points; finally, determine the robot's visual adaptability for path recognition through the dynamic compensation amount and the recognition loss value, and dynamically adjust the path visually recognized by the robot based on the visual adaptability. The above scheme, based on visual adaptability, can achieve dynamic matching of recognition paths even when robot visual recognition is interfered with by the lack of tracking degrees of freedom, thereby improving the robot's ability to recognize running paths.

[0049] To better understand the above technical solutions, a detailed description of the solutions will be provided below in conjunction with the accompanying drawings and specific implementation methods. (Reference) Figure 1 The figure is an exemplary flowchart of a robot vision recognition method according to some embodiments of this application. The robot vision recognition method 100 mainly includes the following steps:

[0050] In step 101, path recognition information collected by the robot during operation is obtained.

[0051] In practical implementation, a vision sensor is installed on the robot to ensure that its vision covers the robot's forward direction and path. Using the sensor's software tools, appropriate parameters such as resolution, frame rate, and exposure time are set to ensure clear images or videos are acquired under different lighting conditions. The vision sensor can be a high-resolution camera, depth camera, LiDAR, etc., and is not limited here. Path image or video data is continuously acquired through the sensor's interface and transmitted to the robot's data storage unit. The path recognition information of the robot during operation can be obtained by reading the robot's data storage unit. In other embodiments, other methods can also be used to obtain the robot's path recognition information during operation, and are not limited here.

[0052] It should be noted that the path recognition information in this application refers to the collection of data such as environmental features, path shape, and obstacle positions collected by the robot during operation. It aims to provide accurate navigation basis for robot visual recognition, ensuring that it can effectively identify and track the predetermined path. The collection of these data is detailed data about the path extracted from the environment by visual sensors and computer vision algorithms. This data includes the path's position, shape, edges, key points, corners, texture features, and dynamic information related to the path (such as real-time changes in the path, obstacle positions, etc.). The path recognition information can guide the robot's navigation and obstacle avoidance, ensuring that the robot can accurately identify, track, and adjust its travel path.

[0053] In step 102, the path recognition information is matched and filtered to obtain the path matching area during visual recognition. The visual label corner points in visual recognition are determined based on the path matching area and the dynamic operation characteristics of the robot.

[0054] In some embodiments, matching and filtering the path identification information to obtain the path matching region for visual recognition can be achieved through the following steps:

[0055] Extract the path label from the path recognition information when the robot performs visual recognition;

[0056] Determine the path prediction value during robot vision recognition;

[0057] The path labels and path predictions are used for feature filtering to obtain the path matching region for visual recognition.

[0058] In practice, firstly, during robot operation, image data of the path is acquired through cameras or other visual sensors. Then, image processing algorithms (such as edge detection and corner detection) are used to extract significant path features from these images. These features can include path corners, forks, edge lines, etc. Based on the extracted features, corresponding path labels are generated. These labels may be numbers, symbols, or other identifiers used to represent key points on the path. For example, in a complex road network, the robot may extract the location of each turning point, intersection, or road sign as a path label. That is, during path recognition, if an intersection is detected, it can be labeled with an "X"; if a right turn is detected, it can be labeled with an "R"; then... Then, big data analysis models, such as regression analysis models and clustering models, are used to analyze the robot's path data in previous operations. Combined with the current operating status, the possible direction or position of its next movement is predicted. This includes the analysis of factors such as forward direction, speed, and position deviation. For example, if the robot has chosen to turn right multiple times on the previous path, it may also prioritize the right turn direction in the current prediction. Next, based on the analysis results, path predictions are generated, including predictions of the next possible node or region on the path. The path prediction can be a specific coordinate range, direction vector, or probability distribution. The process of generating path predictions can use deep reinforcement learning algorithms or path planning algorithms. In other embodiments, other methods can also be used to determine path predictions, which will not be elaborated here.

[0059] In addition, in specific implementation, the path label and the path prediction are subjected to feature filtering to obtain the path matching region for visual recognition. This can be achieved in the following way: the path label and the path prediction are compared and filtered to determine the path label with the highest matching degree among the path predictions. The filtering criteria may include the proximity of geometric position, the consistency of path direction, the feature importance of the label, etc. For example, if the path label "R" indicates a right turn and the path prediction also tends to be a right turn, then these two features will match, and this matching region will be given priority. Then, based on the filtering results, the path matching region for visual recognition is determined. This region is the optimal prediction region of the robot's actual travel path, representing the part of the road that the robot may travel in the current environment. After recognizing the path label "X" (crossroads) and the prediction "right turn", this region can be used as the path matching region. That is, the path matching region is determined as the right-turn road part, and the robot will perform more refined recognition and navigation in this part. In other embodiments, other methods can also be used to determine the path matching region, which is not limited here.

[0060] It should be noted that, in this application, path labels refer to identifiers used to identify key feature points or regions on a path during the path recognition process. These labels can represent important elements in the path, such as corners, intersections, endpoints, starting points, and obstacles, and help the robot make accurate judgments and positioning during visual recognition and path planning. Path prediction refers to the estimated value of the robot's future path based on its current state, historical path data, and environmental information during the path recognition and navigation process. Path prediction can include information such as predicted direction, position, speed, and probability of possible path selection. Path matching region refers to the range or area of ​​the path that the robot may actually travel on, determined by comparing and filtering path recognition information with predicted path data during the robot's visual recognition process. This region represents the part of the path that the robot is most likely to follow in the current environment, and usually includes key points, line segments, or feature regions on the path.

[0061] In some embodiments, reference Figure 2 As shown in the figure, this is an exemplary flowchart illustrating the determination of visual label corner points in visual recognition according to some embodiments of this application. In this embodiment, the determination of visual label corner points in visual recognition based on the path matching area and the dynamic operation characteristics of the robot can be achieved by the following steps:

[0062] In step 1021, the path corner points of the robot during visual recognition are determined based on the path matching region;

[0063] In step 1022, the dynamic operating characteristics of the robot are determined;

[0064] In step 1023, the visual label corner points in visual recognition are determined based on the path corner points and the dynamic running features.

[0065] In practical implementation, firstly, a detailed geometric analysis is performed on the identified path matching region to determine the key features within the region. Typically, path corners appear at path bends, intersections, or significant changes in path edges. Corner detection algorithms (such as Harris corner detection, Shi-Tomasi corner detection, etc.) can be used to identify the actual corners within the path matching region. These algorithms can detect significant inflection points in the image based on gradient changes; these significant inflection points are then designated as path corners. Next, real-time dynamic data is acquired from the robot's sensors (such as accelerometers, gyroscopes, speedometers, etc.). This data includes the robot's current speed, acceleration, heading angle, and turning rate. Based on the collected dynamic data, the robot's current operating state is analyzed. For example, it is analyzed whether the robot needs to decelerate, turn, or make other adjustments when approaching a path corner. The robot's current operating state is then used as the robot's dynamic... The system analyzes the dynamic operation characteristics of the robot. Finally, based on the location of the path corners and the robot's current dynamic operation characteristics, it determines which corners need to be marked as tagged corners in visual recognition. For example, if the robot approaches a sharp turn at a high speed, the corner at the turn will be marked as a tagged corner, prompting the robot to focus on identification and steering adjustments at that point. Then, based on the dynamic characteristics, the position and marking method of the tagged corners are optimized to ensure effective recognition in actual operation. This optimization process may include fine-tuning the corners to adapt to the robot's actual trajectory and speed. Ultimately, the optimized corners are marked as tagged corners in visual recognition, and relevant information (such as coordinates, recognition priority, etc.) is generated for subsequent path recognition and navigation adjustments. The information generated for the tagged corners may include the specific coordinates of the corner, the recognition priority (e.g., high-priority corners may be checked more frequently), and related dynamic adjustment parameters.

[0066] It should be noted that, in this application, the path corner points of the robot during visual recognition refer to key locations in the path that have significant turns, intersections, or other geometric changes during the visual recognition process; the dynamic operation characteristics of the robot refer to various physical parameters and behavioral characteristics of the robot's real-time state during operation. These characteristics include, but are not limited to, the robot's speed, acceleration, turning angle, rate of change of direction, position deviation, and posture. These dynamic operation characteristics reflect the robot's movement mode and state in the current environment and directly affect its ability to recognize paths, make navigation decisions, and avoid obstacles; the labeled corner points in visual recognition refer to key feature points that are identified and marked in the image or video data during visual recognition. These feature points are usually used as reference points for path recognition and target localization.

[0067] In step 103, the robot's body zero position is marked, the body zero position is configured with deviation to obtain the identification deviation range when the robot recognizes the path, and the dynamic compensation amount for visual recognition is determined based on the identification deviation range and the robot's real-time path trajectory.

[0068] In specific implementation, marking the robot's zero position can be achieved in the following way: a fixed position or point on the robot body is used as the zero position reference point. This point is usually an obvious and easily located feature point on the robot, used as the benchmark for all motion and position calculations, such as the center of the robot's chassis, the midpoint of the sensor installation position, or a landmark component (such as the center of the control panel). When the robot starts, the absolute coordinates of the zero position in the environment are measured using position sensors (such as LiDAR, vision sensors, or GPS). This process can be in the ground coordinate system of the robot's location or other known coordinate systems, and the measured coordinate data is recorded into the robot's control system, including the X, Y, and Z coordinates of the zero position (in three-dimensional space), or only the X and Y coordinates (in a two-dimensional plane). For example, the coordinates of the chassis center are recorded as (X0, Y0) as the zero point. In other embodiments, the zero position can be determined according to the actual operating environment of the robot, which is not limited here.

[0069] It should be noted that, in this application, the robot's zero position refers to a reference point or benchmark point set in the robot's own coordinate system. This point is usually a fixed and easily identifiable position on the robot's structure, used as the starting reference for all motion, position calculation and navigation control.

[0070] In some embodiments, reference Figure 3 As shown in the figure, this is an exemplary flowchart illustrating the determination of the identification deviation range during robot path recognition according to some embodiments of this application. In this embodiment, the body zero position is configured with deviation to obtain the identification deviation range during robot path recognition, which can be achieved by the following steps:

[0071] In step 1031, the zero-position deviation range of the robot is set;

[0072] In step 1032, the deviation characteristics of the robot when performing path recognition are determined;

[0073] In step 1033, the degree of freedom for path recognition by the robot is determined based on the deviation characteristics.

[0074] In step 1034, the identification degree of freedom and the zero-position deviation interval are matched to obtain the identification deviation interval for robot path recognition.

[0075] In specific implementation, firstly, based on the design accuracy of the robot body and actual measurement data, a preliminary deviation range is defined. This preliminary deviation range can be influenced by factors such as robot manufacturing tolerances and sensor errors. For example, the zero-position deviation range can be set to ±0.05 meters. The defined preliminary deviation range is then recorded in the robot's control system, serving as the robot's zero-position deviation range. In other embodiments, other methods can be used to determine the robot's zero-position deviation range, which is not limited here. Secondly, data is collected in different path recognition scenarios to record the robot's deviation during actual operation. For example, sensors can be used to record the robot's actual deviation data on different paths, such as speed changes and steering errors. The collected data is analyzed to determine the main deviation characteristics during path recognition, such as error patterns under specific environments and sensor response characteristics. The analysis of the collected data can employ algorithms such as linear regression and hierarchical clustering. Based on the analysis results, specific deviation characteristics are determined, such as the direction, magnitude, and trend of the deviation (determining that in path recognition, the deviation range of the robot's forward direction is ±0.03 meters, and the deviation range of the steering angle is ±2 degrees).

[0076] In addition, in specific implementation, the degree of freedom for robot path recognition can be determined based on the deviation characteristics as follows: collect deviation characteristic data in actual path recognition, including positional deviation (e.g., the difference between the robot's actual position and the target position) and angular deviation (e.g., the difference between the robot's actual orientation and the desired orientation). Based on the collected data, define the characteristics and range of each deviation, typically including the maximum deviation value and the distribution characteristics of the deviation. Then, determine the robot's degree of freedom based on the maximum deviation value. For example, if the maximum positional deviation is ±0.03 meters, then the degree of freedom for positional deviation is 0.03 meters; if the maximum angular deviation is ±2 degrees, then the degree of freedom for angular deviation is 2 degrees. Thus, the degree of freedom for robot path recognition is obtained. In other embodiments, other methods can also be used to determine the degree of freedom for robot path recognition, which is not limited here.

[0077] In addition, in specific implementation, matching the identification degrees of freedom and the zero-position deviation interval to obtain the identification deviation interval for robot path recognition can be achieved in the following way: First, compare the identification degrees of freedom and the zero-position deviation interval. If the degrees of freedom and the deviation interval do not match perfectly, make adjustments to ensure that all actual deviations are covered. For example, if the zero-position deviation interval is ±0.05 meters and the identification degrees of freedom is ±0.03 meters, the final identification deviation interval can be adjusted to ±0.05 meters to cover all deviations. Then, set the identification deviation interval to ±0.05 meters to accommodate all possible actual errors, thus obtaining the identification deviation interval for robot path recognition. In other embodiments, other methods can be used to determine the identification deviation interval for robot path recognition, which is not limited here.

[0078] It should be noted that, in this application, the zero-position deviation range represents the allowable error range between the zero position (i.e., its theoretically expected position or posture) of a robot or other mechanical system and its actual measured position or posture. This range indicates the acceptable zero-position deviation for the robot in actual operation, thereby ensuring that the system can work normally near the zero position. The deviation characteristic represents the error characteristics of the actual value relative to the theoretical or standard value during the visual recognition process. This characteristic includes the magnitude, direction, frequency, distribution pattern, etc. of the deviation, used to describe and analyze the possible errors in the system for compensation or correction. The identification degree of freedom represents the error range or allowable deviation degree that the robot can tolerate when performing path recognition and tracking, including position error, angle error, and other related measurement errors. This identification degree of freedom ensures that the robot can effectively identify and adjust the path within a predetermined error range in practical applications, thereby achieving accurate path tracking and navigation. The identification deviation range represents the allowable error range or deviation range during the robot's path recognition and tracking process, including the maximum position deviation, angle deviation, etc., that the robot may experience when actually recognizing the path, to ensure that the robot can accurately identify and track the path within these error ranges.

[0079] In some embodiments, determining the dynamic compensation amount for visual recognition based on the identification deviation range and the robot's real-time path trajectory can be achieved using the following steps:

[0080] Obtain the robot's real-time path trajectory;

[0081] The dynamic compensation factor for the robot during operation is determined based on the identification deviation range.

[0082] The dynamic compensation amount for visual recognition is determined based on the dynamic compensation factor and the real-time path.

[0083] In practice, sensors on the robot (such as GPS, IMU, LiDAR, or cameras) are used to monitor the robot's movement trajectory in real time. These sensors record data such as the robot's position, speed, acceleration, and direction. The real-time monitored movement trajectory is used as the robot's real-time path trajectory and stored in the robot's storage unit. The robot's real-time path trajectory is obtained by reading the robot's storage unit. In other embodiments, other methods can also be used to determine the robot's real-time path trajectory, which is not limited here.

[0084] It should be noted that the real-time path trajectory of the robot in this application refers to the continuous data of the robot's current position and direction of movement recorded in a time sequence during operation. This data reflects the path traversed by the robot within a specific time period, including its position changes, direction of travel, speed, and other motion characteristics. The real-time path trajectory can capture the robot's movement state in real time and is an important basis for navigation, path tracking, and dynamic adjustment.

[0085] In addition, in specific implementation, firstly, based on the previously defined identification deviation range (e.g., position deviation ±0.1 meters, angle deviation ±3 degrees), the allowable error range is calculated. That is, there is a relationship between position and angle deviations, which can be combined using geometric or weighted methods. For example, during path tracking, when the position deviation is large, the allowable angle deviation may need to be more stringent to maintain path consistency. When combining position and angle deviations, weighting coefficients can be set based on expert experience or relevant data on robot vision recognition. Then, the angle error and position error are multiplied by their respective weighting coefficients and summed. The square root of the sum is then used as the basis for calculation. In other embodiments, the dynamic compensation factor of the robot during operation can also be determined based on changes in the path (such as curvature, speed changes, etc.), which is not limited here. Then, the dynamic compensation factor is compared with the position data and angle data in the real-time path, and the ratio is processed to be dimensionless. After placing all the dimensionless ratios in a two-dimensional coordinate system, two smooth curves are obtained, namely the position curve and angle curve of the robot in real-time movement. The sum of the maximum slopes of the two curves is used as the dynamic compensation amount for visual recognition. In other embodiments, other methods can also be used to determine the dynamic compensation amount for visual recognition, which is not limited here.

[0086] It should be noted that, in this application, the dynamic compensation factor represents a calculated parameter used to correct real-time path deviations during robot vision recognition and path planning. It is determined based on the difference between the robot's current actual motion state (such as position, speed, angle, etc.) and the predetermined target. By applying the dynamic compensation factor, the robot can adjust its motion trajectory in real time, thereby reducing or eliminating deviations in path recognition and ensuring that the robot can more accurately follow the predetermined path or perform specific tasks. The dynamic compensation amount during vision recognition represents the specific adjustment amount calculated based on the real-time path deviation and the dynamic compensation factor during robot vision recognition. It is used to correct the robot's errors in recognizing and tracking paths in real time, ensuring that the robot can accurately recognize and move along the predetermined path. The dynamic compensation amount can include adjustments in position, angle, etc. By applying it to the robot's motion control, the accuracy and reliability of vision recognition can be improved, thereby optimizing the robot's path recognition and navigation capabilities in complex environments.

[0087] In step 104, the edge of the running path in the robot's visual recognition is obtained, and the static offset domain when the robot performs visual recognition of the running path is determined based on the edge of the running path. Then, the recognition loss value during visual recognition is determined by the static offset domain and the label corner point.

[0088] In specific implementation, the edges of the running path in robot visual recognition can be obtained in the following way: First, an environmental image is acquired using a camera and preprocessed to improve image quality; then, an edge detection algorithm is applied to identify the path edges in the image, extract the path contour, and eliminate noise and anomalies through verification and filtering to ensure the continuity and accuracy of the edges; finally, the processed path edge information is recorded as reference data for path tracking, and monitored and adjusted in real time during actual operation to cope with environmental changes and ensure that the robot can accurately move along the predetermined path. In other embodiments, other methods can also be used to obtain the edges of the running path in robot visual recognition, which are not limited here.

[0089] It should be noted that the running path edge in the robot vision recognition in this application represents the boundary of the robot's required tracking path detected and extracted by the vision sensor. This edge information helps the robot identify and stay within the specified path and is an important reference point for navigation during path recognition and tracking.

[0090] In some embodiments, determining the static offset domain for the robot to perform visual recognition of the running path based on the running path edge can be achieved by the following steps:

[0091] The continuous offset baseline is determined based on the edge of the running path when the robot performs visual recognition of the running path.

[0092] Determine the long-tail dataset when the robot visually recognizes its running path;

[0093] The static offset domain is determined based on the continuous offset baseline and the long-tail dataset when the robot performs visual recognition of the running path.

[0094] In practice, firstly, by analyzing the edges of the running path acquired during the robot's visual recognition process, the offset between the robot's actual position on the path and the ideal path is determined. This offset can be obtained by calculating the distance between the path centerline and the robot's actual position. All offsets are arranged in chronological order, and the arranged offsets are used as the continuous offset baseline for the robot's visual recognition of the running path. Then, during the robot's path recognition process, data points with large offsets but low frequency of occurrence (i.e., the so-called "long-tail" data) are identified. This data may represent abnormal offsets of the robot under certain specific conditions (such as complex road conditions, occlusion, or sudden environmental changes). Identifying data points with large offsets but low frequency of occurrence can be achieved using algorithms such as the Isolation Forest algorithm or the Local Anomaly Factor algorithm, which will not be elaborated here. For example, the robot's offset increases significantly when turning, but this situation does not occur frequently. Such data can be classified into the long-tail dataset for analyzing special offset patterns during turns. Finally, based on the determined continuous offset baseline and the long-tail dataset, the static offset domain of the robot on the running path is calculated and determined, that is, the continuous offset is calculated. The baseline's mean and standard deviation are used. The mean represents the average offset under normal conditions, while the standard deviation indicates the range of offset variation. For example, if the calculated mean is 0.02 meters and the standard deviation is 0.05 meters, the offset range under normal conditions is determined based on the mean and standard deviation. For instance, the normal offset range can be set as the mean ± 2 times the standard deviation (0.02 ± 2 × 0.05), which is between -0.08 meters and 0.12 meters. Next, the offsets in the long-tail dataset are analyzed to identify the maximum offset under rare conditions. For example, in the long-tail dataset, it may be found that the offset varies significantly in certain situations. The offset can reach ±0.15 meters. Based on the offset domain under normal conditions and the extreme cases in the long-tail data, the static offset domain is determined comprehensively. For example, if the offset domain under normal conditions is -0.08 meters to 0.12 meters, but considering that the maximum offset in the long-tail data can reach ±0.15 meters, the static offset domain can be set to -0.15 meters to 0.15 meters. This is the static offset domain when the robot performs visual recognition of the running path. In other embodiments, other methods can also be used to determine the static offset domain when the robot performs visual recognition of the running path. This is not limited here.

[0095] It should be noted that, in this application, the continuous offset baseline refers to the baseline of a continuous series of offsets recorded by the robot during the visual recognition of the running path. This offset reflects the actual positional deviation of the robot relative to the ideal path during its movement. The long-tail dataset refers to the set of offset data recorded by the robot during the visual recognition of the running path in rare or extreme cases. In path recognition, the long-tail dataset helps to identify and analyze the robot's offset behavior in atypical scenarios, thereby enhancing the robustness of the system. The static offset domain refers to the fixed offset range determined when the robot performs visual recognition of the running path. Within this range, the robot can deviate from the ideal path without affecting normal operation or triggering corrective measures. The static offset domain is used to determine the allowable path deviation of the robot during visual recognition, ensuring that the robot can maintain stable operation when dealing with atypical situations and avoiding frequent triggering of unnecessary corrective measures due to small offsets.

[0096] In some embodiments, determining the recognition loss value for visual recognition using the static offset domain and the label corner points can be achieved through the following steps:

[0097] The recognition concentration during robot vision recognition is determined based on the static offset domain.

[0098] The recognition discreteness during robot vision recognition is determined based on the corner points of the label.

[0099] The recognition loss value during visual recognition is determined by the recognition concentration and the recognition discreteness.

[0100] It should be noted that in this application, the concentration of recognition refers to the degree of overlap between the path recognition result and the expected path during the robot's visual recognition process. The static offset domain provides the allowable offset range. The higher the concentration, the closer the robot's recognition result on the path is to the expectation. The dispersion of recognition refers to the dispersion of the path recognition result during the robot's visual recognition process, that is, the degree of deviation from the ideal path.

[0101] In practical implementation, firstly, based on the size of the static offset domain, the proportion of the robot's actual path points falling within the static offset domain during visual recognition is statistically analyzed. This proportion is used as the recognition concentration during robot visual recognition. For example, if the static offset domain is ±0.1 meters, and 80% of the path points fall within this range during recognition, then the recognition concentration is 0.8. Secondly, by analyzing the positional deviations of each label corner point relative to the expected corner point in the recognition results, this positional deviation is used as the recognition dispersion during robot visual recognition. For example, if a label corner point deviates from the expected corner point by 0.05 meters at a turn, then the recognition dispersion can be calculated by summing all deviations. The variance is used as the recognition discrete quantity during robot visual recognition. Finally, the loss value can be determined by calculating the difference between the lumped quantity and the discrete quantity. The larger the loss value, the worse the recognition effect, and the smaller the loss value, the better the recognition effect. The recognition loss value can be determined as follows: Recognition loss value = 1 - (Recognition lumped quantity - Recognition discrete quantity). For example, if the robot's recognition lumped quantity on a straight path is 0.85, but the discrete quantity at a critical turn is 0.2, then the recognition loss value may be 0.35. In other embodiments, other methods can also be used to determine the recognition loss value during visual recognition, which is not limited here.

[0102] It should be noted that the recognition loss value in this application represents a numerical measure used to quantify the degree to which the robot deviates from the ideal path during path recognition. It reflects the degree of error or deviation that occurs during the recognition process.

[0103] In step 105, the visual adaptability of the robot in path recognition is determined by the dynamic compensation amount and the recognition loss value, and the path visually recognized by the robot is dynamically adjusted according to the visual adaptability.

[0104] In some embodiments, determining the robot's visual adaptability to path recognition using the dynamic compensation amount and the recognition loss value can be achieved through the following steps:

[0105] The first adaptive component for the robot's path recognition is determined based on the dynamic compensation amount;

[0106] The second adaptive component for the robot's path recognition is determined based on the recognition loss.

[0107] The visual adaptation of the robot during path recognition is determined based on the first adaptation component and the second adaptation component.

[0108] In practice, firstly, the dynamic compensation value is transformed into a first adaptation component. This component reflects the robot's flexibility and reaction speed in changing environments. Typically, through normalization or standardization methods, the dynamic compensation value is converted into a value between 0 and 1, where 0 represents complete inability to adapt and 1 represents high adaptation. This yields the robot's first adaptation component for path recognition. Then, the recognition loss value is transformed into a second adaptation component. This component reflects the robot's path recognition accuracy in different environments. Similarly, through normalization or standardization, the recognition loss value is converted into a value between 0 and 1, where 0 represents extremely poor recognition performance. A value of 1 indicates an ideal recognition effect. Finally, based on the first and second adaptation components, the overall visual fitness of the robot is calculated. A weighted average or product model can be used, i.e., visual fitness = first adaptation component * second adaptation component. For example, if the robot's dynamic compensation in a dynamic environment is 0.8 (flexible response) and the second adaptation component corresponding to the recognition loss value is 0.6 (relatively accurate recognition), then the final fitness is 0.8 × 0.6 = 0.48. This means that the robot's visual fitness in the current environment is 48%, indicating that there is still room for optimization in terms of coping with path changes and recognition accuracy.

[0109] It should be noted that, in this application, the first adaptation component represents a metric of the adaptability evaluated based on the robot's dynamic compensation during path recognition. It reflects the robot's ability to adapt to environmental changes by adjusting its movement or path planning in a dynamic environment. The second adaptation component represents a metric of the adaptability evaluated based on the robot's recognition loss value during path recognition. It reflects the accuracy of the robot in path deviation during visual recognition, i.e., the gap between the recognition result and the actual path. Visual adaptability represents a metric of the robot's overall adaptability to environmental changes and path recognition challenges in the path recognition task, by comprehensively considering dynamic compensation capability and recognition accuracy. Visual adaptability reflects the robot's ability to adjust its path in real time in a dynamic environment and the accuracy of its path recognition.

[0110] In specific implementation, dynamically adjusting the path visually recognized by the robot based on the visual adaptability can be achieved in the following way: First, based on the calculated visual adaptability, a specific adjustment strategy is formulated. For cases with high adaptability, the robot may only need fine-tuning; while for cases with low adaptability, larger adjustments are required. For example, an adaptability threshold can be set, and when the visual adaptability falls below a certain value, more stringent path adjustment measures are taken. Then, the relationship between visual adaptability and actual path adjustment is determined. The adaptability value can be set to be proportional to the path adjustment amount; that is, the higher the adaptability, the smaller the path adjustment amount; the lower the adaptability, the larger the path adjustment amount. Finally, based on the calculated adjustment amount, the robot's path is actually adjusted, including modifying the robot's control commands, such as adjusting direction, changing speed, and replanning the path, to ensure that the robot maintains stable movement during adjustments. To avoid sudden actions affecting its stability, the adjusted results are fed back into the visual adaptability calculation so that the dynamic adjustment strategy can be optimized. If the robot still has path deviations, the visual adaptability and path adjustment strategy are further adjusted. For example, if the robot's current visual adaptability is 0.3 (low), it indicates that its path recognition and dynamic compensation capabilities are poor. According to the set adjustment strategy, it may be necessary to significantly correct the path. If the robot detects a deviation of 0.5 meters from the target path and a directional deviation of 10 degrees during operation, according to the adjustment strategy, it is calculated that the path needs to be corrected by 0.5 meters and the direction needs to be adjusted by 10 degrees. The robot replans the path based on these adjustment instructions and makes real-time corrections.

[0111] Furthermore, in another aspect of this application, in some embodiments, this application provides a robot vision recognition system, with reference to... Figure 4 The figure is a schematic diagram of exemplary hardware and / or software of a robot vision recognition system according to some embodiments of this application. The robot vision recognition system 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below:

[0112] The acquisition module 401 in this application is mainly used to acquire path recognition information collected by the robot during operation;

[0113] Processing module 402, in this application, is mainly used to match and filter the path recognition information to obtain the path matching area during visual recognition, and to determine the visual label corner points in visual recognition based on the path matching area and the dynamic operation characteristics of the robot.

[0114] The processing module 402 is also used to mark the zero position of the robot body, configure the deviation of the zero position of the body to obtain the identification deviation range when the robot path is identified, and determine the dynamic compensation amount when visual recognition based on the identification deviation range and the real-time path trajectory of the robot.

[0115] The processing module 402 is further configured to acquire the edge of the running path in robot visual recognition, determine the static offset domain when the robot performs visual recognition of the running path based on the edge of the running path, and then determine the recognition loss value when performing visual recognition based on the static offset domain and the label corner point.

[0116] The execution module 403 in this application is mainly used to determine the visual adaptability of the robot when recognizing the path through the dynamic compensation amount and the recognition loss value, and to dynamically adjust the path visually recognized by the robot based on the visual adaptability.

[0117] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the robot vision recognition method described above.

[0118] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a robot vision recognition method according to some embodiments of this application. The robot vision recognition method in the above embodiments can... Figure 5 The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0119] The processor 501 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the robot vision recognition method in this application.

[0120] The communication bus 502 can be used to transmit information between the aforementioned components.

[0121] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0122] The memory 503 stores program code for executing the scheme of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. In the above embodiments, the robot vision recognition method can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0123] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0124] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0125] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0126] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described robot vision recognition method.

[0127] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0128] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A robot visual recognition method, characterized in that, Includes the following steps: The path recognition information collected by the robot during operation is obtained, and the path recognition information represents the set of environmental features, path shape and obstacle positions collected by the robot during operation; The path recognition information is matched and filtered to obtain the path matching area during visual recognition. Based on the path matching area and the dynamic operation characteristics of the robot, the visual label corner points in visual recognition are determined. The visual label corner points represent key feature points that are identified and marked in the image or video data during visual recognition. The robot's body zero position is marked, which represents a reference point or benchmark point set in the robot's own coordinate system. The body zero position is configured with deviation to obtain the identification deviation range when the robot recognizes the path. The identification deviation range represents the allowable error range or deviation range during the robot's path recognition and tracking process. The dynamic compensation amount for visual recognition is determined based on the identification deviation range and the robot's real-time path trajectory. The edge of the running path in the robot's visual recognition is obtained, and the static offset domain is determined when the robot performs visual recognition of the running path based on the edge of the running path. The static offset domain represents the fixed offset range determined when the robot performs visual recognition of the running path. Then, the recognition loss value during visual recognition is determined by the static offset domain and the label corner point. The visual adaptability of the robot in path recognition is determined by the dynamic compensation amount and the recognition loss value. The visual adaptability is a measure of the robot's overall adaptability to environmental changes and path recognition challenges. The path visually recognized by the robot is dynamically adjusted based on the visual adaptability. Specifically, the recognition loss value determined by the static offset domain and the label corner points during visual recognition includes: The recognition concentration is determined based on the static offset domain. The recognition concentration refers to the degree of overlap between the path recognition result and the expected path during the robot's visual recognition process. The recognition discreteness is determined based on the corner points of the label, whereby the recognition discreteness refers to the discreteness of the path recognition result during the robot's visual recognition process. The recognition loss value during visual recognition is determined by the recognition concentration and the recognition discreteness. Specifically, determining the robot's visual adaptability during path recognition through the dynamic compensation amount and the recognition loss value includes: The first adaptive component of the robot during path recognition is determined based on the dynamic compensation amount, that is, the dynamic compensation amount is normalized or standardized to obtain the first adaptive component. The first adaptive component represents the measurement value of the adaptive ability evaluated based on the robot's dynamic compensation amount during the path recognition process. The second adaptation component of the robot is determined based on the recognition loss value, that is, the recognition loss value is normalized or standardized to obtain the second adaptation component. The second adaptation component represents the measurement value of the adaptation ability evaluated based on the robot's recognition loss value during the path recognition process. The visual adaptation of the robot during path recognition is determined based on the first adaptation component and the second adaptation component.

2. The method as described in claim 1, characterized in that, The path recognition information is matched and filtered to obtain the path matching region for visual recognition, specifically including: Extract the path label from the path recognition information when the robot performs visual recognition; Determine the path prediction value during robot vision recognition. The path prediction value represents the estimated value of the robot's future travel path during path recognition and navigation, based on the robot's current state, historical path data, and environmental information. The path prediction value includes the predicted direction, position, speed, and probability of possible path selection. The path labels and path predictions are used for feature filtering to obtain the path matching region for visual recognition.

3. The method as described in claim 1, characterized in that, The dynamic compensation amount for visual recognition, determined based on the identification deviation range and the robot's real-time path trajectory, specifically includes: Obtain the robot's real-time path trajectory; The dynamic compensation factor for the robot during operation is determined based on the identification deviation range. The dynamic compensation factor is a calculated parameter used to correct real-time path deviation during robot vision recognition and path planning. It is determined based on the difference between the robot's current actual motion state and the predetermined target. The dynamic compensation amount for visual recognition is determined based on the dynamic compensation factor and the real-time path.

4. The method as described in claim 1, characterized in that, The static offset domain determined based on the edge of the running path for visual recognition by the robot specifically includes: The continuous offset baseline is determined based on the edge of the running path when the robot performs visual recognition of the running path. The continuous offset baseline represents the baseline of a series of continuous offsets recorded by the robot during the visual recognition of the running path. The offset reflects the actual position deviation of the robot relative to the ideal path during the movement. Determine the long-tail dataset when the robot visually recognizes its running path; The static offset domain is determined based on the continuous offset baseline and the long-tail dataset when the robot performs visual recognition of the running path.

5. The method as described in claim 1, characterized in that, The path recognition information of the robot during operation is obtained by reading the robot's data storage unit.

6. A robot vision recognition system, which performs robot vision recognition using the method described in any one of claims 1 to 5, characterized in that, The system includes: The acquisition module is used to acquire path recognition information collected by the robot during operation; The processing module is used to match and filter the path recognition information to obtain the path matching area during visual recognition, and to determine the visual label corner points in visual recognition based on the path matching area and the dynamic operation characteristics of the robot. The processing module is also used to mark the zero position of the robot body, configure the deviation of the zero position of the body to obtain the identification deviation range when the robot path is recognized, and determine the dynamic compensation amount when visual recognition based on the identification deviation range and the real-time path trajectory of the robot. The processing module is also used to obtain the edge of the running path in robot visual recognition, determine the static offset domain when the robot performs visual recognition of the running path based on the edge of the running path, and then determine the recognition loss value when performing visual recognition based on the static offset domain and the label corner point. The execution module is used to determine the robot's visual adaptation to path recognition by the dynamic compensation amount and the recognition loss value, and to dynamically adjust the path visually recognized by the robot based on the visual adaptation.

7. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to retrieve the code and execute the robot vision recognition method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot vision recognition method as described in any one of claims 1 to 5.

Images