Robot visual identification method, system and device and storage medium

Abstract

The invention provides a robot visual identification method, system and device and a storage medium. The method comprises the following steps: determining visual label angular points in visual identification according to a path matching region and dynamic operation characteristics of a robot; determining a dynamic compensation amount during visual identification according to the identification deviation interval and the real-time path track of the robot, determining a static offset domain when the robot performs visual identification on the operation path according to the edge of the operation path, and further determining an identification loss value during visual identification according to the static offset domain and the label angular point; and determining the visual fitness of the robot during path identification through the dynamic compensation amount and the identification loss value, and dynamically adjusting the path visually identified by the robot according to the visual fitness. According to the scheme, on the basis of the visual fitness, dynamic matching of the identification path can be achieved when robot visual identification is interfered by tracking freedom degree missing, and therefore the identification capacity of the robot on the running path is improved.

Description

Technical Field

[0001] The present application relates to the field of visual recognition technology, and more specifically, to a robot visual recognition method, system, device and storage medium. Background Art

[0002] Visual recognition refers to the process of using computer vision and image processing technology to automatically detect, analyze and understand objects, scenes or features from images or videos, and obtain environmental information through cameras or other visual sensors. The visual recognition system uses a series of algorithms, including image preprocessing, feature extraction, target detection and classification, to identify and interpret the content in the image. With the support of deep learning, this technology can handle complex visual tasks such as face recognition, object detection, scene understanding, etc. Visual recognition is widely used in fields such as autonomous driving, robot navigation, security monitoring and intelligent manufacturing, which improves the automation and intelligence level of the system.

[0003] In the prior art, the visual recognition of robots is usually achieved by integrating high-precision cameras and other visual sensors, which are responsible for capturing images or video data of the surrounding environment. Subsequently, the collected visual information is preprocessed using computer vision algorithms to improve image quality and highlight key features. Next, key information such as edges, corners, textures, etc. in the image are identified through feature extraction technology, and these features are analyzed and classified using pattern recognition and machine learning algorithms to identify and understand objects, paths, and other important elements in the environment. However, in the visual recognition of robots, due to the influence of lighting changes, occlusion, background complexity, blurry or uncommon path features, etc., the path information identified by the robot during the visual recognition of the running path is There will be a problem of lack of tracking freedom, where tracking freedom refers to the ability of the robot to effectively track and identify its running path through sensors and algorithms during the visual recognition process. When performing visual recognition, the robot has the flexibility and accuracy to accurately follow the predetermined path under different environmental conditions (such as lighting changes, occlusions, background complexity, path blur or uncommon features, etc.). Therefore, the lack of tracking freedom will cause the robot to accumulate errors in visual recognition, resulting in the weakening of the robot's ability to identify the running path, thereby reducing the robot's dynamic matching ability during visual recognition; therefore, how to achieve dynamic matching of the recognition path when the robot's visual recognition is disturbed by the lack of tracking freedom, thereby improving the robot's ability to identify the running path has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a robot vision recognition method, system, device and storage medium, which can realize dynamic matching of recognition paths when the robot vision recognition is disturbed by the lack of tracking freedom, thereby improving the robot's ability to recognize the running path.

[0005] In a first aspect, the present application provides a robot visual recognition method, comprising the following steps: Obtain path recognition information collected by the robot during operation; Matching and screening the path recognition information to obtain a path matching area during visual recognition, and determining a visual label corner point in visual recognition according to the path matching area and the dynamic operation characteristics of the robot; Marking the robot's body zero position, performing deviation configuration on the body zero position, obtaining an identification deviation interval during robot path identification, and determining a dynamic compensation amount during visual identification according to the identification deviation interval and the robot's real-time path trajectory; Acquire the running path edge in the robot visual recognition, determine the static offset domain when the robot visually recognizes the running path according to the running path edge, and then determine the recognition loss value during visual recognition according to the static offset domain and the label corner point; The visual fitness of the robot during path recognition is determined by the dynamic compensation amount and the recognition loss value, and the path visually recognized by the robot is dynamically adjusted according to the visual fitness.

[0006] In some embodiments, the path identification information is matched and screened to obtain a path matching area during visual identification, specifically including: Extracting a path label when the robot performs visual recognition from the path recognition information; Determine the path prediction quantity during robot vision recognition; The path label and the path prediction quantity are subjected to feature screening to obtain a path matching area during visual recognition.

[0007] In some embodiments, determining the dynamic compensation amount during visual recognition according to the recognition deviation interval and the real-time path trajectory of the robot specifically includes: Get the real-time path trajectory of the robot; Determining a dynamic compensation factor of the robot during operation according to the identified deviation interval; The amount of dynamic compensation during visual recognition is determined according to the dynamic compensation factor and the real-time path.

[0008] In some embodiments, determining the static offset domain when the robot visually identifies the running path according to the running path edge specifically includes: Determine a continuous offset baseline when the robot visually identifies the running path according to the running path edge; Determine the long-tail dataset when the robot performs visual recognition of the running path; A static offset domain when the robot performs visual recognition on a running path is determined according to the continuous offset baseline and the long-tail data set.

[0009] In some embodiments, determining the recognition loss value during visual recognition by the static offset domain and the label corner point specifically includes: Determining the recognition concentration of the robot during visual recognition according to the static offset domain; Determine the recognition discrete amount during robot visual recognition according to the label corner points; The recognition loss value during visual recognition is determined by the recognition concentration and the recognition dispersion.

[0010] In some embodiments, determining the visual adaptability of the robot during path recognition by using the dynamic compensation amount and the recognition loss value specifically includes: Determine a first adaptation component of the robot when identifying a path according to the dynamic compensation amount; Determining a second adaptation component of the robot when identifying a path according to the recognition loss degree; The visual adaptability of the robot during path recognition is determined according to the first adaptability component and the second adaptability component.

[0011] In some embodiments, the path identification information of the robot during operation is obtained by reading the robot data storage unit.

[0012] In a second aspect, the present application provides a robot visual recognition system, comprising: An acquisition module is used to obtain path recognition information collected by the robot during operation; A processing module, used for matching and screening the path recognition information to obtain a path matching area during visual recognition, and determining a visual label corner point in visual recognition according to the path matching area and the dynamic operation characteristics of the robot; The processing module is further used to mark the robot's body zero position, perform deviation configuration on the body zero position, obtain an identification deviation interval during robot path identification, and determine a dynamic compensation amount during visual identification based on the identification deviation interval and the robot's real-time path trajectory; The processing module is further used to obtain the running path edge in the robot's visual recognition, determine the static offset domain when the robot visually recognizes the running path according to the running path edge, and then determine the recognition loss value during visual recognition according to the static offset domain and the label corner point; The execution module is used to determine the visual fitness of the robot when identifying a path through the dynamic compensation amount and the recognition loss value, and dynamically adjust the path visually recognized by the robot according to the visual fitness.

[0013] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned robot visual recognition method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned robot visual recognition method is implemented.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: In the robot visual recognition method, system, device and storage medium provided by the present application, first, the path recognition information collected by the robot during operation is obtained; secondly, the path recognition information is matched and screened to obtain the path matching area during visual recognition, and the visual label corner points in the visual recognition are determined according to the path matching area and the dynamic operation characteristics of the robot; then, the robot's body zero position is marked, and the body zero position is deviation-configured to obtain the recognition deviation interval during the robot's path recognition, and the dynamic compensation amount during visual recognition is determined according to the recognition deviation interval and the real-time path trajectory of the robot; then, the running path edge in the robot's visual recognition is obtained, and the static offset domain of the robot when visually identifying the running path is determined according to the running path edge, and then the recognition loss value during visual recognition is determined according to the static offset domain and the label corner points; finally, the visual fitness of the robot during path recognition is determined by the dynamic compensation amount and the recognition loss value, and the path visually recognized by the robot is dynamically adjusted according to the visual fitness.

[0016] It can be seen that in the present application, dynamic path matching of the robot can be achieved when the visual recognition is disturbed by the lack of tracking freedom, thereby improving the robot's ability to identify the running path; first, in order to correct the robot's errors in identifying and tracking the path in real time, the dynamic compensation amount during visual recognition is determined according to the identification deviation interval and the real-time path trajectory of the robot, so as to ensure that the robot can accurately identify and move along the predetermined path, and optimize the robot's path recognition and navigation capabilities in complex environments; secondly, in order to reflect the degree of error or deviation occurring in the visual recognition process, the recognition loss value during visual recognition is determined by the static offset domain and the label corner point, and the recognition loss value can measure the robot's deviation from the ideal path during path recognition. The degree of the path; then, the visual adaptability of the robot in path recognition is determined by the dynamic compensation amount and the recognition loss value, aiming to adapt to the environmental changes in the robot's visual recognition by adjusting its movement or path planning, while improving the robot's ability to adjust the path in real time in a dynamic environment, and effectively avoiding the loss of tracking freedom caused by errors in the robot's identification and tracking of the path and errors in the visual recognition process; finally, the path visually recognized by the robot is dynamically adjusted according to the visual adaptability; in summary, the technical solution provided by the present application can realize dynamic matching of the recognition path when the robot's visual recognition is disturbed by the lack of tracking freedom, thereby improving the robot's ability to recognize the running path. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an exemplary flow chart of a robot visual recognition method according to some embodiments of the present application; Figure 2 is a schematic diagram of an exemplary process of determining visual label corner points in visual recognition according to some embodiments of the present application; Figure 3 is a schematic diagram of an exemplary process for determining an identification deviation interval when identifying a robot path according to some embodiments of the present application; Figure 4 is a schematic diagram of exemplary hardware and / or software of a robot visual recognition system according to some embodiments of the present application; Figure 5 It is a structural schematic diagram of a computer device for implementing a robot visual recognition method according to some embodiments of the present application. DETAILED DESCRIPTION

[0018] The core of this application is to first obtain the path recognition information collected by the robot during operation; secondly, match and screen the path recognition information to obtain the path matching area during visual recognition, and determine the visual label corner points in visual recognition according to the path matching area and the dynamic operation characteristics of the robot; then, mark the robot's body zero position, and configure the deviation of the body zero position to obtain the recognition deviation interval during robot path recognition, and determine the dynamic compensation amount during visual recognition according to the recognition deviation interval and the real-time path trajectory of the robot; then, obtain the running path edge in the robot's visual recognition, and determine the static offset domain when the robot visually recognizes the running path according to the running path edge, and then determine the recognition loss value during visual recognition according to the static offset domain and the label corner points; finally, determine the visual fitness of the robot during path recognition through the dynamic compensation amount and the recognition loss value, and dynamically adjust the path recognized by the robot according to the visual fitness. The above scheme is based on visual fitness, and can realize dynamic matching of the recognition path when the robot's visual recognition is disturbed by the lack of tracking freedom, thereby improving the robot's recognition ability of the running path.

[0019] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 , which is an exemplary flow chart of a robot visual recognition method according to some embodiments of the present application. The robot visual recognition method 100 mainly includes the following steps: In step 101, path identification information collected by the robot during operation is obtained.

[0020] In the specific implementation, the visual sensor is installed on the robot to ensure that its vision covers the robot's forward direction and path. Through the sensor's software tool, appropriate parameters such as resolution, frame rate, exposure time, etc. are set to ensure that clear images or videos are obtained under different lighting conditions. The visual sensor can be a high-resolution camera, a depth camera, a lidar, etc., which are not limited here. Through the sensor interface, the path image or video data is continuously collected, and the data stream is transmitted to the robot data storage unit. The path recognition information of the robot during operation can be obtained by reading the robot data storage unit. In other embodiments, other methods can also be used to obtain the path recognition information of the robot during operation, which are not limited here.

[0021] It should be noted that the path recognition information in this application refers to a collection of data such as environmental characteristics, path shape and obstacle position collected by the robot during operation, which aims to provide accurate navigation basis for the robot's visual recognition and ensure that it effectively identifies and tracks the predetermined path. Among them, this collection of data is detailed data about the path extracted and generated from the environment by visual sensors and computer vision algorithms. These data include the path's position, shape, edges, key points, corners, texture features, and dynamic information related to the path (such as real-time changes in the path, obstacle positions, etc.). The path recognition information can guide the robot's navigation and obstacle avoidance, ensuring that the robot can accurately identify, track and adjust its travel path.

[0022] In step 102, the path recognition information is matched and screened to obtain a path matching area during visual recognition, and the visual label corner points in visual recognition are determined according to the path matching area and the dynamic operation characteristics of the robot.

[0023] In some embodiments, matching and screening the path identification information to obtain the path matching area during visual identification can be achieved by using the following steps: Extracting a path label when the robot performs visual recognition from the path recognition information; Determine the path prediction quantity during robot vision recognition; The path label and the path prediction quantity are subjected to feature screening to obtain a path matching area during visual recognition.

[0024] In the specific implementation, first, during the operation of the robot, the image data of the path is obtained through the camera or other visual sensors, and then the image processing algorithm (such as edge detection, corner detection, etc.) is used to extract the significant path features from these images. The features may include the corners, bifurcation points, edge lines, etc. of the path. According to the extracted features, the corresponding path labels are generated. These labels may be numbers, symbols or other identification forms to indicate the key points on the path. For example: in a complex road network, the robot may extract the location of each turning point, intersection or road sign as a path label, that is, in the process of identifying the path, if an intersection is detected, it can be marked with an "X" label; if a right turn is detected, it can be marked with an "R" label; then Then, a big data analysis model, such as a regression analysis model, a clustering model, etc., is used to analyze the path data of the robot in previous operations, and combined with the current operating status, its next possible moving direction or position is inferred, including analysis of factors such as the forward direction, speed, and position deviation. For example, if the robot chooses to turn right many times on the previous path, the right turn direction may also be given priority in the current prediction. Then, based on the analysis results, a path prediction quantity is generated, including a prediction of the possible next node or area on the path. The path prediction quantity can be a specific coordinate range, direction vector or probability distribution. Among them, the process of generating the path prediction quantity can adopt a deep reinforcement learning algorithm or a path planning algorithm, etc. In other embodiments, other methods can also be used to determine the path prediction quantity, which will not be repeated here.

[0025] In addition, in a specific implementation, the path label and the path prediction quantity are subjected to feature screening, and the path matching area during visual recognition can be obtained in the following manner, namely, the path label is compared and screened with the path prediction quantity to determine the path label with the highest matching degree in the path prediction quantity. The screening criteria may include the proximity of geometric position, the consistency of path direction, the feature importance of the label, etc. For example, if the path label "R" indicates a right turn, and the path prediction quantity also tends to turn right, then the two features will match, and this matching area will be given priority. Then, based on the screening results, the path matching area during visual recognition is determined. This area is the optimal prediction area of ​​the actual path of the robot, representing the part of the road that the robot may travel in the current environment. After identifying the path label "X" (intersection) and the prediction quantity "right turn", the area can be used as the path matching area, that is, the path matching area is determined as the part of the road that turns right, and the robot will perform more detailed identification and navigation in this part. In other embodiments, other methods can also be used to determine the path matching area, which is not limited here.

[0026] It should be noted that the path label in the present application refers to an identifier used to identify key feature points or areas on the path during the path recognition process. These labels can represent important elements in the path, such as corners, intersections, end points, starting points, obstacles, etc., and help the robot to make accurate judgments and positioning during visual recognition and path planning; the path prediction quantity refers to the estimated value of the robot's future travel path based on the robot's current state, historical path data and environmental information during the path recognition and navigation process. The path prediction quantity may include information such as predicted direction, position, speed and possible path selection probability; the path matching area refers to the path range or area where the robot may actually travel, which is determined by comparing and screening the path recognition information with the estimated path data during the robot's visual recognition process. This area represents the part of the path that the robot is most likely to follow in the current environment, and usually includes key points, line segments or feature areas on the path.

[0027] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flow chart of determining the visual label corner points in visual recognition according to some embodiments of the present application. In this embodiment, determining the visual label corner points in visual recognition according to the path matching area and the dynamic operation characteristics of the robot can be implemented by the following steps: In step 1021, the path corner points of the robot during visual recognition are determined according to the path matching area; In step 1022, dynamic operation characteristics of the robot are determined; In step 1023, the visual label corner points in visual recognition are determined according to the path corner points and the dynamic operation characteristics.

[0028] In the specific implementation, first, a detailed geometric analysis is performed on the identified path matching area to determine the key features in the area. Usually, path corners appear at the bends of the path, intersections, or significant changes in the edge of the path. Corner detection algorithms (such as Harris corner detection, Shi-Tomasi corner detection, etc.) can be used to identify actual corners in the path matching area. The algorithm can detect significant turning points in the image based on the gradient changes of the image; significant turning points in the image are used as path corners; then, real-time dynamic data is obtained from the robot's sensors (such as accelerometers, gyroscopes, speedometers, etc.). The data includes the robot's current speed, acceleration, direction angle, turning rate, etc. According to the collected dynamic data, the current operating status of the robot is analyzed. For example, it is analyzed whether the robot needs to slow down, turn, or make other adjustments when approaching the path corner, and the current operating status of the robot is used as the robot's dynamic status. dynamic operation characteristics; finally, according to the position of the path corner points and the current dynamic operation characteristics of the robot, determine which corner points need to be marked as label corner points in visual recognition. For example, when the robot approaches a sharp turn at a high speed, the corner points at the turn will be marked as label corner points, prompting the robot to focus on recognition and steering adjustment here. Then, according to the dynamic characteristics, the position and marking method of the label corner points are optimized to ensure that they can be effectively recognized in actual operation. The optimization process may include fine-tuning the corner points to adapt to the actual trajectory and speed of the robot. Finally, the optimized corner points are marked as label corner points in visual recognition, and relevant information (such as coordinates, recognition priority, etc.) is generated for subsequent path recognition and navigation adjustment. The information for generating label corner points may include the specific coordinate position of the corner points, the priority during recognition (such as high-priority corner points may be checked more frequently), and related dynamic adjustment parameters.

[0029] It should be noted that in the present application, the path corner points of the robot during visual recognition represent the key positions in the path with significant turns, intersections or other geometric changes during the visual recognition process; the dynamic operation characteristics of the robot represent the various physical parameters and behavioral characteristics of the robot's real-time state during operation, including but not limited to the robot's speed, acceleration, steering angle, direction change rate, position deviation, posture, etc. These dynamic operation characteristics reflect the robot's movement mode and state in the current environment, and directly affect its path recognition, navigation decision-making and obstacle avoidance capabilities; the labeled corner points in visual recognition represent the key feature points that are identified and marked in the image or video data during visual recognition, and the feature points are usually used as reference points for path recognition and target positioning.

[0030] In step 103, the robot's body zero position is marked, and the body zero position is configured with a deviation to obtain an identification deviation interval during robot path identification, and the dynamic compensation amount during visual identification is determined based on the identification deviation interval and the robot's real-time path trajectory.

[0031] In specific implementation, marking the zero position of the robot body can be achieved in the following manner, namely: a fixed position or point on the robot body is used as a zero position reference point. This point is usually an obvious and easy-to-locate feature point on the robot, which is used as a reference for all motion and position calculations, such as the center of the robot's chassis, the midpoint of the sensor installation position, or a landmark component (such as the center of the control panel); when the robot is started, a position sensor (such as a lidar, a visual sensor, or a GPS) is used to measure the absolute coordinates of the body zero position in the environment. This process can be in the ground coordinate system of the robot's location or other known coordinate systems, and the measured coordinate data is recorded in the robot's control system, including the X, Y, and Z coordinates of the zero position (in three-dimensional space), or only the X and Y coordinates (on a two-dimensional plane). For example, the coordinates of the center of the chassis are recorded as (X0, Y0) as the zero point. In other embodiments, the body zero position can be determined based on the actual operating environment of the robot, which is not limited here.

[0032] It should be noted that the robot's body zero position in the present application refers to a datum point or reference point set in the robot's own coordinate system. This point is usually a fixed and easily identifiable position on the robot structure, which is used as the starting reference for all movements, position calculations and navigation control.

[0033] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flow chart of determining the identification deviation interval when the robot path is identified according to some embodiments of the present application. In this embodiment, the body zero position is configured with deviation, and the identification deviation interval when the robot path is identified can be obtained by the following steps: In step 1031, the zero position deviation interval of the robot is set; In step 1032, the deviation characteristics of the robot when performing path recognition are determined; In step 1033, the recognition degree of freedom of the robot when performing path recognition is determined according to the deviation characteristics; In step 1034, the identification degree of freedom and the zero position deviation interval are matched to obtain the identification deviation interval during robot path recognition.

[0034] In the specific implementation, first, based on the design accuracy of the robot body and the actual measurement data, a preliminary deviation interval is defined, wherein the preliminary deviation interval can be factors such as the robot manufacturing tolerance and the sensor error. For example, the zero-position deviation interval is set to ±0.05 meters, and then the defined preliminary deviation interval is recorded in the control system of the robot, that is, the defined preliminary deviation interval is used as the zero-position deviation interval of the robot. In other embodiments, other methods can also be used to determine the zero-position deviation interval of the robot, which is not limited here; secondly, data is collected in different path recognition scenarios to record the deviation of the robot in actual operation. For example, sensors can be used to record the actual deviation data of the robot traveling on different paths, such as speed changes, steering errors, etc. The collected data is analyzed to determine the main deviation characteristics in the path recognition process, such as error patterns in specific environments, sensor response characteristics, etc., wherein linear regression, hierarchical clustering and other algorithms can be used to analyze the collected data, and specific deviation characteristics, such as the direction, amplitude and change trend of the deviation, are determined according to the analysis results (determined that in path recognition, the deviation range of the robot's forward direction is ±0.03 meters, and the deviation range of the steering angle is ±2 degrees).

[0035] In addition, in a specific implementation, determining the recognition freedom of the robot when performing path identification based on the deviation characteristics can be achieved in the following manner: collecting deviation feature data in actual path identification, including position deviation (for example, the difference between the actual position of the robot and the target position) and angle deviation (for example, the difference between the actual orientation of the robot and the expected orientation), and defining the characteristics and range of each deviation based on the collected data, usually including a maximum deviation value and a distribution characteristic of the deviation, and then determining the recognition freedom of the robot based on the maximum deviation value, for example: if the maximum position deviation is ±0.03 meters, then the degree of freedom of the position deviation is 0.03 meters, and the maximum angle deviation is ±2 degrees, then the degree of freedom of the angle deviation is 2 degrees, that is, the recognition freedom of the robot when performing path identification is obtained. In other embodiments, other methods can also be used to determine the recognition freedom of the robot when performing path identification, which is not limited here.

[0036] In addition, in the specific implementation, the identification degrees of freedom and the zero-position deviation interval are matched to obtain the identification deviation interval during robot path recognition, which can be achieved in the following manner: first, the identification degrees of freedom and the zero-position deviation interval are compared. If the degrees of freedom and the deviation interval do not completely match, adjustments are made to ensure that all actual deviations are covered. For example, the zero-position deviation interval is ±0.05 meters, and the identification degrees of freedom is ±0.03 meters. The final identification deviation interval can be adjusted to ±0.05 meters to cover all deviations. Then, the identification deviation interval is set to ±0.05 meters to accommodate all possible actual errors, thereby obtaining the identification deviation interval during robot path recognition. In other embodiments, other methods can be used to determine the identification deviation interval during robot path recognition, which is not limited here.

[0037] It should be noted that in the present application, the zero-position deviation interval represents the allowable error range between the zero position of the robot or other mechanical system (i.e., the position or posture it should be in theory) and the actual measured position or posture. This range represents the zero-position deviation that the robot can accept in actual operation, thereby ensuring that the system can work normally near the zero position; the deviation feature represents the error characteristics of the actual value relative to the theoretical or standard value in the visual recognition process. The feature includes the size, direction, frequency, distribution pattern, etc. of the deviation, which is used to describe and analyze the errors that may exist in the system for compensation or correction; the identification degree of freedom represents the error range or allowable deviation degree that the robot can tolerate when performing path recognition and tracking, including position error, angle error and other related measurement errors. The identification degree of freedom ensures that the robot can effectively identify and adjust the path within a predetermined error range in actual applications, thereby achieving accurate path tracking and navigation; the identification deviation interval represents the allowable error range or deviation interval during the robot's path recognition and tracking process, including the maximum position deviation, angle deviation, etc. that may occur when the robot actually recognizes the path, so as to ensure that the robot can accurately identify and track the path within these error ranges.

[0038] In some embodiments, determining the dynamic compensation amount during visual recognition according to the recognition deviation interval and the real-time path trajectory of the robot can be implemented by the following steps: Get the real-time path trajectory of the robot; Determining a dynamic compensation factor of the robot during operation according to the identified deviation interval; The amount of dynamic compensation during visual recognition is determined according to the dynamic compensation factor and the real-time path.

[0039] In specific implementation, sensors on the robot (such as GPS, IMU, lidar or camera) are used to monitor the robot's motion trajectory in real time. These sensors will record data such as the robot's position, speed, acceleration and direction. The motion trajectory monitored in real time is used as the robot's real-time path trajectory and stored in the robot's storage unit. The robot's real-time path trajectory is obtained by reading the robot's storage unit. In other embodiments, other methods can also be used to determine the robot's real-time path trajectory, which is not limited here.

[0040] It should be noted that the real-time path trajectory of the robot in this application represents the continuous data of the current position and movement direction of the robot recorded in time series during its operation. The data reflects the path that the robot has passed within a specific time, including its position changes, direction of travel, speed and other movement characteristics. The real-time path trajectory can capture the movement status of the robot in real time and is an important basis for navigation, path tracking and dynamic adjustment.

[0041] In addition, in the specific implementation, first, according to the previously defined identification deviation interval (for example, position deviation ±0.1 meter, angle deviation ±3 degrees), the allowable error range is calculated, that is: there is a relationship between the position and angle deviations, which can be combined by geometric or weighted methods. For example, in the path tracking process, when the position deviation is large, the allowable angle deviation may need to be stricter to maintain path consistency. Among them, when the position deviation and the angle deviation are combined, the weight coefficient can be set by expert experience or relevant information of robot visual recognition, and then the angle error and the position error are multiplied by the corresponding weight coefficients respectively and then added, and the square root of the addition result is obtained, and the square root of the addition result is used as The dynamic compensation factor of the robot during operation can be determined in other embodiments according to the change of the path (such as curvature, speed change, etc.), which is not limited here. Then, the dynamic compensation factor is respectively ratioed with the position data and angle data in the real-time path, and the ratio is dimensionlessly processed. After all the dimensionlessly processed ratios are placed in a two-dimensional coordinate system, two smooth curves are obtained, namely, the position curve and the angle curve of the robot during real-time motion. The sum of the maximum slopes on the two curves is used as the dynamic compensation amount during visual recognition. In other embodiments, other methods can also be used to determine the dynamic compensation amount during visual recognition, which is not limited here.

[0042] It should be noted that the dynamic compensation factor in the present application represents a calculation parameter used to correct the real-time path deviation during the robot's visual recognition and path planning process. It is determined based on the difference between the robot's current actual motion state (such as position, speed, angle, etc.) and the predetermined target. By applying the dynamic compensation factor, the robot can make real-time adjustments to its motion trajectory, thereby reducing or eliminating deviations in path recognition, ensuring that the robot can more accurately follow the predetermined path or perform specific tasks; the dynamic compensation amount during visual recognition represents the specific adjustment amount calculated based on the real-time path deviation and the dynamic compensation factor during the robot's visual recognition process. It is used to correct the robot's errors in identifying and tracking paths in real time to ensure that the robot can accurately identify and move along the predetermined path. The dynamic compensation amount may include adjustments in position, angle, etc. By applying it to the robot's motion control, the accuracy and reliability of visual recognition can be improved, thereby optimizing the robot's path recognition and navigation capabilities in complex environments.

[0043] In step 104, the running path edge in the robot visual recognition is obtained, and the static offset domain of the robot when visually recognizing the running path is determined according to the running path edge, and then the recognition loss value during visual recognition is determined by the static offset domain and the label corner point.

[0044] In specific implementation, the following method can be used to obtain the running path edge in the robot visual recognition, namely: first, use the camera to capture the environment image and perform preprocessing to improve the image quality; then, use the edge detection algorithm to identify the path edge in the image, extract the path contour, and eliminate noise and anomalies through verification and filtering to ensure the continuity and accuracy of the edge; finally, record the processed path edge information as reference data for path tracking, and monitor and adjust it in real time during actual operation to cope with environmental changes and ensure that the robot can accurately move along the predetermined path. In other embodiments, other methods can also be used to obtain the running path edge in the robot visual recognition, which is not limited here.

[0045] It should be noted that the edge of the running path in the robot visual recognition in this application represents the boundary of the path that the robot needs to track detected and extracted by the visual sensor. This edge information helps the robot identify and maintain operation within the specified path, and is an important reference point for navigation during path recognition and tracking.

[0046] In some embodiments, determining the static offset domain when the robot visually identifies the running path according to the running path edge can be implemented by the following steps: Determine a continuous offset baseline when the robot visually identifies the running path according to the running path edge; Determine the long-tail dataset when the robot performs visual recognition of the running path; A static offset domain when the robot performs visual recognition on a running path is determined according to the continuous offset baseline and the long-tail data set.

[0047] In the specific implementation, first, by analyzing the edge of the running path obtained during the robot's visual recognition process, the offset between the actual position of the robot on the path and the ideal path is determined. The offset can be obtained by calculating the distance between the center line of the path and the actual position of the robot. All offsets are arranged in chronological order, and the arranged offsets are used as the continuous offset baseline when the robot visually recognizes the running path; then, in the robot's path recognition process, data points with large offsets but low frequency (the so-called "long-tail" data) are identified. This data may represent abnormal offsets of the robot in certain specific situations (such as complex road conditions, occlusions or sudden environmental changes). Among them, the identification of data points with large offsets but low frequency can be implemented by using an isolated forest algorithm or a local anomaly factor algorithm, which will not be repeated here. For example, the offset of the robot increases significantly when turning, but this situation does not occur frequently. Such data can be classified into a long-tail data set for analyzing special offset patterns when turning; finally, based on the determined continuous offset baseline and long-tail data set, the static offset domain of the robot on the running path is calculated and determined, that is: the continuous offset is calculated The mean and standard deviation of the baseline. The mean represents the average offset under normal circumstances, and the standard deviation represents the range of variation of the offset. For example, the calculated mean is 0.02 meters and the standard deviation is 0.05 meters. Then, based on the mean and standard deviation, the offset domain under normal circumstances is determined. For example, the normal offset domain can be set to mean ± 2 times the standard deviation (0.02 ± 2 × 0.05), that is, between -0.08 meters and 0.12 meters. Then, the offset in the long-tail data set is analyzed to identify the maximum offset in rare cases. For example, in the long-tail data set, it may be found that the offset in some cases It can reach ±0.15 meters, and then based on the offset domain under normal circumstances and the extreme cases in the long-tail data, the static offset domain is comprehensively determined. For example, if the offset domain under normal circumstances is -0.08 meters to 0.12 meters, but considering that the maximum offset in the long-tail data can reach ±0.15 meters, the static offset domain can be set to -0.15 meters to 0.15 meters, that is, the static offset domain when the robot performs visual recognition of the running path is obtained. In other embodiments, other methods can also be used to determine the static offset domain when the robot performs visual recognition of the running path, which is not limited here.

[0048] It should be noted that the continuous offset baseline in the present application refers to the baseline of a continuous series of offsets recorded during the robot's visual recognition of the running path. The offset reflects the actual position deviation of the robot relative to the ideal path during the movement; the long-tail data set refers to the offset data set recorded in rare or extreme cases during the robot's visual recognition of the running path. In path recognition, the long-tail data set helps identify and analyze the offset behavior of the robot in atypical scenarios, thereby enhancing the robustness of the system; the static offset domain refers to the fixed offset range determined when the robot performs visual recognition of the running path. Within this range, the robot can deviate from the ideal path without affecting normal operation or triggering corrective measures. The static offset domain is used to determine the path deviation allowed by the robot during the visual recognition process, to ensure that the robot can maintain stable operation when dealing with atypical situations, and to avoid frequent triggering of unnecessary corrective measures due to small offsets.

[0049] In some embodiments, determining the recognition loss value during visual recognition by using the static offset domain and the label corner point can be implemented by the following steps: Determining the recognition concentration of the robot during visual recognition according to the static offset domain; Determine the recognition discrete amount during robot visual recognition according to the label corner points; The recognition loss value during visual recognition is determined by the recognition concentration and the recognition dispersion.

[0050] It should be noted that the recognition concentration in this application refers to the degree of overlap between the path recognition result and the expected path during the robot's visual recognition process. The static offset domain provides the allowable offset range. The higher the concentration, the closer the robot's recognition result on the path is to the expectation. The recognition discreteness refers to the discreteness of the path recognition result during the robot's visual recognition process, that is, the degree of deviation from the ideal path.

[0051] In the specific implementation, first, according to the size of the static offset domain, the proportion of the robot's actual path points that fall within the static offset domain during the visual recognition process is counted, and this proportion is used as the recognition concentration of the robot's visual recognition. For example, the static offset domain is ±0.1 meters, and 80% of the path points in the recognition process fall within this range, then the recognition concentration is 0.8; then, by analyzing the position deviation of each label corner point relative to the expected corner point in the recognition result, the position deviation is used as the recognition discrete quantity of the robot's visual recognition. For example, at a turn, the label corner point deviates from the expected corner point by 0.05 meters, then the recognition discrete quantity can be obtained by summing up all the deviations. The variance is determined, that is, the variance is used as the recognition discrete amount during robot visual recognition; finally, the loss value can be determined by calculating the difference between the concentration and the discrete amount. The larger the loss value, the worse the recognition effect, and the smaller the loss value, the better the recognition effect. The recognition loss value can be determined in the following way, namely: recognition loss value = 1-(recognition concentration-recognition discrete amount). For example, the recognition concentration of the robot on a straight path is 0.85, but the discrete amount at a key turn is 0.2, then the recognition loss value may be 0.35. In other embodiments, other methods can also be used to determine the recognition loss value during visual recognition, which is not limited here.

[0052] It should be noted that the recognition loss value during visual recognition in this application represents a numerical measurement used to quantify the degree to which the robot deviates from the ideal path during path recognition during the visual recognition process. It reflects the degree of error or deviation that occurs during the recognition process.

[0053] In step 105, the visual fitness of the robot when identifying a path is determined by the dynamic compensation amount and the recognition loss value, and the path visually recognized by the robot is dynamically adjusted according to the visual fitness.

[0054] In some embodiments, determining the visual adaptability of the robot during path recognition by using the dynamic compensation amount and the recognition loss value can be achieved by using the following steps: Determine a first adaptation component of the robot when identifying a path according to the dynamic compensation amount; Determining a second adaptation component of the robot when identifying a path according to the recognition loss degree; The visual adaptability of the robot during path recognition is determined according to the first adaptability component and the second adaptability component.

[0055] In the specific implementation, first, the dynamic compensation amount is converted into the first adaptation component, which reflects the flexibility and reaction speed of the robot in a changing environment. Usually, the dynamic compensation amount is converted into a value between 0 and 1 through normalization or standardization methods, where 0 means no adaptation at all and 1 means high adaptation, that is, the first adaptation component of the robot's path recognition is obtained; then, the recognition loss value is converted into the second adaptation component, which reflects the accuracy of the robot's path recognition in different environments. Similarly, the recognition loss value is converted into a value between 0 and 1 through normalization or standardization, where 0 means extremely poor recognition effect. 1 indicates that the recognition effect is ideal. Finally, the overall visual fitness of the robot is calculated based on the first fitness component and the second fitness component. The weighted average or product model can be used, that is: visual fitness = first fitness component * second fitness component. For example, the dynamic compensation of the robot in a dynamic environment is 0.8 (flexible response), and the second fitness component corresponding to the recognition loss value is 0.6 (relatively accurate recognition). The final fitness is 0.8×0.6=0.48, which means that the visual fitness of the robot in the current environment is 48%, indicating that there is still room for optimization in dealing with path changes and recognition accuracy.

[0056] It should be noted that the first adaptation component in the present application represents a measure of the adaptability evaluated according to the dynamic compensation amount of the robot during the path recognition process, which reflects the robot's ability to adapt to environmental changes by adjusting its movement or path planning in a dynamic environment; the second adaptation component represents a measure of the adaptability evaluated according to the robot's recognition loss value during the path recognition process, which reflects the accuracy of the robot's path deviation during the visual recognition process, that is, the gap between the recognition result and the actual path; visual fitness represents a measure of the robot's overall adaptability to environmental changes and path recognition challenges in the path recognition task by comprehensively considering the dynamic compensation ability and recognition accuracy. Visual fitness reflects the robot's ability to adjust the path in real time in a dynamic environment and the accuracy of its path recognition.

[0057] In specific implementation, the dynamic adjustment of the path visually recognized by the robot according to the visual fitness can be achieved in the following manner: First, a specific adjustment strategy is formulated based on the calculated visual fitness. For situations with high fitness, the robot may only need to make fine adjustments; for situations with low fitness, larger adjustments are required. For example, a fitness threshold can be set. When the visual fitness is lower than a certain value, more stringent path adjustment measures are taken; then, the relationship between visual fitness and actual path adjustment is determined. The fitness value can be set to be proportional to the path adjustment amount, that is, the higher the fitness, the smaller the path adjustment amount; the lower the fitness, the larger the path adjustment amount; finally, according to the calculated adjustment amount, the robot path is actually adjusted, including modifying the robot's control instructions, such as adjusting the direction, changing the speed, replanning the path, etc., to ensure that the robot can maintain smooth movement during adjustments. Avoid sudden movements that affect its stability, and feed back the adjusted effect into the visual fitness calculation so that the dynamic adjustment strategy can be optimized. If the robot still has path deviation, further adjust the visual fitness and path adjustment strategy. For example: the robot's current visual fitness is 0.3 (low), indicating that its path recognition and dynamic compensation capabilities are poor. According to the set adjustment strategy, the path may need to be significantly corrected; the robot detects a deviation of 0.5 meters from the target path during operation, and the direction deviation is 10 degrees. According to the adjustment strategy, it is calculated that the path needs to be corrected by 0.5 meters and the direction adjusted by 10 degrees. The robot replans the path according to these adjustment instructions and makes real-time corrections.

[0058] In addition, in another aspect of the present application, in some embodiments, the present application provides a robot visual recognition system, referring to Figure 4 , which is a schematic diagram of exemplary hardware and / or software of a robot visual recognition system according to some embodiments of the present application, the robot visual recognition system 400 includes: an acquisition module 401, a processing module 402 and an execution module 403, which are described as follows: Acquisition module 401, in this application, acquisition module 401 is mainly used to acquire path identification information collected by the robot during operation; Processing module 402, in the present application, the processing module 402 is mainly used to match and screen the path recognition information, obtain the path matching area during visual recognition, and determine the visual label corner points in visual recognition according to the path matching area and the dynamic operation characteristics of the robot; The processing module 402 is further used to mark the robot's body zero position, perform deviation configuration on the body zero position, obtain the recognition deviation interval during robot path recognition, and determine the dynamic compensation amount during visual recognition according to the recognition deviation interval and the robot's real-time path trajectory; The processing module 402 is further used to obtain the edge of the running path in the robot's visual recognition, determine the static offset domain when the robot visually recognizes the running path according to the running path edge, and then determine the recognition loss value during visual recognition according to the static offset domain and the label corner point; Execution module 403, in the present application, the execution module 403 is mainly used to determine the visual fitness of the robot during path recognition through the dynamic compensation amount and the recognition loss value, and dynamically adjust the path visually recognized by the robot according to the visual fitness.

[0059] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned robot visual recognition method.

[0060] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing a robot visual recognition method according to some embodiments of the present application. The robot visual recognition method in the above embodiment can be Figure 5 The computer device 500 shown in the figure is implemented, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503 and at least one communication interface 504.

[0061] The processor 501 may be a general-purpose central processing unit (CPU), or an application specific integrated circuit (ASIC) or one or more for controlling the execution of the robot vision recognition method in the present application.

[0062] The communication bus 502 may be used to transmit information between the above-mentioned components.

[0063] The memory 503 may be a read only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read only memory (EEPROM), a compact disc read only memory (CD ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 503 may exist independently and be connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0064] The memory 503 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The robot visual recognition method in the above embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0065] The communication interface 504 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0066] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi CPU) processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0067] The above-mentioned computer device may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer device.

[0068] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned robot visual recognition method is implemented.

[0069] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0070] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A robot visual recognition method, characterized in that: The steps include: Obtain path recognition information collected by the robot during operation; Matching and screening the path recognition information to obtain a path matching area during visual recognition, and determining a visual label corner point in visual recognition according to the path matching area and the dynamic operation characteristics of the robot; Marking the robot's body zero position, performing deviation configuration on the body zero position, obtaining an identification deviation interval during robot path identification, and determining a dynamic compensation amount during visual identification according to the identification deviation interval and the robot's real-time path trajectory; Acquire the running path edge in the robot visual recognition, determine the static offset domain when the robot visually recognizes the running path according to the running path edge, and then determine the recognition loss value during visual recognition according to the static offset domain and the label corner point; The visual fitness of the robot during path recognition is determined by the dynamic compensation amount and the recognition loss value, and the path visually recognized by the robot is dynamically adjusted according to the visual fitness.

2. The method according to claim 1, characterized in that The path identification information is matched and screened, and the path matching area obtained during visual recognition specifically includes: Extracting a path label when the robot performs visual recognition from the path recognition information; Determine the path prediction quantity during robot vision recognition; The path label and the path prediction quantity are subjected to feature screening to obtain a path matching area during visual recognition.

3. The method according to claim 1, characterized in that Determining the dynamic compensation amount during visual recognition according to the recognition deviation interval and the real-time path trajectory of the robot specifically includes: Get the real-time path trajectory of the robot; Determining a dynamic compensation factor of the robot during operation according to the identified deviation interval; The amount of dynamic compensation during visual recognition is determined according to the dynamic compensation factor and the real-time path.

4. The method according to claim 1, characterized in that Determining the static offset domain when the robot visually recognizes the running path according to the running path edge specifically includes: Determine a continuous offset baseline when the robot visually identifies the running path according to the running path edge; Determine the long-tail dataset when the robot performs visual recognition of the running path; A static offset domain when the robot performs visual recognition on a running path is determined according to the continuous offset baseline and the long-tail data set.

5. The method according to claim 1, characterized in that The recognition loss value during visual recognition determined by the static offset domain and the label corner point specifically includes: Determining the recognition concentration of the robot during visual recognition according to the static offset domain; Determine the recognition discrete amount during robot visual recognition according to the label corner points; The recognition loss value during visual recognition is determined by the recognition concentration and the recognition dispersion.

6. The method according to claim 1, characterized in that Determining the robot's visual adaptability for path recognition by using the dynamic compensation amount and the recognition loss value specifically includes: Determine a first adaptation component of the robot when identifying a path according to the dynamic compensation amount; Determining a second adaptation component of the robot when identifying a path according to the recognition loss degree; The visual adaptability of the robot during path recognition is determined according to the first adaptability component and the second adaptability component.

7. The method according to claim 1, characterized in that The path identification information of the robot during operation is obtained by reading the robot data storage unit.

8. A robot visual recognition system, characterized in that: include: An acquisition module is used to obtain path recognition information collected by the robot during operation; A processing module, used for matching and screening the path recognition information to obtain a path matching area during visual recognition, and determining a visual label corner point in visual recognition according to the path matching area and the dynamic operation characteristics of the robot; The processing module is further used to mark the robot's body zero position, perform deviation configuration on the body zero position, obtain an identification deviation interval during robot path identification, and determine a dynamic compensation amount during visual identification based on the identification deviation interval and the robot's real-time path trajectory; The processing module is further used to obtain the running path edge in the robot's visual recognition, determine the static offset domain when the robot visually recognizes the running path according to the running path edge, and then determine the recognition loss value during visual recognition according to the static offset domain and the label corner point; The execution module is used to determine the visual fitness of the robot when identifying a path through the dynamic compensation amount and the recognition loss value, and dynamically adjust the path visually recognized by the robot according to the visual fitness.

9. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the robot visual recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the robot visual recognition method according to any one of claims 1 to 7 is implemented.

Images