A robot recharging method and system based on image recognition and electronic equipment

Abstract

The specification discloses an image recognition-based robot charging method, system and electronic equipment, which has the advantages of low cost, high precision, wide recognition range, etc. The method comprises: acquiring a work scene image by using a monocular camera on the robot, and extracting a charging pile image from the work scene image; performing image recognition on the charging pile image to obtain position information of a feature icon on the charging pile in the charging pile image; judging a positional relationship between the robot and the charging pile according to the position information; performing path planning according to the positional relationship, generating charging path parameters, and controlling the robot to move to the charging pile for charging based on the charging path parameters. The system comprises: a charging pile image extraction module, a feature icon positioning module, a robot positioning module, and a path planning control module. The processor in the electronic equipment implements the method when executing a program.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automation control technology, in particular to a robot charging method and system based on image recognition and electronic equipment. BACKGROUND

[0002] There are various automatic charging principles for robots commonly used in the market, such as infrared signal-based, Bluetooth positioning-based, ultrasonic positioning-based, laser positioning-based, and vision positioning-based.

[0003] In the infrared signal-based charging method, the robot determines its position by receiving the number and position of infrared signals emitted by the charging base, and then plans a route to return to the charging pile for charging. Such positioning method has low accuracy. In the Bluetooth signal-based charging method, Bluetooth positioning is achieved by measuring signal strength. Such positioning method is more suitable for straight-line positioning, and the Bluetooth signal power is generally low and easily interfered by other signals in the environment. In the ultrasonic positioning-based charging method, ultrasonic ranging is greatly affected by multipath effect and non-line-of-sight propagation, and the manufacturing cost of the circuit is relatively high. The laser positioning method and the vision positioning method often require more sensing devices and complex algorithms to achieve high-precision positioning. SUMMARY

[0004] Therefore, the embodiments of the present application provide a robot charging method and system based on image recognition and electronic equipment, which have the advantages of low cost, high precision, wide recognition range, etc.

[0005] According to a first aspect, the embodiments of the present application provide a robot charging method based on image recognition, which is applied to a robot system including a robot and a charging pile. The method includes:

[0006] acquiring a working scene image by using a monocular camera on the robot, and extracting a charging pile image from the working scene image;

[0007] performing image recognition on the charging pile image to obtain position information of a feature icon on the charging pile in the charging pile image;

[0008] determining a positional relationship between the robot and the charging pile according to the position information;

[0009] performing path planning according to the positional relationship, generating charging path parameters, and controlling the robot to move to the charging pile for charging based on the charging path parameters.

[0010] Optionally, acquiring a working scene image by using a monocular camera on the robot, and extracting a charging pile image from the working scene image, includes:

[0011] correcting image distortion of the working scene image according to the intrinsic matrix and the distortion parameters of the monocular camera;

[0012] cutting the four sides of the corrected working scene image according to a preset proportion to generate the charging pile image.

[0013] Optionally, image recognition is performed on the charging pile image to obtain position information of a feature icon on the charging pile in the charging pile image, including:

[0014] contour detection is performed on the charging pile image to obtain contour feature data of each object in the charging pile image;

[0015] comparing the contour feature data with the feature icon to extract and determine the position information of the feature icon in the charging pile image.

[0016] Optionally, a plurality of feature icons are arranged at different positions on the charging pile.

[0017] comparing the contour feature data with the feature icon to extract and determine the position information of the feature icon in the charging pile image, including:

[0018] determining a plurality of contours according to the contour feature data;

[0019] determining geometric feature data of a current contour, and comparing the geometric feature data with a geometric feature parameter range corresponding to a plurality of feature icons;

[0020] in response to the geometric feature data conforming to the geometric feature parameter range, successively comparing morphological features of the current contour with morphological features of a plurality of feature icons;

[0021] in response to a feature icon having morphological features that match the current contour, regarding the current contour as a feature contour;

[0022] determining the feature icon corresponding to the feature contour, and recording position information of the feature contour in the charging pile image.

[0023] Optionally, judging a positional relationship between the robot and the charging pile according to the position information, including:

[0024] selecting an origin on the charging pile to construct a world coordinate system, and determining coordinate information of the feature icon in the world coordinate system;

[0025] determine a translation vector and Euler angles of the robot relative to the origin in the world coordinate system by using a three-dimensional pose estimation algorithm based on the coordinate information of the feature icon in the world coordinate system and the position information of the feature icon in the charging pile image;

[0026] The position relationship includes the translation vector and the Euler angles.

[0027] Optionally, the charging path parameters include a first rotation angle, a first distance, a second rotation angle and a second distance.

[0028] According to the position relationship, the charging path parameters are generated by path planning, including:

[0029] A turning point is arranged directly in front of the charging pile, and the turning point is located on the Y-axis of the world coordinate system.

[0030] According to the position relationship, it is determined whether the robot is located directly in front of the charging pile.

[0031] In response to the robot being located directly in front of the charging pile, it is determined whether the advancing direction of the robot is directly opposite the charging pile.

[0032] In response to the advancing direction of the robot being directly opposite the charging pile, a distance between the robot and the charging pile is determined according to the position relationship, and the distance is taken as the second distance.

[0033] In response to the advancing direction of the robot not being directly opposite the charging pile, an included angle between the advancing direction of the robot and the Y-axis is determined and taken as the first rotation angle, and a distance between the robot and the charging pile is taken as the second distance.

[0034] In response to the robot not being located directly in front of the charging pile, it is determined whether the robot is located near the turning point according to the position relationship.

[0035] In response to the robot being located near the turning point, a distance from the robot to the Y-axis and a foot point are determined.

[0036] The advancing direction of the robot and an included angle between the robot and the foot point are taken as the first rotation angle, a distance from the robot to the Y-axis is taken as the first distance, an included angle between the foot point and the Y-axis is taken as the second rotation angle, and a distance from the foot point to the charging pile is taken as the second distance.

[0037] In response to the robot not being located near the turning point, a distance from the robot to the turning point is determined according to the position relationship.

[0038] The first rotation angle is an angle between a direction in which the robot moves and a line connecting the robot and the inflection point, and the first distance is a distance from the robot to the inflection point.

[0039] Optionally, the robot is controlled to move to the charging pile for charging based on the charging path parameter, including:

[0040] The robot is controlled to rotate according to the first rotation angle so that the direction in which the robot moves is directed to the inflection point.

[0041] The robot is controlled to move for the first time, and a moving length of the first time is the first distance.

[0042] The robot is controlled to rotate according to the second rotation angle so that the direction in which the robot moves is directed to the charging pile.

[0043] The robot is controlled to move for the second time, and a moving length of the second time is the second distance.

[0044] Optionally, the plurality of feature icons include a cross-shaped icon arranged at a center position of a horizontal surface of the charging pile base, and two two-dimensional code icons arranged on a vertical plane of a side wall of the charging pile.

[0045] In a second aspect, the embodiments of the present specification also provide a robot charging back system based on image recognition, and the system includes:

[0046] A charging pile image extraction module is configured to acquire a working scene image by using a monocular camera on the robot, and extract a charging pile image from the working scene image.

[0047] A feature icon positioning module is configured to perform image recognition on the charging pile image to acquire position information of a feature icon on the charging pile in the charging pile image.

[0048] A robot positioning module is configured to determine a positional relationship between the robot and the charging pile according to the position information.

[0049] A path planning control module is configured to perform path planning according to the positional relationship, generate a charging path parameter, and control the robot to move to the charging pile for charging based on the charging path parameter.

[0050] In a third aspect, the embodiments of the present specification also provide an image recognition-based robot charging back electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the image recognition-based robot charging back method as described in the first aspect when executing the program.

[0051] As can be seen from the above, the image recognition-based robot charging back method, system and electronic device provided by one or more optional embodiments of the present specification have the following beneficial technical effects:

[0052] The monocular camera is used to obtain the charging pile image in the working scene, the position information of the feature icon on the charging pile is determined through image recognition, and the accurate position relationship of the robot relative to the charging pile is determined through a three-dimensional pose estimation algorithm. Further, path planning can be performed based on the position relationship, so that the robot can accurately move to the charging head for charging according to the planned path. In this way, the image visual positioning method has a large recognition range, and only a monocular camera is needed to accurately and efficiently control the robot to return to charge, and the application cost is low. BRIEF DESCRIPTION OF DRAWINGS

[0053] The features and advantages of the present application will be more clearly understood through reference to the following drawings, which are presented as illustrative and should not be construed as limiting the application, in which:

[0054] Figure 1 A schematic diagram of an image recognition-based robot charging back method provided by one or more optional embodiments of the present specification is shown;

[0055] Figure 2 A schematic diagram of a method for extracting a charging pile image in an image recognition-based robot charging back method provided by one or more optional embodiments of the present specification is shown;

[0056] Figure 3 A schematic diagram of a method for determining feature icon position information in an image recognition-based robot charging back method provided by one or more optional embodiments of the present specification is shown;

[0057] Figure 4 A schematic diagram of a feature icon on a charging pile in an image recognition-based robot charging back method provided by one or more optional embodiments of the present specification is shown;

[0058] Figure 5 A schematic diagram of a planned path for controlling the movement of a robot in an image recognition-based robot charging back method provided by one or more optional embodiments of the present specification is shown;

[0059] Figure 6A structure schematic diagram of a robot return charging system based on image recognition is shown according to one or more optional embodiments of the present specification.

[0060] Figure 7 An electronic device structure schematic diagram of a robot return charging system based on image recognition is shown according to one or more optional embodiments of the present specification. DETAILED DESCRIPTION

[0061] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0062] Currently, there are various automatic return charging principles for robots on the market, such as infrared signal-based, Bluetooth positioning-based, ultrasonic positioning-based, laser positioning-based, and vision positioning-based.

[0063] In the infrared signal-based return charging method, the robot determines its own position by receiving the number and position of the infrared signals emitted by the charging base, and then plans a route to return to the charging pile for charging. Such a positioning method has low positioning accuracy. In the Bluetooth signal-based return charging method, the Bluetooth measures the signal strength for positioning. Such a method is more suitable for positioning in a straight line direction, and the Bluetooth signal power is generally low and is easily disturbed by other signals in the environment. The ultrasonic positioning-based return charging method is greatly affected by multipath effect and non-line-of-sight propagation in ultrasonic ranging, and has a higher requirement for circuit manufacturing cost. The laser positioning method and the vision positioning method often require more sensing devices and complex algorithms to achieve high-precision positioning. In some vision positioning schemes, high-definition scene images are acquired and image content recognition is performed to determine the position of the charging base. Such a scheme requires the configuration of high-performance image sensors and has a high requirement for image data processing capability, greatly increasing the application cost of the method.

[0064] In view of the above problems, the purpose of the embodiments of the present specification is to propose a robot return charging method based on image recognition. By performing image recognition on the charging pile provided with a feature icon, the positional relationship between the robot and the charging pile is determined, and then a return charging path is planned for return charging movement control. Such a method can achieve accurate positioning of the charging pile based on monocular vision, thereby improving the return charging efficiency.

[0065] Based on the above purpose, in one aspect, the embodiments of the present specification provide a robot return charging method based on image recognition.

[0066] AsFigure 1 As shown, one of the embodiments provided by the present specification is a robot charging method based on image recognition, applied to a robot system including a robot and a charging pile. The method includes:

[0067] S1: Obtain a working scene image using a monocular camera on the robot, and extract a charging pile image from the working scene image.

[0068] The robot can be issued a charging instruction, and after receiving the charging instruction, the robot can control the monocular camera to obtain a working scene image near the robot. The monocular camera can be arranged at the front of the robot, and the camera direction is consistent with the direction of travel of the robot.

[0069] After obtaining the working scene image, the charging pile image can be further extracted therefrom. The charging pile image refers to the main part image containing the charging pile in the working scene image.

[0070] It can be understood that when there is no charging pile in the working scene image, a control instruction can be issued to control the charging pile to move to the vicinity of the charging pile and adjust the orientation of the robot so that the monocular camera can capture the charging pile.

[0071] S2: Perform image recognition on the charging pile image to obtain the position information of the feature icon on the charging pile in the charging pile image.

[0072] A plurality of feature icons can be arranged at different positions on the charging pile. By performing image recognition on the charging pile image, the feature icons are identified and the corresponding position information is determined.

[0073] S3: Determine the positional relationship between the robot and the charging pile according to the position information.

[0074] In the robot system, there is a certain pose relationship between the camera coordinate system based on the monocular camera and the world coordinate system based on the charging pile. For the pose relationship between the two, a three-dimensional pose estimation algorithm can be used to calculate the position information of the robot relative to the charging pile according to the positional relationship of the feature icons in the charging pile image. The position information can include the translation vector and Euler angle of the monocular camera (the robot) relative to the charging pile.

[0075] S4: Perform path planning according to the position relationship, generate charging path parameters, and control the robot to move to the charging pile for charging based on the charging path parameters.

[0076] The image recognition-based robot recharging method described above uses a monocular camera to acquire images of charging stations in the work environment. Image recognition determines the positional information of feature icons on the charging station, and a 3D pose estimation algorithm determines the precise positional relationship of the robot relative to the charging station. Furthermore, path planning can be performed based on this positional relationship, allowing the robot to accurately move to the charging head for charging along the planned path. This image-based visual positioning method has a large recognition range and only requires a monocular camera for accurate and efficient robot recharging control, resulting in low application costs.

[0077] like Figure 2 As shown, in one or more optional embodiments of this specification, a robot recharging method based on image recognition is provided, which uses a monocular camera on the robot to acquire images of the working scene and extracts images of charging piles from the images of the working scene, including:

[0078] S201: Perform image distortion correction on the working scene image based on the intrinsic parameter matrix and distortion parameters of the monocular camera.

[0079] The working scene image acquired by the monocular camera in the robot may have image distortion. Image distortion correction can be performed on the working scene image using the intrinsic parameter matrix and distortion parameters of the monocular camera, thereby avoiding the impact of image distortion on subsequent operations and ensuring the accuracy of the recognized content and the final positioning result.

[0080] S202: The edges of the corrected work scene image are cropped according to a preset ratio to generate the charging pile image.

[0081] In general, the work scene image includes many other elements besides the charging station, such as the ceiling, walls, and floor. This part is irrelevant to the embodiments described in this specification.

[0082] Therefore, this meaningless content can be removed. The work scene image can be cropped around its perimeter according to a preset ratio. For example, the top third and bottom eighth of the work scene image can be cropped, and the left and right quarters of the image can be cropped, leaving the main part including the charging pile, which can be used as the charging pile image. The preset ratio can be flexibly set according to the actual situation.

[0083] This approach, which treats only the main body, including the charging station, as the target for subsequent processing, can significantly reduce the amount of data and computation required for subsequent image processing, and improve the accuracy of recognition and positioning.

[0084] In the robot charging method based on image recognition provided in one or more optional embodiments of the present specification, image recognition is performed on the charging pile image to obtain position information of a feature icon on the charging pile in the charging pile image, including:

[0085] Contour detection is performed on the charging pile image to obtain contour feature data of each object in the charging pile image.

[0086] The contour feature data is compared with the feature icon to extract the position information of the feature icon in the charging pile image.

[0087] In the contour detection of the charging pile image, image gray data of the charging pile image can be extracted, histogram equalization processing is performed on the charging pile image, and adaptiveThreshold is used to segment the image edge contour to perform binaryzation processing on the image, so as to determine the contour line in the charging pile image. Further, image morphological processing such as inflation, corrosion, open-close operation, etc. can be performed on the determined contour line to enhance the image and generate a clearer contour. The contour feature data includes all contour data determined from the charging pile image.

[0088] As shown in Figure 3 In the robot charging method based on image recognition provided in one or more optional embodiments of the present specification, a plurality of feature icons are arranged on different parts of the charging pile. The contour feature data is compared with the feature icon to extract the position information of the feature icon in the charging pile image, including:

[0089] S301: Determine a plurality of contours according to the contour feature data.

[0090] S302: Determine the geometric feature data of the current contour, and compare the geometric feature data with the geometric feature parameter range of a plurality of feature icons.

[0091] The geometric feature data can include contour perimeter, contour area, and contour density.

[0092] S303: In response to the geometric feature data meeting the geometric feature parameter range, morphological features of the current contour are compared with morphological features of a plurality of feature icons in sequence.

[0093] S304: In response to the existence of a feature icon whose morphological feature matches the current contour, the current contour is taken as a feature contour.

[0094] S305: Determine the feature icon corresponding to the feature contour, and record the position information of the feature contour in the charging pile image.

[0095] By comparing the geometric feature data and the morphological feature, the feature contour corresponding to the feature icon on the charging pile can be selected from the multiple contours in the charging pile image.

[0096] As shown in FIG. 1, in some optional embodiments, the multiple feature icons include a cross-shaped icon arranged at the center of the horizontal surface of the charging pile base, and two two-dimensional code icons arranged on the rear surface of the charging pile. Figure 4 The charging power supply contact is arranged at the joint between the base and the rear edge of the charging pile. It should be noted that, Figure 4 The charging pile shown in FIG. 1 is only one optional charging pile schematic diagram. The multiple feature icons in the charging pile can also have multiple other cases, such as a star-shaped feature icon, a circular ring-shaped feature icon, a hook-shaped feature icon, or other special-shaped feature icons, etc. The positions of the multiple feature icons on the charging pile can also be flexibly set.

[0097] In the charging pile, the cross-shaped icon on the horizontal plane and the two-dimensional code icon on the vertical plane are arranged as feature icons, which can improve the spatial recognition ability of the charging method for the charging pile and reduce the misrecognition rate.

[0098] Taking the cross-shaped icon as an example, the position information of the cross-shaped icon in the charging pile image can be determined by comparing the contour feature data with the cross-shaped icon, which can be implemented by the following steps:

[0099] Iterate all contours to determine whether all contours have been checked.

[0100] If all contours have not been checked, determine whether the perimeter, area and contour density of the current contour meet the geometric feature parameter range in sequence.

[0101] If all meet the geometric feature parameter range, determine whether the morphological feature of the current contour is consistent with the cross-shaped icon.

[0102] If the morphological feature of the current contour is consistent with the cross-shaped icon, record the position information of the current contour and the coordinate data of the multiple points in the current contour in the image.

[0103] As an optional embodiment, whether the morphological feature of the current contour is consistent with the cross-shaped icon can be determined by the following steps:

[0104] determine whether the distance |P1P2| between the two points is less than the cross long side threshold value;

[0105] If |P1P2| is less than the cross side threshold value, determine the points P3, P4 in the current contour that are farthest from the line on which P1 and P2 lie;

[0106] Compare the distance |P3P4| between the two points with the cross short side threshold value to determine whether it is less than the cross short side threshold value;

[0107] If |P3P4| is less than the cross short side threshold value, determine the distances d1, d2 of all points in the current contour from P1 and P2, and the distances d3, d4 of all points from P3 and P4;

[0108] Determine whether d1, d2 and d3, d4 satisfy the corresponding length range;

[0109] If it is satisfied, calculate the ratio between |P1P2| and |P3P4| to determine whether the ratio satisfies the ratio threshold value range;

[0110] If it is satisfied, determine the intersection point P of the line segments P1P2 and P3P4 x , determine whether the ratio between |P1P x | and |P x P2| and the ratio between |P3P x | and |P x P4| satisfy the corresponding ratio threshold value range;

[0111] If it is satisfied, determine whether the angle between the line segments P1P2 and P3P4 satisfies the angle threshold value range;

[0112] If all the above determination conditions are satisfied, it can be determined that the current contour is a characteristic contour corresponding to a cross-shaped icon. Record the coordinate information of P1, P2, P3 and P4 to represent the position information of the characteristic contour in the charging pile image.

[0113] In one or more optional embodiments of the present specification, a robot recharging method based on image recognition is provided. According to the position information, the position relationship between the robot and the charging pile is determined, comprising:

[0114] Select an origin on the charging pile to construct a world coordinate system, and determine the coordinate information of the characteristic icon in the world coordinate system.

[0115] determine the translation vector and the Euler angle of the robot relative to the origin in the world coordinate system based on the coordinate information of the feature icon in the world coordinate system and the position information of the feature icon in the charging pile image; the position relationship includes the translation vector and the Euler angle.

[0116] In the world coordinate system, the center point of the upper surface of the charging pile base can be selected as the origin of the world coordinate system, the forward direction of the charging pile is the positive direction of the y-axis, the right-hand direction is the positive direction of the x-axis, and the vertical upward direction is the positive direction of the z-axis. Based on the world coordinate system, a three-dimensional pose estimation algorithm can be used to determine the position relationship between the robot and the charging pile. The three-dimensional pose estimation algorithm can be solvepnp algorithm. The world coordinate data of the feature icon and the corresponding position information of the feature image in the charging pile image are input into the solvepnp algorithm as input data, so that the position relationship between the robot and the charging pile can be determined. The solvepnp algorithm can directly return a rotation matrix and a translation matrix, and further conversion of the rotation matrix and the translation matrix can determine the translation vector and the Euler angle of the monocular camera (robot) relative to the origin in the world coordinate system.

[0117] In one embodiment of the present application, the charging path parameters include a first rotation angle γ1, a first distance dis1, a second rotation angle γ2, and a second distance dis2.

[0118] According to the position relationship, the charging path parameters are generated, including:

[0119] A turning point G is arranged in front of the charging pile, and the turning point is located on the Y-axis of the world coordinate system.

[0120] According to the position relationship, it is determined whether the robot is located in front of the charging pile;

[0121] In response to the robot being located in front of the charging pile, it is determined whether the advancing direction of the robot is directly opposite to the charging pile;

[0122] In response to the advancing direction of the robot being directly opposite to the charging pile, the distance between the robot and the charging pile is determined according to the position relationship, and the distance is taken as the second distance dis2.

[0123] In response to the fact that the robot's direction of travel is not directly facing the charging pile, the angle between the robot's direction of travel and the Y-axis is determined and used as the first rotation angle γ1, and the distance between the robot and the charging pile is used as the second distance dis2;

[0124] In response to the robot not being directly in front of the charging station, determine whether the robot is near the inflection point based on the positional relationship;

[0125] In response to the robot being near the inflection point, the distance of the robot to the Y-axis and the foot of the perpendicular are determined;

[0126] The first rotation angle γ1 is defined as the angle between the robot's direction of travel and the line connecting the robot and the vertical foot; the first distance dis1 is defined as the distance from the robot to the Y-axis; the second rotation angle γ2 is defined as the angle between the robot and the line connecting the vertical foot and the Y-axis; and the second distance dis2 is defined as the distance from the vertical foot to the charging pile.

[0127] In response to the robot not being near the inflection point, the distance from the robot to the inflection point is calculated and determined based on the positional relationship;

[0128] The first rotation angle γ1 is defined as the angle between the robot's direction of travel and the line connecting the robot and the inflection point; the first distance dis1 is defined as the distance from the robot to the inflection point; the second rotation angle γ2 is defined as the angle between the line connecting the robot and the inflection point and the Y-axis; and the second distance dis2 is defined as the distance from the inflection point to the charging pile.

[0129] like Figure 5 The diagram shows a planned path for controlling the robot's movement. In one or more optional embodiments of this specification, a robot recharging method based on image recognition is provided, controlling the robot to move to the charging station for charging based on the charging path parameters, including:

[0130] Control the robot to rotate according to the first rotation angle γ1 so that the robot's direction of travel points to the inflection point G;

[0131] The robot is controlled to make its first movement, and the distance traveled in the first movement is the first distance dis1;

[0132] The robot is controlled to rotate according to the second rotation angle γ2 so that the robot's direction of travel points towards the charging pile;

[0133] The robot is controlled to make a second movement, the distance traveled in the second movement being the second distance dis2.

[0134] According to the above method, the robot can be accurately moved to the charging pile for charging.

[0135] It should be noted that the method of one or more embodiments of the present specification can be performed by a single device, such as a computer or a server, etc. The method of the present embodiment can also be applied to a distributed scenario, which is completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of the present specification, and the multiple devices interact with each other to complete the method.

[0136] It should be noted that the above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0137] Based on the same inventive concept, the present specification also provides a robot back charging system based on image recognition corresponding to the method of any of the above embodiments.

[0138] Reference Figure 6 , the robot back charging system based on image recognition, the method is applied to a robot system, including a robot and a charging pile, the back charging system includes:

[0139] a charging pile image extraction module for acquiring a working scene image using a monocular camera on the robot, and extracting a charging pile image from the working scene image;

[0140] a feature icon positioning module for image recognition on the charging pile image to obtain position information of a feature icon on the charging pile in the charging pile image;

[0141] a robot positioning module for determining the positional relationship between the robot and the charging pile according to the position information; and

[0142] a path planning control module for path planning according to the positional relationship, generating charging path parameters, and controlling the robot to move to the charging pile for charging based on the charging path parameters.

[0143] In an image recognition-based robot charging system provided by one or more optional embodiments of the present specification, the charging pile image extraction module is further configured to perform image distortion correction on the working scene image according to intrinsic matrix and distortion parameters of the monocular camera; after image distortion correction, content recognition is performed on the working scene image to determine whether a charging pile exists in the current working scene; in response to the existence of a charging pile in the current working scene, the region where the charging pile is located is taken as a region of interest, and an image of the region of interest is extracted as the charging pile image.

[0144] In an image recognition-based robot charging system provided by one or more optional embodiments of the present specification, the feature icon positioning module is further configured to perform contour detection on the charging pile image to obtain contour feature data in the charging pile image; and compare the contour feature data with the feature icon to extract position information of the feature icon in the charging pile image.

[0145] In an image recognition-based robot charging system provided by one or more optional embodiments of the present specification, a plurality of feature icons are arranged at different positions on the charging pile. The feature icon positioning module is further configured to determine a plurality of contours according to the contour feature data, determine geometric feature data of a current contour, compare the geometric feature data with a range of geometric feature parameters corresponding to a plurality of feature icons, and in response to the geometric feature data conforming to the range of geometric feature parameters, compare morphological features of the current contour with morphological features of a plurality of feature icons in sequence, and in response to a feature icon whose morphological features match the current contour, take the current contour as a feature contour, determine the feature icon corresponding to the feature contour, and record position information of the feature contour in the charging pile image.

[0146] In an image recognition-based robot charging system provided by one or more optional embodiments of the present specification, the robot positioning module is further configured to select an origin on the charging pile to construct a world coordinate system, determine coordinate information of the feature icon in the world coordinate system, and based on the coordinate information of the feature icon in the world coordinate system and the position information of the feature icon in the charging pile image, determine a translation vector and Euler angles of the robot relative to the origin in the world coordinate system by using a three-dimensional pose estimation algorithm. The position relationship includes the translation vector and the Euler angles.

[0147] In an image recognition-based robot return charging system provided by one or more optional embodiments of the present disclosure, the charging path parameters include a first rotation angle, a first distance, a second rotation angle, and a second distance. The path planning control module is further configured to set a turning point in front of the charging pile, the turning point being located on the Y-axis of the world coordinate system; determine whether the robot is located in front of the charging pile according to the positional relationship; in response to the robot being located in front of the charging pile, determine whether the advancing direction of the robot is directly opposite the charging pile; in response to the advancing direction of the robot being directly opposite the charging pile, determine the distance between the robot and the charging pile according to the positional relationship and take the distance as the second distance; in response to the advancing direction of the robot not being directly opposite the charging pile, determine the included angle between the advancing direction of the robot and the Y-axis and take the included angle as the first rotation angle, and determine the distance between the robot and the charging pile as the second distance; in response to the robot not being located in front of the charging pile, determine whether the robot is located near the turning point according to the positional relationship; in response to the robot being located near the turning point, determine the distance from the robot to the Y-axis and a foot point; take the advancing direction of the robot and the included angle between the robot and the foot point as the first rotation angle, take the distance from the robot to the Y-axis as the first distance, take the included angle between the foot point and the Y-axis as the second rotation angle, and take the distance from the foot point to the charging pile as the second distance; in response to the robot not being located near the turning point, determine the distance from the robot to the turning point according to the positional relationship; take the advancing direction of the robot and the included angle between the robot and the turning point as the first rotation angle, take the distance from the robot to the turning point as the first distance, take the included angle between the robot and the turning point and the Y-axis as the second rotation angle, and take the distance from the turning point to the charging pile as the second distance.

[0148] In an image recognition-based robot return charging system provided by one or more optional embodiments of the present disclosure, the path planning control module is further configured to control the robot to rotate according to the first rotation angle so that the advancing direction of the robot points to the turning point, control the robot to move for a first time, the advancing length of the first time being the first distance, control the robot to rotate according to the second rotation angle so that the advancing direction of the robot points to the charging pile, and control the robot to move for a second time, the advancing length of the second time being the second distance.

[0149] In one or more optional embodiments provided in the specification, a robot recharging system based on image recognition, wherein a plurality of feature icons include a cross-shaped icon arranged at the center of the horizontal surface of the charging pile base, and two two-dimensional code icons arranged on the vertical plane of the side wall of the charging pile.

[0150] For the convenience of description, the above device is described as various modules described respectively in function. Of course, in the implementation of one or more embodiments of the specification, the functions of each module can be implemented in the same or more software and / or hardware.

[0151] The device of the above embodiment is used to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described here.

[0152] Figure 7 A more specific electronic device hardware structure schematic diagram provided by the embodiment is shown, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040 and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030 and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0153] The processor 1010 can be implemented in the form of a general-purpose CPU (Central Processing Unit, central processor), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the specification.

[0154] The memory 1020 can be implemented in the form of a ROM (Read Only Memory, read-only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the specification are implemented by software or firmware, the related program codes are saved in the memory 1020 and called and executed by the processor 1010.

[0155] The input / output interface 1030 is configured to connect an input / output module to realize information input and output. The input / output module can be configured in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0156] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0157] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0158] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only include components necessary for implementing the embodiments of the present disclosure, and does not necessarily include all components shown in the figure.

[0159] The electronic device of the above embodiments is used to implement the corresponding methods in the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here.

[0160] Based on the same inventive concept, the disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the image recognition-based robot recharging method according to any one of the above embodiments.

[0161] The computer readable medium of the embodiments can include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0162] The storage medium of the above embodiments stores computer instructions for causing the computer to perform the image recognition based robot recharging method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here again.

[0163] Those skilled in the art can understand that all or part of the processes in the above embodiments can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0164] The system, device, module or unit described in the above embodiments can be implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0165] For the convenience of description, the above device is described as various units respectively described in functions. Of course, the functions of each unit can be implemented in the same or more software and / or hardware when implementing the present application.

[0166] Those skilled in the art will appreciate that embodiments of the present specification can be devised without the specific details of the figures described above. Embodiments of the present specification can be implemented with or without the use of software depending on the particular application. Embodiments of the present specification can be implemented using a programmed general purpose computer, a computer network, or other dedicated computing system. Each process can be run in hardware, software, or some combination thereof.

[0167] It should also be noted that the term "comprising" or "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not limited to those elements, but can also include other elements not expressly listed or inherent to such processes, methods, articles, or apparatuses. Without further limitation, an element preceded by "comprising" does not, without more limitations, foreclose the existence of additional identical elements in the process, method, article, or apparatus that includes the recited element.

[0168] The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0169] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0170] Those of ordinary skill in the art will understand that the above discussion of any embodiment is merely exemplary and is not intended to suggest the scope of the present disclosure (including the claims), which is limited to these examples; under the idea of the present disclosure, the above embodiments or technical features between different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes as described above. In order to be brief, they are not provided in detail.

[0171] While the present disclosure has been described in connection with certain embodiments, it will be understood that many modifications, substitutions, and improvements can be made by those of ordinary skill in the art to the embodiments without departing from the spirit and scope of the disclosure. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0172] It is intended, therefore, that the scope of the disclosure be limited by the broadest interpretation of the appended claims to include all such modifications and alterations, and any further modifications in the spirit and scope of the above-described embodiments.

Claims

1. A robot recharging method based on image recognition, characterized in that, The method is applied to a robot system, the robot system including a robot and a charging station, and the method includes: The robot uses a monocular camera to acquire images of the working scene and extracts images of charging piles from the images of the working scene; the monocular camera is positioned directly in front of the robot, and the camera's direction of travel is consistent with the robot's direction of travel. The step of acquiring a working scene image using a monocular camera on the robot and extracting a charging pile image from the working scene image includes: correcting the image distortion of the working scene image according to the intrinsic parameter matrix and distortion parameters of the monocular camera; and cropping the perimeter of the corrected working scene image according to a preset ratio to generate the charging pile image. Image recognition is performed on the charging pile image to obtain the location information of the feature icons on the charging pile in the charging pile image; The positional relationship between the robot and the charging station is determined based on the location information; Based on the positional relationship, a path is planned to generate charging path parameters, and the robot is controlled to move to the charging station for charging based on the charging path parameters. Determining the positional relationship between the robot and the charging station based on the location information includes: A world coordinate system is constructed by selecting an origin on the charging pile, and the coordinate information of the feature icon in the world coordinate system is determined. Based on the coordinate information of the feature icon in the world coordinate system and the position information of the feature icon in the charging pile image, the translation vector and Euler angle of the robot relative to the origin in the world coordinate system are calculated and determined using a three-dimensional pose estimation algorithm; wherein, the three-dimensional pose estimation algorithm is the Solvnp algorithm; The positional relationship includes the translation vector and the Euler angle; wherein, the feature icons include a cross-shaped icon set at the center of the horizontal surface of the charging pile base, and two QR code icons set on the vertical plane of the side wall of the charging pile.

2. The method according to claim 1, characterized in that, Image recognition is performed on the charging pile image to obtain the location information of the feature icons on the charging pile in the charging pile image, including: Contour detection is performed on the charging pile image to obtain contour feature data of each object in the charging pile image; The contour feature data is compared with the feature icon to extract and determine the location information of the feature icon in the charging pile image.

3. The method according to claim 2, characterized in that, The charging pile has multiple feature icons set on different parts; The contour feature data is compared with the feature icon to extract and determine the position information of the feature icon in the charging pile image, including: Multiple contours are determined based on the contour feature data; Determine the geometric feature data of the current contour, and compare the geometric feature data with the corresponding geometric feature parameter ranges of multiple feature icons; In response to the geometric feature data conforming to the range of geometric feature parameters, the morphological features of the current contour are sequentially compared with the morphological features of multiple feature icons; In response to the existence of a feature icon whose morphological features match the current contour, the current contour is taken as the feature contour; The feature icon corresponding to the feature contour is determined, and the position information of the feature contour in the charging pile image is recorded.

4. The method according to claim 1, characterized in that, The charging path parameters include a first rotation angle, a first distance, a second rotation angle, and a second distance; Based on the location relationships, path planning is performed to generate charging path parameters, including: An inflection point is set directly in front of the charging pile, and the inflection point is located on the Y-axis of the world coordinate system; Determine whether the robot is located directly in front of the charging station based on the positional relationship; In response to the robot being located directly in front of the charging station, determine whether the robot's direction of travel is facing the charging station; In response to the robot's direction of travel being directly opposite the charging station, the distance between the robot and the charging station is determined based on the positional relationship and used as the second distance; In response to the robot's travel direction not being directly facing the charging pile, the angle between the robot's travel direction and the Y-axis is determined and used as the first rotation angle, and the distance between the robot and the charging pile is used as the second distance; In response to the robot not being directly in front of the charging station, determine whether the robot is near the inflection point based on the positional relationship; In response to the robot being near the inflection point, the distance of the robot to the Y-axis and the foot of the perpendicular are determined; The first rotation angle is the angle between the robot's direction of travel and the line connecting the robot and the vertical foot; the first distance is the distance from the robot to the Y-axis; the second rotation angle is the angle between the robot and the line connecting the vertical foot and the Y-axis; and the second distance is the distance from the vertical foot to the charging pile. In response to the robot not being near the inflection point, the distance from the robot to the inflection point is calculated and determined based on the positional relationship; The first rotation angle is defined as the angle between the robot's direction of travel and the line connecting the robot and the inflection point; the first distance is defined as the distance from the robot to the inflection point; the second rotation angle is defined as the angle between the line connecting the robot and the inflection point and the Y-axis; and the second distance is defined as the distance from the inflection point to the charging pile.

5. The method according to claim 4, characterized in that, Controlling the robot to move to the charging station for charging based on the charging path parameters includes: Control the robot to rotate according to the first rotation angle so that the robot's direction of travel points to the inflection point; The robot is controlled to make its first movement, and the distance traveled in the first movement is the first distance. Control the robot to rotate according to the second rotation angle so that the robot's direction of travel points towards the charging pile; The robot is controlled to make a second movement, the distance traveled in the second movement being the second distance.

6. A robot recharging system based on image recognition, characterized in that, The system is applied to a robot system, including a robot and a charging station, and the recharging system includes: The charging pile image extraction module is used to acquire images of the working scene using a monocular camera on the robot, and extract images of the charging pile from the images of the working scene; the monocular camera is set in front of the robot, and the camera's direction is consistent with the robot's direction of travel; Specifically, the charging pile image extraction module is used to perform image distortion correction on the working scene image based on the intrinsic parameter matrix and distortion parameters of the monocular camera; and to crop the perimeter of the corrected working scene image according to a preset ratio to generate the charging pile image. The feature icon positioning module is used to perform image recognition on the charging pile image to obtain the location information of the feature icons on the charging pile in the charging pile image; A robot positioning module is used to determine the positional relationship between the robot and the charging station based on the position information; and The path planning and control module is used to plan a path based on the positional relationship, generate charging path parameters, and control the robot to move to the charging pile for charging based on the charging path parameters. Determining the positional relationship between the robot and the charging station based on the location information includes: A world coordinate system is constructed by selecting an origin on the charging pile, and the coordinate information of the feature icon in the world coordinate system is determined. Based on the coordinate information of the feature icon in the world coordinate system and the position information of the feature icon in the charging pile image, the translation vector and Euler angle of the robot relative to the origin in the world coordinate system are calculated and determined using a three-dimensional pose estimation algorithm; wherein, the three-dimensional pose estimation algorithm is the Solvnp algorithm; The positional relationship includes the translation vector and the Euler angle; wherein, the feature icons include a cross-shaped icon set at the center of the horizontal surface of the charging pile base, and two QR code icons set on the vertical plane of the side wall of the charging pile.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 5.

Images