Charging Port Identification Method, Its Device and Electronic Device

Abstract

The present invention discloses a charging port identification method, device and electronic device. The method comprises: acquiring a charging port image; extracting a pixel area corresponding to the charging port and a first pixel point in the pixel area from the charging port image, wherein the first pixel point is a pixel point corresponding to the first center point of the charging port in the pixel area; determining a first charging position corresponding to the charging port based on at least the area of the pixel area and the position of the first pixel point in the pixel area; acquiring a second pixel point in the pixel area, wherein the second pixel point is a pixel point corresponding to the second center point of the charging hole in the charging port in the pixel area; generating posture information corresponding to the charging port based on the positions of the first charging position and the second pixel point in the pixel area, wherein the posture information is the posture information of the charging gun to be inserted into the charging port. The above method realizes the precise positioning of the three-dimensional posture of the charging port of the electric vehicle.

Description

Technical Field

[0001] The present invention relates to the field of industrial robot control, and in particular to a charging port identification method, a device thereof, and an electronic device. Background Art

[0002] Charging technology is a key component in building an internet-based public charging service network for electric vehicles. The charging experience largely determines the user experience. Due to the limitations of current battery technology, electric vehicles can only travel a limited distance after a few hours of charging. Currently, users are required to manually charge their electric vehicles after use, a method that compromises user experience and consumes significant human resources.

[0003] With the development of visual positioning technology, the key to featureless charging port identification and positioning has been overcome. Through intelligent learning methods, this technology effectively solves the challenge of reliable and stable charging port identification in all-weather lighting conditions. This has also resulted in an implementable technical solution that enables accurate and reliable identification without modifying the vehicle's native charging port.

[0004] Currently, charging port recognition algorithms generally use feature point extraction. SURF and SIFT are two classic local feature detection algorithms that are robust to illumination, noise, and small-scale viewpoint variations. However, these algorithms are prone to misdetecting feature points, resulting in inaccurate 3D pose measurements of electric vehicle charging ports. Furthermore, these algorithms require high computational effort, and their effectiveness decreases significantly when there are fewer local feature points. Summary of the Invention

[0005] The main purpose of the present invention is to provide a charging port identification method, a device and an electronic device thereof, so as to solve the problem of inaccurate three-dimensional position positioning of the charging port of an electric vehicle in the prior art.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present invention, a charging port identification method is provided, comprising: acquiring a charging port image; extracting a pixel area corresponding to the charging port and a first pixel point in the pixel area from the charging port image, wherein the first pixel point is a pixel point corresponding to a first center point of the charging port in the pixel area; determining a first charging position corresponding to the charging port based at least on the area of the pixel area and the position of the first pixel point in the pixel area; acquiring a second pixel point in the pixel area, wherein the second pixel point is a pixel point corresponding to a second center point of the charging hole in the charging port in the pixel area; generating posture information corresponding to the charging port based on the positions of the first charging position and the second pixel point in the pixel area, wherein the posture information is posture information of a charging gun to be inserted into the charging port.

[0007] Optionally, the charging port includes a charging port base, and the charging hole passes through the charging port base. The pixel area corresponding to the charging port is extracted from the charging port image, including: establishing a three-dimensional coordinate system with the first center point of the charging port as the coordinate origin, the extension directions of any two straight lines perpendicular to each other in the plane where the charging port base is located as the X-axis and Y-axis, and the penetration direction of the charging hole as the Z-axis; extracting a circular pixel point set from the charging port image, wherein each circle corresponding to the circular pixel point set is a circular area with the coordinate origin of the three-dimensional coordinate system as the center and a radius in a preset radius set as the radius; obtaining a straight line pixel point set that satisfies a preset ratio with the preset radius set from the charging port image; extracting a first circle and a first straight line from the circular pixel point set and the straight line pixel point set, wherein the first circle and the first straight line constitute a closed area in the same plane, and the closed area is used as the pixel area corresponding to the charging port.

[0008] Optionally, the first charging position corresponding to the charging port is determined based at least on the area of the pixel area and the position of the first pixel point in the pixel area, including: obtaining the focal length of the image collector, wherein the image collector is used to collect the image of the charging port; establishing a two-dimensional coordinate system with the center point of the charging port image as the coordinate origin and the extension directions of any two perpendicular straight lines in the charging port image as the x-axis and y-axis; determining the spatial position of the first center point relative to the image collector based on the projected area of the charging port, the area of the pixel area, and the x and y coordinate values of the first pixel point in the two-dimensional coordinate system, wherein the spatial position includes a first lateral distance and a first longitudinal distance; determining the distance between the image collector and the charging port based on the first lateral distance, the first longitudinal distance, and the focal length of the image collector.

[0009] Optionally, obtaining the second pixel point in the pixel area includes: extracting a characteristic pattern in the pixel area; and determining a center point of the characteristic pattern as the second pixel point.

[0010] Optionally, extracting characteristic graphics in a pixel area includes: extracting multiple contour graphics in the pixel area; obtaining a set of circumscribed rectangles of each contour graphic in the multiple contour graphics; determining at least one first contour graphic from the multiple contour graphics according to preset conditions, wherein the above-mentioned preset conditions include: the set of circumscribed rectangles of the first contour graphics satisfies a preset aspect ratio range, and the length of the set of circumscribed rectangles of the first contour graphics satisfies a preset length; fitting at least one first contour graphic into at least one target graphic; and determining a characteristic graphic from at least one target graphic based on a preset shape, wherein the preset shape includes a circle and an ellipse.

[0011] Optionally, extracting the characteristic pattern in the pixel area further includes: when there are multiple target patterns and multiple first characteristic patterns are determined from the multiple target patterns based on a preset shape, removing the target patterns whose length-to-short diameter ratio is inconsistent with the preset length-to-short diameter ratio from the multiple first characteristic patterns to obtain the characteristic pattern.

[0012] Optionally, based on the first charging position and the position of the second pixel point in the pixel area, posture information corresponding to the charging port is generated, including: when the number of extracted feature graphics is greater than a preset number, based on the projection relationship between the position of the second pixel point in the pixel area and the three-dimensional coordinate system, the second charging position of the charging port in the charging port is determined, wherein the preset number is less than the number of charging holes, and the preset number is greater than 1 / 2 of the number of charging holes; based on the first charging position and the second charging position, the posture information corresponding to the charging port is generated.

[0013] According to another aspect of the present invention, a charging port identification device is provided, including: a first acquisition module for acquiring a charging port image; an extraction module for extracting a pixel area corresponding to the charging port and a first pixel point in the pixel area from the charging port image, wherein the first pixel point is a pixel point in the pixel area corresponding to the first center point of the charging port; a determination module for determining a first charging position corresponding to the charging port based on at least the area of the pixel area and the position of the first pixel point in the pixel area; a second acquisition module for acquiring a second pixel point in the pixel area, wherein the second pixel point is a pixel point in the pixel area corresponding to the second center point of the charging hole in the charging port; a generation module for generating posture information corresponding to the charging port based on the first charging position and the position of the second pixel point in the pixel area, wherein the posture information is posture information of the charging gun to be inserted into the charging port.

[0014] According to another aspect of the present invention, a charging system is also provided, including: a vehicle to be charged, the vehicle to be charged having a charging port; a charging robot, the charging robot having a charging gun; and a controller, the controller being used to receive posture information corresponding to the charging port generated by the above-mentioned charging port identification method, and inserting the charging gun into the charging port based on the posture information.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the charging port identification method as described above.

[0016] The technical solution of the present invention provides a charging port identification method. The method captures a charging port image, extracts a pixel region corresponding to the charging port and a first pixel point in the pixel region from the charging port image, wherein the first pixel point is a pixel point corresponding to a first center point of the charging port in the pixel region, and determines a first charging position corresponding to the charging port based on at least the area of the pixel region and the position of the first pixel point in the pixel region. A second pixel point corresponding to a second center point of the charging hole in the charging port is then acquired in the pixel region. Position information corresponding to the charging port is then generated based on the positions of the first charging position and the second pixel point in the pixel region. The above method utilizes a combined graphical approach to identify the charging port. Unlike the feature point extraction method used in the prior art, the method can first identify the external features of the charging port to achieve a rough positioning of the charging port, and then obtain the contour information of the charging hole in the charging port to achieve a precise positioning of the charging port position. This method solves the problem of inaccurate three-dimensional position positioning of electric vehicle charging ports in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0018] Figure 1 is a flowchart illustrating a charging port identification method according to an exemplary embodiment;

[0019] Figure 2 is a flowchart illustrating another charging port identification method according to an exemplary embodiment;

[0020] Figure 3 is a schematic diagram of a charging port image obtained in the charging port recognition method according to Example 1 of the present invention;

[0021] Figure 4 This is a schematic diagram of a camera pinhole imaging model in a charging port identification method according to Example 1 of the present invention;

[0022] Figure 5 yes Figure 4 The diagram shown is a schematic diagram of the relationship between pixel coordinates and physical coordinates according to the principle diagram of the camera pinhole imaging model;

[0023] Figure 6 is a device block diagram of a charging port identification method according to embodiment 2 of the present invention;

[0024] Figure 7 is a device block diagram of a terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," and the like in the specification and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate for the embodiments of the present invention described herein. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0028] Example 1

[0029] This embodiment provides Figure 1 The charging port identification method shown. Figure 1 FIG. 1 is a flow chart of a charging port identification method according to embodiment 1 of the present invention. Figure 1 As shown, the method includes the following steps:

[0030] Step S202, acquiring a charging port image;

[0031] Step S204: extracting a pixel region corresponding to the charging port and a first pixel point in the pixel region from the charging port image, wherein the first pixel point is a pixel point corresponding to a first center point of the charging port in the pixel region;

[0032] Step S206, determining a first charging position corresponding to the charging port based at least on the area of the pixel region and the position of the first pixel point in the pixel region;

[0033] Step S208, obtaining a second pixel point in the pixel area, wherein the second pixel point is a pixel point in the pixel area corresponding to a second center point of the charging hole in the charging port;

[0034] In step S210 , position information corresponding to the charging port is generated based on the first charging position and the position of the second pixel point in the pixel area, wherein the position information is position information of the charging gun to be inserted into the charging port.

[0035] The above method uses a graphic combination method to identify the charging port. Unlike the feature point extraction method used in the existing technology, the Hough transform algorithm can be used to identify the external features of the charging port to complete the rough positioning of the charging port position. Then, the Canny operator can be further combined to obtain the contour information of the charging hole in the charging port to complete the precise positioning of the charging port posture, thus solving the problem of inaccurate three-dimensional posture positioning of the electric vehicle charging port in the existing technology.

[0036] In the above step S202, an image collector is used to acquire an image of the charging port.

[0037] Specifically, after acquiring the charging port image (loading the image) through data acquisition, the charging port image can be preprocessed first, the charging port image can be converted into a grayscale image (converting the grayscale image), and a suitable convolution kernel can be selected to filter the image (such as Gaussian filtering), such as Figure 2 shown.

[0038] In step S204, the pixel region corresponding to the charging port is extracted from the charging port image. Then, based on the acquired charging port image, the pixel point at the center of the charging port, i.e., the first pixel point, is determined. The outer periphery of the charging port is composed of a circle and a straight line. Based on this feature, the charging port is identified and located.

[0039] As an optional embodiment, the charging port includes a charging port base, and the charging hole passes through the charging port base. A three-dimensional coordinate system is established with the first center point of the charging port as the coordinate origin, the extension directions of any two straight lines perpendicular to each other in the plane where the charging port base is located as the X-axis and Y-axis, and the penetration direction of the charging hole as the Z-axis; a circular pixel point set is extracted from the charging port image, wherein each circle corresponding to the circular pixel point set is a circular area with the coordinate origin of the three-dimensional coordinate system as the center and a radius in a preset radius set as the radius; a straight line pixel point set that satisfies a preset ratio with the preset radius set is obtained from the charging port image; a first circle and a first straight line are extracted from the circular pixel point set and the straight line pixel point set, wherein the first circle and the first straight line constitute a closed area in the same plane, and the closed area is used as the pixel area corresponding to the charging port.

[0040] For example, the charging port image is as follows: Figure 3 As shown, there are a total of nine cylindrical charging holes, which are DC protection ground, DC power supply positive, DC power supply negative, vehicle connection confirmation terminal, low-voltage auxiliary power supply positive, low-voltage auxiliary power supply negative, charging terminal communication and power supply terminal connection confirmation. Figure 3The charging holes of the middle charging port are mainly divided into three types of holes: large circle (DC-, DC+) with an outer diameter of 25.4mm and an inner diameter of 12mm; medium circle (PE) with an outer diameter of 15.1mm and an inner diameter of 6mm; small circle (S-, CC2, CC1, S+, A-, A+) with an outer diameter of 10.3mm and an inner diameter of 3mm.

[0041] Specifically, for the charging port image with a camera as the image collector, we can use the side view of the car as the reference plane and the center of the circular feature around the charging port as the coordinate origin to establish a Figure 3 The three-dimensional coordinate system shown. Since the center of the semicircle in the charging port area coincides with the center of the charging port, it can be roughly considered the center of the charging port. Due to symmetry, the origin of the coordinate system does not shift regardless of whether the area rotates about the X or Y axis (when the car is parked normally on the road, the charging port does not rotate much about the Z axis). Therefore, the origin of this coordinate system can be used as the target point, that is, the three-dimensional coordinates of this point in the camera coordinate system are the required coarse positioning results. For example, in the above three-dimensional coordinate system, the coordinate value of DC- is (-17.00, +1, 20, 0.00), the coordinate value of DC+ is (17.00, +1, 20, 0.00), the coordinate value of CC2 is (0.00, -21, 00, 0.00), the coordinate value of CC1 is (0.00, -9, 00, 0.00), the coordinate value of S- is (-12, 00, -21, 00, 10.00), the coordinate value of S+ is (12, 00, -21, 00, 10.00), the coordinate value of A- is (-14, 25, 21, 00, 10.00), and the coordinate value of A- is (14, 25, 21, 00, 10.00).

[0042] In order to obtain the above-mentioned circular pixel point set, the circular feature detection of the charging port can be performed based on the Hough circle algorithm according to the approximate recognition range of the charging port from the camera, the maximum and minimum radius values of the detection circle are set, and a preset radius set is obtained. The detected circular information is extracted based on the preset radius set; in order to obtain the above-mentioned straight line pixel point set, the Hough line can be used to perform straight line detection within the circle enclosed by the circular pixel point set. If a first straight line is detected whose radius meets the preset ratio with the first circle, and the circle and the first straight line form a closed area in the same plane, then the closed area formed by the first straight line and the circle can be considered as the charging port area, such as Figure 2 shown.

[0043] The basic idea of the Hough circle algorithm is that every non-zero pixel may be a point on the circle. By converting it into polar coordinates, a voting mechanism is used to find the circle.

[0044] Specifically, the operation of obtaining the above circular pixel point set based on the Hough circle algorithm is as follows:

[0045] The equation of a circle in Cartesian coordinates can be defined as:

[0046]

[0047] Where x and y are the positions of each pixel, a and b are the coordinates of the center of the circle, and r is the radius of the circle.

[0048] For a given point (x, y), we can draw all circles in the three-dimensional coordinate system and perform the above operation on all coordinate points in the space. If two different points intersect in the abr space, that is, they have a common set of (a, b, r), they are on the same circle. The more intersections there are, the more points the circle is composed of. By setting a threshold, we can obtain the circular features around the charging port.

[0049] For example, in the above operation, according to the actual scenario, the detection radius range is set to [120, 320]. In order to detect the appropriate circular feature, the detection center distance can be set to infinity to eliminate irrelevant interference.

[0050] The basic idea of the Hough line algorithm is to transform between rectangular coordinates and polar coordinates, map a line through the points, traverse all pixels, and the intersection of the polar coordinate curve is the possible line detected.

[0051] Specifically, the operation of obtaining the above straight line pixel point set based on the Hough line algorithm is as follows:

[0052] A straight line can be represented by two variables in the two-dimensional image space. We transform it into polar coordinates, so the straight line expression can be expressed as:

[0053]

[0054] Right now:

[0055] ρ=x cosθ+y sinθ (3)

[0056] Where ρ is the distance from the origin to the straight line, θ is the angle between the perpendicular line from the origin to the straight line and the x-axis, and x and y are the positions of each pixel on the image.

[0057] Substitute each point (x, y) on the image boundary into equation (3) to obtain the ρ value corresponding to each θ; in the parameter space, find the unit corresponding to ρ and θ, and add 1 to the accumulator of the unit; when the above operation is performed on all points in the rectangular coordinate system, check the value of each accumulator in the parameter space. The ρ and θ corresponding to the unit with the largest accumulator are the parameters of the straight line equation in the rectangular coordinate system.

[0058] For example, during the above operation, according to the actual scenario, the line segment distance accuracy is set to 1 pixel, the line segment angle accuracy is set to CV_PI / 180 pixels, the accumulator threshold parameter is set to 100, the minimum length is set to 20 pixels, and the maximum split of the same straight line is set to 5 pixels.

[0059] In the above step S206, the first charging position of the charging port can be obtained by using similar triangles based on the area of the pixel area corresponding to the charging port in step S204 and the position information of the center point of the charging port in the image, that is, the position coordinates of the roughly determined charging port position are calculated. The robot can drive the charging gun to make the first adjustment to complete the process of roughly positioning the charging port area, such as Figure 2 shown.

[0060] As an optional embodiment, the first charging position corresponding to the charging port is determined based on at least the area of the pixel area and the position of the first pixel point in the pixel area, including: obtaining the focal length of the image collector, wherein the image collector is used to collect the charging port image; establishing a two-dimensional coordinate system with the center point of the charging port image as the coordinate origin and the extension directions of any two straight lines perpendicular to each other in the charging port image as the x-axis and y-axis; determining the spatial position of the first center point relative to the image collector based on the projected area of the charging port, the area of the pixel area, and the x and y coordinate values of the first pixel point in the two-dimensional coordinate system, wherein the spatial position includes a first lateral distance and a first longitudinal distance, the first lateral distance being the lateral distance of the first center point relative to the image collector, and the first longitudinal distance being the longitudinal distance of the first center point relative to the image collector; determining the distance between the image collector and the charging port based on the first lateral distance, the first longitudinal distance, and the focal length of the image collector.

[0061] Specifically, the calculation principle of the coarse positioning of the charging port position is obtained based on the camera pinhole imaging model, such as Figure 4 As shown. c is the camera coordinate system, O is the image physical coordinate system, O i is the image pixel coordinate, O w is the world coordinate system, and the dotted line axis is the optical axis. w The coordinate system is established at the intersection of the plane of the charging port rectangular area and the optical axis; w and O c The coordinate system remains consistent. Since coarse positioning does not require calculation of the charging port posture, it is assumed here that the plane of the charging port rectangular area is parallel to the camera imaging plane O-xy.

[0062] Figure 5 The closed figure composed of the large circle and the straight line is the charging port area in the three-dimensional world coordinates, and its circle diameter D wThe closed figure composed of small circles and straight lines is the projection of the charging port area on the camera imaging surface. A is the target point, that is, the center point of the charging port area. w The coordinates in the coordinate system are (X, Y, 0). a is the projection point of the target point on the camera imaging surface, which is at O i The coordinates of the coordinate system are (u, v), and its coordinates in the O coordinate system are a(x, y). w Coordinate system and O c The distance of the coordinate system is D z Because O w and O c The coordinate system has the same posture, so the target point A is at O c The three-dimensional coordinates in the coordinate system are (X, Y, Dz), which is the result of rough positioning. According to the principle of similar triangles, we can get:

[0063]

[0064] Where X, Y are the actual spatial positions of the target point relative to the camera. x, y are the pixel positions of the target point in the image; D w Represents the size of the target circular feature; D i Represents the size of the pixels occupied by the circular feature.

[0065] Figure 5 The relationship between pixel coordinates and physical coordinates is shown in the figure. Taking the camera resolution of the automatic charging system as 1292×964 as an example, the O coordinate system is in O i The coordinates in the coordinate system are (646, 482), so O i The relationship between the coordinates and the O coordinates is:

[0066]

[0067] According to the Pythagorean theorem, the world coordinate system O can be obtained w The distance D from the origin to point A is:

[0068]

[0069] The pixel size of the camera is 3.75um×3.75um, so the distance d from point a to the origin of the O coordinate system is:

[0070]

[0071] According to the principle of similar triangles, O w and O c Distance D of the coordinate system z , as follows:

[0072]

[0073] Where f is the focal length of the camera, Dz is the distance from the camera to the charging port, and D is the world coordinate system O w The distance from the origin to point A, d is the distance from point a to the origin of the O coordinate system.

[0074] In summary, it is deduced that the target point A is at O c The coordinate value in the coordinate system, that is, the final result of rough positioning is:

[0075]

[0076] According to formula (9), the coarse positioning of the charging port position information can be calculated, thereby guiding the robot end to move to the camera focal length position, providing a guarantee for the charging port posture positioning.

[0077] In the above step S208 , obtaining the second pixel point in the pixel area includes: extracting a characteristic pattern in the pixel area; and determining the center point of the characteristic pattern as the second pixel point.

[0078] As an optional embodiment, extracting characteristic graphics in a pixel area includes: extracting multiple contour graphics in the pixel area; obtaining a set of circumscribed rectangles of each contour graphic in the multiple contour graphics; determining at least one first contour graphic from the multiple contour graphics according to preset conditions, wherein the above-mentioned preset conditions include: the set of circumscribed rectangles of the first contour graphics satisfies a preset aspect ratio range, and the length of the set of circumscribed rectangles of the first contour graphics satisfies a preset length; fitting at least one first contour graphic into at least one target graphic; and determining a characteristic graphic from at least one target graphic based on a preset shape, wherein the preset shape includes a circle and an ellipse.

[0079] When there are multiple target patterns and multiple first characteristic patterns are determined from the multiple target patterns based on a preset shape, target patterns whose length-to-short diameter ratio is inconsistent with the preset length-to-short diameter ratio can be removed from the multiple first characteristic patterns to obtain a characteristic pattern.

[0080] Specifically, since the charging port image is usually affected by various noises, including photoelectric noise, digital noise, compression noise, etc., Gaussian filtering can be performed based on the pixel area identified by coarse positioning. Then, the contour information of the charging port image (original image) and its gradient map can be extracted using the Canny operator. All the bounding rectangles of the contour information of the two are obtained, and the aspect ratio of the bounding rectangles is screened. The contour information with an aspect ratio between 1 and 1.2 is retained. Then, the contour information is fitted into a circle or ellipse using the quadratic curve normalization method. Finally, the position information and the aspect ratio of the fitted figure are screened to eliminate the misdetected figures, such as Figure 2 shown.

[0081] The quadratic curve standardization method is used to fit the graphic features. In the plane coordinate system, the points on the fitted graph can be expressed by a quadratic polynomial. In order to obtain the standard curve equation, the coordinates of the plane coordinate system are first fitted to determine the coefficients of the quadratic curve equation in the plane coordinate system. Then, the quadratic mixed multiplication terms and linear terms are eliminated through translation and rotation between the coordinate systems. Finally, the curve equation in the standard coordinate system is obtained, thereby determining the parameters of the fitted graph.

[0082] Suppose the coordinates of n points observed on the quadratic curve on the plane are (x i ,y i )=M i (i=1, 2, ..., n), the quadratic curve equation can be expressed as:

[0083] a0+a1x+a2y+a3x 2 +a4xy+a5y 2 =0 (10)

[0084] Where a1, a2…, a5 are unknown coefficients. After normalizing the coordinates, the quadratic equation can be expressed as:

[0085] 1+a′1x′+a′2y′+a′3x 2 +a′4x′y′+a′5y 2 =0 (11)

[0086] In the formula, a′1, a′2…a′5 are unknown coefficients. The error equation for formula (11) is:

[0087]

[0088] The initial values of parameters a′1, a′2…a′5 are set to 0, and the error equations are listed for all measurement points. The equations are solved, and the absolute values of the solved parameters are taken. The iteration is continued until convergence is achieved, thereby obtaining the coefficients of the curve equation. The geometric parameter characteristic parameters of the normalized fitting graph can be expressed as:

[0089]

[0090] Combining the contour information of the two, according to the length-to-width ratio of the fitting figure, circles with a ratio of 1 or ellipses with a ratio between 1 and 1.2 are screened out, and interference circles are eliminated according to the relative position relationship, and finally the actual effective feature figure is obtained. When there are partially overlapping fitting figures on the same feature figure, the average value of the centers of these fitting figures is calculated as the feature point of the feature figure, that is, the second center point.

[0091] In the above step S210, generating the posture information corresponding to the charging port based on the first charging position and the position of the second pixel point in the pixel area may include: when the number of extracted feature graphics is greater than a preset number, determining the second charging position of the charging port in the charging port based on the projection relationship between the position of the second pixel point in the pixel area and the three-dimensional coordinate system, wherein the preset number is less than the number of charging holes, and the preset number is greater than 1 / 2 of the number of charging holes; generating the posture information corresponding to the charging port based on the first charging position and the second charging position.

[0092] Specifically, Figure 3 Taking the charging port shown in the figure as an example, the preset number can be set to 6. When the number of characteristic patterns is greater than 6, the position of the center pixel point of each charging pinhole is combined with the position of the charging pinhole relative to the three-dimensional coordinate point to obtain the posture information of the charging port relative to the camera according to the PnP algorithm, and then converted into the posture information executed by the robot to accurately locate the complete charging port area. The above posture information guides the robot to complete the plug-in task, such as Figure 2 shown.

[0093] The PnP algorithm uses a 3D to 2D point mapping relationship to determine the pose. Once the real coordinates of N spatial points in the world coordinate system and the projections of the corresponding spatial points on the image are known, the pose of the camera can be calculated, namely the movement t(x, y, z) and rotation R(Rx, Ry, Rz) vectors of the two. There are multiple solution methods based on the PnP algorithm, and different methods require different numbers of feature points. Since there are feature points corresponding to multiple charging holes in the charging port, the Direct Linear Transform (DLT) algorithm can be used to solve the pose of the charging port.

[0094] The principle of the DLT algorithm is as follows: a point P in the charging port image, its coordinates are P(X, Y, Z, 1) T , the characteristic point on the imaging surface is x1=(μ1,v1,1) T At this point, the camera's position t(x, y, z) and posture R(Rx, Ry, Rz) are unknown. Define R, t as a 3×4 matrix that contains translation and rotation information, as shown in Equation (14):

[0095]

[0096] Eliminate s with the last line and obtain two constraints, as shown in formula (15):

[0097]

[0098] To simplify the representation, define the row vector of T:

[0099]

[0100] Thus, the two constraints can be rewritten as:

[0101]

[0102] Each characteristic point of the charging port can give two constraint equations such as Equation (17). When the number of characteristic points of the charging port is N, the following set of equations can be obtained:

[0103]

[0104] It can be seen from formula (18) that when using this method to locate the charging port, at least six feature points of the charging port can be obtained to complete the position solution of the charging port.

[0105] Example 2

[0106] According to an embodiment of the present invention, a device for implementing the above-mentioned charging port identification method is also provided. Figure 6 This is a structural block diagram of a charging port identification device according to embodiment 2 of the present invention. As shown in Figure 6, the device includes: a first acquisition module 302, an extraction module 304, a determination module 306, a second acquisition module 308, and a generation module 310. The device is described in detail below.

[0107] A first acquisition module 302 is used to acquire an image of the charging port;

[0108] An extraction module 304 is configured to extract a pixel region corresponding to the charging port and a first pixel point in the pixel region from the charging port image, wherein the first pixel point is a pixel point in the pixel region corresponding to a first center point of the charging port;

[0109] a determination module 306 for determining a first charging position corresponding to the charging port based on at least an area of the pixel region and a position of the first pixel point in the pixel region;

[0110] A second acquisition module 308 is configured to acquire a second pixel point in the pixel area, wherein the second pixel point is a pixel point in the pixel area corresponding to a second center point of the charging hole in the charging port;

[0111] The generating module 310 is used to generate the posture information corresponding to the charging port according to the first charging position and the position of the second pixel point in the pixel area, wherein the posture information is the posture information of the charging gun to be inserted into the charging port.

[0112] It should be noted here that the above-mentioned first acquisition module 302, extraction module 304, determination module 306, second acquisition module 308 and generation module 310 correspond to steps S202 to S210 in Example 1. The instances and application scenarios implemented by multiple modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1.

[0113] Example 3

[0114] An embodiment of the present invention can provide a charging system, including: a vehicle to be charged, the vehicle to be charged having a charging port; a charging robot, the charging robot having a charging gun; a controller, the controller being used to receive posture information corresponding to the charging port generated by the above-mentioned charging port identification method, and inserting the charging gun into the charging port based on the posture information.

[0115] Using the charging port recognition method described in Example 1, the positional information corresponding to the charging port is generated. While the specific local structures of different vehicle models may vary, as long as they comply with national standards, the robot can intelligently learn by collecting image data of unfamiliar charging ports, reconstructing their features, and storing them in an internal database. Subsequently, the robot can accurately locate the position of similar charging ports it encounters.

[0116] Based on repeated experiments with charging robots, the charging port image recognition processing system has a high degree of position recognition of the charging port, with a position recognition accuracy of 0.01mm and an angle accuracy of 0.01rad, or 0.57° (camera positioning accuracy).

[0117] Example 4

[0118] An embodiment of the present invention may provide an electronic device, which may be a terminal or a server. In this embodiment, the electronic device, as a terminal, may be any computer terminal device in a computer terminal group. Alternatively, in this embodiment, the terminal may also be a terminal device such as a mobile terminal.

[0119] Optionally, in this embodiment, the terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0120] Optionally, Figure 7 FIG. 1 is a block diagram of a terminal according to an exemplary embodiment. Figure 7 As shown, the terminal may include: one or more (only one is shown in the figure) processors 41, and a memory 42 for storing processor executable instructions; wherein the processor is configured to execute instructions to implement any of the above-mentioned charging port identification methods.

[0121] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the charging port identification method and device in the embodiments of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned charging port identification method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the charging port image; extract the pixel area corresponding to the charging port and the first pixel point in the pixel area from the charging port image, wherein the first pixel point is the pixel point corresponding to the first center point of the charging port in the pixel area; determine the first charging position corresponding to the charging port based on at least the area of the pixel area and the position of the first pixel point in the pixel area; obtain the second pixel point in the pixel area, wherein the second pixel point is the pixel point corresponding to the second center point of the charging hole in the charging port in the pixel area; generate posture information corresponding to the charging port based on the first charging position and the position of the second pixel point in the pixel area, wherein the posture information is the posture information of the charging gun to be inserted into the charging port.

[0123] Optionally, the processor may also execute program code for the following steps: the charging port includes a charging port base, the charging port passes through the charging port base, and a pixel area corresponding to the charging port is extracted from the charging port image, including: establishing a three-dimensional coordinate system with the first center point of the charging port as the coordinate origin, the extension directions of any two straight lines perpendicular to each other in the plane where the charging port base is located as the X-axis and Y-axis, and the penetration direction of the charging port as the Z-axis; extracting a circular pixel point set from the charging port image, wherein each circle corresponding to the circular pixel point set is a circular area with the coordinate origin of the three-dimensional coordinate system as the center and a radius in a preset radius set as the radius; obtaining a straight line pixel point set that satisfies a preset ratio with the preset radius set from the charging port image; extracting a first circle and a first straight line from the circular pixel point set and the straight line pixel point set, wherein the first circle and the first straight line constitute a closed area in the same plane, and the closed area is used as the pixel area corresponding to the charging port.

[0124] Optionally, the processor may also execute program code for the following steps: determining a first charging position corresponding to the charging port based at least on the area of the pixel region and the position of the first pixel point in the pixel region, including: obtaining the focal length of the image collector, wherein the image collector is used to collect images of the charging port; establishing a two-dimensional coordinate system with the center point of the charging port image as the coordinate origin and the extension directions of any two perpendicular straight lines in the charging port image as the x-axis and y-axis; determining the spatial position of the first center point relative to the image collector based on the projected area of the charging port, the area of the pixel region, and the x and y coordinate values of the first pixel point in the two-dimensional coordinate system, wherein the spatial position includes a first lateral distance and a first longitudinal distance; determining the distance between the image collector and the charging port based on the first lateral distance, the first longitudinal distance, and the focal length of the image collector.

[0125] Optionally, the processor may further execute program code of the following steps: obtaining a second pixel point in the pixel area, including: extracting a characteristic pattern in the pixel area; and determining the center point of the characteristic pattern as the second pixel point.

[0126] Optionally, the processor may also execute program code for the following steps: extracting characteristic graphics in a pixel area, including: extracting multiple contour graphics in the pixel area; obtaining a set of circumscribed rectangles of each contour graphic in the multiple contour graphics; determining at least one first contour graphic from the multiple contour graphics according to preset conditions, wherein the preset conditions include: the set of circumscribed rectangles of the first contour graphic satisfies a preset aspect ratio range, and the length of the set of circumscribed rectangles of the first contour graphic satisfies a preset length; fitting at least one first contour graphic into at least one target graphic; and determining a characteristic graphic from at least one target graphic based on a preset shape, wherein the preset shape includes a circle and an ellipse.

[0127] Optionally, the processor may also execute the program code of the following steps: extracting characteristic patterns in the pixel area, further comprising: when there are multiple target patterns and multiple first characteristic patterns are determined from the multiple target patterns based on a preset shape, removing the target patterns whose length-to-short diameter ratio is inconsistent with the preset length-to-short diameter ratio from the multiple first characteristic patterns to obtain the characteristic pattern.

[0128] Optionally, the processor may also execute the program code of the following steps: generating posture information corresponding to the charging port based on the first charging position and the position of the second pixel point in the pixel area, including: when the number of extracted feature graphics is greater than a preset number, determining the second charging position of the charging port in the charging port based on the projection relationship between the position of the second pixel point in the pixel area and the three-dimensional coordinate system, wherein the preset number is less than the number of charging holes, and the preset number is greater than 1 / 2 of the number of charging holes; generating posture information corresponding to the charging port based on the first charging position and the second charging position.

[0129] It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only. For example, the terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 7 It does not limit the structure of the above electronic equipment. For example, it may also include Figure 7 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 7 Different configurations shown.

[0130] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0131] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0132] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0134] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0135] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0136] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0137] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A charging port identification method, characterized in that: include: Get the charging port image; Extracting a pixel region corresponding to the charging port and a first pixel point in the pixel region from the charging port image, wherein the first pixel point is a pixel point in the pixel region corresponding to a first center point of the charging port; Determining a first charging position corresponding to the charging port based at least on the area of the pixel region and the position of the first pixel point in the pixel region, comprising: obtaining a focal length of an image collector, wherein the image collector is used to capture the image of the charging port; establishing a two-dimensional coordinate system with the center point of the charging port image as the coordinate origin and the extension directions of any two mutually perpendicular straight lines in the charging port image as the x-axis and y-axis; determining the spatial position of the first center point relative to the image collector based on the projected area of the charging port, the area of the pixel region, and the x and y coordinate values of the first pixel point in the two-dimensional coordinate system, wherein the spatial position includes a first lateral distance and a first longitudinal distance; determining the distance between the image collector and the charging port based on the first lateral distance, the first longitudinal distance, and the focal length of the image collector; Obtaining a second pixel point in the pixel area includes: extracting a characteristic pattern in the pixel area; determining a center point of the characteristic pattern as the second pixel point, wherein the second pixel point is a pixel point in the pixel area corresponding to a second center point of a charging hole in the charging port, the charging port includes a charging port base, and establishing a three-dimensional coordinate system with the first center point of the charging port as a coordinate origin, with the extension directions of any two mutually perpendicular straight lines in a plane where the charging port base is located as the X-axis and the Y-axis, and with the penetration direction of the charging hole as the Z-axis; Based on the positions of the first charging position and the second pixel point in the pixel area, the posture information corresponding to the charging port is generated, including: when the number of the extracted characteristic patterns is greater than the preset number, based on the projection relationship between the position of the second pixel point in the pixel area and the three-dimensional coordinate system, the second charging position of the charging port in the charging port is determined, wherein the preset number is less than the number of the charging ports, and the preset number is greater than 1 / 2 of the number of the charging ports; based on the first charging position and the second charging position, the posture information corresponding to the charging port is generated, wherein the posture information is the posture information of the charging gun to be inserted into the charging port.

2. The charging port identification method according to claim 1, characterized in that: The charging hole penetrates the charging port base, and extracting a pixel area corresponding to the charging port from the charging port image includes: Extracting a circular pixel point set from the charging port image, wherein each circle corresponding to the circular pixel point set is a circular area having the origin of the three-dimensional coordinate system as the center and a radius selected from a preset radius set as the radius; Acquire a set of straight line pixel points that satisfy a preset ratio with the preset radius set from the charging port image; A first circle and a first straight line are extracted from the circular pixel point set and the straight line pixel point set, wherein the first circle and the first straight line form a closed area in the same plane, and the closed area is used as the pixel area corresponding to the charging port.

3. The charging port identification method according to claim 1, characterized in that: Extracting the characteristic pattern in the pixel area includes: extracting a plurality of contour graphics in the pixel area; Obtaining a set of bounding rectangles of each of the plurality of contour graphics; Determining at least one first outline graphic from the plurality of outline graphics according to preset conditions, wherein the preset conditions include: a set of circumscribed rectangles of the first outline graphics satisfying a preset aspect ratio range, and a length of the set of circumscribed rectangles of the first outline graphics satisfying a preset length; fitting the at least one first contour figure into at least one target figure; The characteristic pattern is determined from the at least one target pattern according to a preset shape, wherein the preset shape includes a circle and an ellipse.

4. The charging port identification method according to claim 3, characterized in that: The extracting of the characteristic pattern in the pixel area further includes: In the case where there are multiple target patterns and multiple first characteristic patterns are determined from the multiple target patterns based on the preset shape, target patterns whose length-to-short diameter ratio is inconsistent with the preset length-to-short diameter ratio are removed from the multiple first characteristic patterns to obtain the characteristic pattern.

5. A charging port identification device, characterized in that: include: A first acquisition module is used to acquire an image of the charging port; an extraction module, configured to extract, from the charging port image, a pixel region corresponding to the charging port and a first pixel point in the pixel region, wherein the first pixel point is a pixel point in the pixel region corresponding to a first center point of the charging port; A determination module, configured to determine a first charging position corresponding to the charging port based at least on the area of the pixel region and the position of the first pixel point in the pixel region, comprising: obtaining a focal length of an image collector, wherein the image collector is configured to capture an image of the charging port; establishing a two-dimensional coordinate system with the center point of the charging port image as the coordinate origin and the extension directions of any two mutually perpendicular straight lines in the charging port image as the x-axis and y-axis; determining a spatial position of the first center point relative to the image collector based on the projected area of the charging port, the area of the pixel region, and the x and y coordinate values of the first pixel point in the two-dimensional coordinate system, wherein the spatial position includes a first lateral distance and a first longitudinal distance; determining the distance between the image collector and the charging port based on the first lateral distance, the first longitudinal distance, and the focal length of the image collector; a second acquisition module, configured to acquire a second pixel point in the pixel area, comprising: extracting a characteristic pattern in the pixel area; and determining a center point of the characteristic pattern as the second pixel point, wherein the second pixel point is a pixel point in the pixel area corresponding to a second center point of the charging hole in the charging port, the charging port including a charging port base, and establishing a three-dimensional coordinate system with the first center point of the charging port as a coordinate origin, with the extension directions of any two mutually perpendicular straight lines in a plane where the charging port base is located as the X-axis and the Y-axis, and with the penetration direction of the charging hole as the Z-axis; A generation module is used to generate posture information corresponding to the charging port based on the positions of the first charging position and the second pixel point in the pixel area, including: when the number of the extracted characteristic patterns is greater than a preset number, determining the second charging position of the charging port in the charging port based on the projection relationship between the position of the second pixel point in the pixel area and the three-dimensional coordinate system, wherein the preset number is less than the number of the charging ports, and the preset number is greater than 1 / 2 of the number of the charging ports; generating the posture information corresponding to the charging port based on the first charging position and the second charging position, wherein the posture information is the posture information of the charging gun to be inserted into the charging port.

6. A charging system, characterized in that: include: A vehicle to be charged, wherein the vehicle to be charged has a charging port; A charging robot having a charging gun; A controller, the controller being configured to receive posture information corresponding to the charging port generated by the charging port identification method according to any one of claims 1 to 4, and to insert the charging gun into the charging port based on the posture information.

7. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the charging port identification method according to any one of claims 1 to 4.

Images