Charging seat posture acquisition method and device, electronic equipment and readable storage medium

Abstract

The invention provides a charging seat posture acquisition method and device, electronic equipment and a readable storage medium, and the method comprises the steps: obtaining an image of a charging seat and a preset reference model comprising a first coordinate of a preset part of the charging seat in a preset coordinate system, converting the first coordinate into an image coordinate system of the image through a coordinate transformation matrix, and obtaining a second coordinate, adjusting the coordinate transformation matrix, and returning to the operation of converting the first coordinate into the image coordinate system through the coordinate transformation matrix to obtain a second coordinate until a preset cycle termination condition is met, obtaining a target coordinate transformation matrix, and obtaining the second coordinate through the target coordinate transformation matrix to meet a preset coordinate transformation requirement. According to the method, radar point cloud data does not need to be collected and processed, the method is simple, and the processing efficiency is high.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a method, device, electronic device and readable storage medium for acquiring a charging seat posture. Background Art

[0002] With the development of new energy vehicles, the charging technology of new energy vehicles has also developed, and users' demand for automatic charging of vehicles has also increased.

[0003] In related technologies, the image of the charging seat is captured by a camera, and the point cloud data of the charging seat is captured by a lidar. The position of the charging seat is obtained by combining the image of the charging seat and the point cloud data. The charging robot locates the charging seat according to the posture of the charging seat, and inserts the charging gun into the charging seat for charging.

[0004] However, the related technical methods require comprehensive processing of point cloud data collected by the lidar and image data collected by the camera to obtain the charging seat posture. The amount of data that needs to be processed is large, and the lidar is required to implement the solution. The equipment requirements are high and the processing method is complex. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention provide a charging seat posture acquisition method, device, electronic device and readable storage medium that overcome the above problems or at least partially solve the above problems.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] In the first aspect, an embodiment of the present application discloses a method for acquiring the posture of a charging seat, comprising: acquiring an image of the charging seat and a preset reference model of the charging seat; the preset reference model includes a first coordinate of the preset component in a preset coordinate system; converting the first coordinate to the image coordinate system of the image through a coordinate transformation matrix to obtain a second coordinate; adjusting the coordinate transformation matrix, and returning to the operation of converting the first coordinate to the image coordinate system through the coordinate transformation matrix to obtain the second coordinate, until a preset loop termination condition is met, and a target coordinate transformation matrix is ​​obtained; wherein the second coordinate obtained through the target coordinate transformation matrix meets the preset coordinate transformation requirements; and the posture of the charging seat is obtained according to the target coordinate transformation matrix.

[0008] In the second aspect, an embodiment of the present application discloses a device for acquiring the posture of a charging seat, comprising: a first acquisition module for acquiring an image of the charging seat and a preset reference model of the charging seat; the preset reference model includes the first coordinates of the preset components of the charging seat in a preset coordinate system; a second acquisition module for converting the first coordinates into the image coordinate system of the image through a coordinate transformation matrix to obtain a second coordinate; a third acquisition module for adjusting the coordinate transformation matrix and returning to the operation of converting the first coordinates into the image coordinate system through the coordinate transformation matrix to obtain the second coordinates until the preset loop termination condition is met and the target coordinate transformation matrix is ​​obtained; wherein the second coordinates obtained through the target coordinate transformation matrix meet the preset coordinate transformation requirements; a fourth acquisition module for obtaining the posture of the charging seat according to the target coordinate transformation matrix.

[0009] In a third aspect, an embodiment of the present application discloses an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0010] In a fourth aspect, an embodiment of the present application discloses a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0011] In an embodiment of the present application, an image of charging coordinates and a preset reference model of the charging station are acquired. The preset reference model includes first coordinates of preset components of the charging station in a preset coordinate system. The first coordinates are converted to the image coordinate system of the image using a coordinate transformation matrix to obtain second coordinates. The coordinate transformation matrix is ​​then adjusted to obtain a coordinate transformation matrix corresponding to a loop operation. The operation of converting the first coordinates to the image coordinate system using the coordinate transformation matrix to obtain the second coordinates is then repeated until a preset loop termination condition is satisfied, resulting in a target coordinate transformation matrix. The second coordinates obtained using the target coordinate transformation matrix meet the preset coordinate transformation requirements. Based on this target coordinate transformation matrix, the charging station pose can be accurately determined. This embodiment does not require acquiring and processing radar data to determine the charging station pose based on radar point cloud data, nor does it require calculating the 3D coordinates of the charging station based on two images obtained by a binocular camera and then determining the charging station pose based on the 3D coordinates. This embodiment obtains the charging station pose by combining the image and the target coordinate transformation matrix obtained from the preset reference model. Compared to related art methods, this embodiment requires less data processing, has high processing efficiency, low hardware requirements, and a simple method. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1This is a flowchart of a method for acquiring a charging seat posture provided by an embodiment of the present invention;

[0013] Figure 2 This is a schematic diagram of the first coordinates of a preset component in a reference model provided by an embodiment of the present invention;

[0014] Figure 3 This is a flowchart of another method for acquiring a charging seat posture provided by an embodiment of the present invention;

[0015] Figure 4 1 is a schematic diagram of detection results of a target detection model provided by an embodiment of the present invention;

[0016] Figure 5 This is a flowchart of another method for acquiring a charging seat posture provided by an embodiment of the present invention;

[0017] Figure 6 This is a block diagram of a charging seat posture acquisition device provided by an embodiment of the present invention;

[0018] Figure 7 is a block diagram of an electronic device provided in an embodiment of the present application;

[0019] Figure 8 This is a block diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0021] refer to Figure 1 , which shows a method for acquiring a charging seat posture provided by an embodiment of the present application, the method comprising:

[0022] Step 101: Acquire an image of a charging base and a preset reference model of the charging base.

[0023] The preset reference model includes the first coordinates of the preset component in the preset coordinate system.

[0024] The charging station can be a charging station in a vehicle. A monocular camera installed in the charging device can capture an image of the charging station. The monocular camera captures a two-dimensional (2D) image of the charging station. The charging device is a device used to dock with the vehicle's charging station to charge the docked vehicle. For example, the charging device can be a robotic arm with a charging gun connected to it, or a robot that controls the docking of the charging gun with the charging station.

[0025] The preset components are key parts used to position the charging station. For example, they can be the terminals in the charging station that connect to the charging gun. During automatic charging, the robotic arm aligns the charging gun with the charging station, connecting the charging gun and the terminals in the charging station to charge the vehicle. By setting the preset components as the terminals of the charging station and acquiring the charging station's position based on the preset components, we can ensure that the obtained charging station posture accurately instructs the robotic arm to successfully connect the charging gun and the terminals of the charging station.

[0026] The preset reference model of the charging base is a preset reference model that is compatible with the charging base. For example, the reference model is used to characterize at least one of the following characteristics of the charging base: shape, size, and position of a preset component in the charging base; wherein, the position of the preset component in the charging base can be represented by the first coordinate in the preset coordinate system.

[0027] For example, if a charging dock has seven terminals and the preset components are the terminals of the charging dock, the preset reference model can include at least one of the following characteristics of the charging dock: its shape and dimensions, and the first coordinate of each terminal in a preset coordinate system. The preset coordinate system can be a pre-built coordinate system or a world coordinate system.

[0028] For example, the model of the charging station can be obtained, and a preset reference model compatible with the charging station can be obtained based on the pre-stored correspondence between the charging station model and the reference model. For example, the charging station model can be obtained by scanning a barcode or text containing the charging station model information on the vehicle using the charging device. The charging station model can also be obtained based on an image of the charging station.

[0029] For example, the image of the charging station is a 2D image captured by a monocular camera. The preset reference model is a pre-built three-dimensional (3D) model.

[0030] For example, the preset component is a terminal, and there are 7 terminals. Each terminal has a 3D first coordinate, where the first coordinate can be used to represent the initial posture of the charging station. In the reference model, 7 cylinders can be used to represent the outline of the terminal, and the coordinates of the center of the cylinder can be used to represent the first coordinate of the terminal. Figure 2In the embodiment, the center of the central cylinder (i.e., the terminal located at the center) is used as the origin of the coordinate system, and a preset coordinate system is established in millimeters, wherein the coordinates of the seven terminals A1 to A7 are: (-8.0, 11.2, 0.0), (8.0, 11.2, 0.0), (-16.0, 0.0, 0.0), (0.0, 0.0, 0.0), (16.0, 0.0, 0.0), (-8.0, -13.9, 0.0), (8.0, -13.9, 0.0).

[0031] Step 102: Convert the first coordinates into the image coordinate system of the image through a coordinate transformation matrix to obtain a second coordinate.

[0032] For example, the coordinate transformation matrix may include a rotation matrix and a translation vector; or, the coordinate transformation matrix is ​​a matrix obtained by integrating the rotation matrix and the translation vector. For example, the coordinate transformation matrix is ​​multiplied by the first coordinate to obtain the second coordinate.

[0033] Step 103, adjust the coordinate transformation matrix, and return to the operation of converting the first coordinate into the image coordinate system through the coordinate transformation matrix to obtain the second coordinate, until the preset loop termination condition is met, and the target coordinate transformation matrix is ​​obtained; wherein the second coordinate obtained through the target coordinate transformation matrix meets the preset coordinate transformation requirements.

[0034] Specifically, this step includes at least one loop processing process, specifically, adjusting the coordinate transformation matrix to obtain the coordinate transformation matrix corresponding to the loop operation, and returning the coordinate transformation matrix to convert the first coordinate into the image coordinate system to obtain the second coordinate operation, until the preset loop termination condition is met and the target coordinate transformation matrix is ​​obtained.

[0035] The second coordinates obtained by the target coordinate transformation matrix meet the preset coordinate conversion requirements.

[0036] For example, the preset loop termination condition may include: the number of loops reaches a preset threshold, or in the current loop operation, the second coordinate obtained by the coordinate transformation matrix meets the preset coordinate transformation requirement.

[0037] Furthermore, if the loop termination condition is that the number of loops reaches a preset threshold, and the preset coordinate transformation requirements include: the reprojection error of the second coordinate corresponding to the target coordinate transformation matrix is ​​smaller than the reprojection errors of other second coordinates; then when the number of loops reaches the preset threshold, the loop operation is terminated, and from the coordinate transformation matrices corresponding to each loop operation, the target coordinate transformation matrix corresponding to the second coordinate that meets the preset coordinate transformation requirements is selected. The reprojection error of the second coordinate is obtained by the third coordinate and the second coordinate of the preset part in the image in the image coordinate system. The coordinate transformation matrix corresponding to the loop operation is the adjusted coordinate transformation matrix obtained by adjusting the coordinate transformation matrix in the loop operation. The coordinate transformation matrix corresponding to the first loop operation can be a preset initial coordinate transformation matrix.

[0038] Furthermore, if the loop termination condition is that in the current loop operation, the second coordinate obtained by the coordinate transformation matrix meets the preset coordinate transformation requirements, and the preset coordinate transformation requirements include: the reprojection error of the second coordinate corresponding to the target coordinate transformation matrix is ​​less than or equal to the preset reprojection error threshold, and the reprojection error of the second coordinate corresponding to the target coordinate transformation matrix reaches the minimum; then the loop is terminated when the preset coordinate transformation requirements are met, and the coordinate transformation matrix corresponding to the current loop operation is determined as the target coordinate transformation matrix.

[0039] In this embodiment, the number of loops is at least one. During each loop operation, the following operations are performed: the coordinate transformation matrix is ​​adjusted to obtain the coordinate transformation matrix corresponding to the loop operation; the first coordinate is converted to the image coordinate system using the coordinate transformation matrix to obtain the second coordinate. Because the loop operation is performed at least once, a corresponding coordinate transformation matrix is ​​obtained for each loop operation. Therefore, after the loop operation is terminated, at least one coordinate transformation matrix is ​​obtained.

[0040] The second coordinates obtained by the target coordinate transformation matrix meet the preset coordinate transformation requirements.

[0041] Based on this embodiment, at least one coordinate transformation matrix can be obtained, and the coordinate transformation matrix corresponds to the loop operation one-to-one. For the coordinate transformation matrix corresponding to each loop operation, the first coordinate is converted to the image coordinate system using the coordinate transformation matrix, thereby obtaining a second coordinate corresponding to the coordinate transformation matrix. The coordinate matrix corresponding to the first loop operation can be a preset coordinate transformation matrix.

[0042] For example, the preset coordinate transformation requirement may be that a reprojection error of a second coordinate obtained by performing coordinate transformation on the first coordinate based on the coordinate transformation matrix satisfies a preset error requirement.

[0043] Step 104 : Obtain the position and posture of the charging base according to the target coordinate transformation matrix.

[0044] For example, the target coordinate transformation matrix includes a rotation matrix and a translation vector; the rotation matrix represents the rotation angle, and the translation vector represents the translation amount. Furthermore, the rotation matrix and translation vector in the target coordinate transformation matrix can be used to determine the position and posture of the charging station. The position and posture include 6D data, specifically including the translation amounts along the X, Y, and Z axes of the camera coordinate system, as well as the pitch, yaw, and roll angles.

[0045] In the field of vehicle charging, such as new energy vehicles, charging robots can automatically charge vehicles, thereby realizing a fully intelligent vehicle usage scenario with autonomous driving, automatic parking, and automatic charging. By automatically charging the vehicle, users can have a more convenient vehicle charging experience. Specifically, the charging robot locates the position of the charging base, plans the motion trajectory of the robotic arm, adjusts the position of the charging base plug at the end of the robotic arm to match the position of the charging base, and then inserts the charging plug into the charging base to charge the vehicle.

[0046] In related technologies, the charging station is usually located based on traditional visual detection methods and target segmentation methods. For example, in related technologies, a binocular camera can be used to obtain an image of the charging station, and then a laser scanning device can be used to obtain the depth point cloud data of the charging station. The image and point cloud data are then processed together to obtain the posture of the charging station, and the charging station is located based on the posture. However, this method requires the binocular camera and laser scanning device in the charging robot to implement, which will lead to high cost, large amount of data processing, and low processing efficiency of the charging robot.

[0047] Related technologies can also use cameras and lidar to collect vehicle images and three-dimensional point cloud data, pre-process the images and the three-dimensional point cloud data collected by the lidar, and then automatically identify the two-dimensional position of the charging port through a deep learning model. The least squares method is used to obtain three-dimensional coordinates to accurately locate the spatial position and posture of the port. However, this method also relies on expensive hardware such as lidar, which leads to high costs for charging robot products. The algorithm also needs to process a large amount of point cloud data obtained by the lidar, resulting in high computational overhead and low processing efficiency.

[0048] In this embodiment, an image of the charging coordinates and a preset reference model of the charging station are acquired. The preset reference model includes first coordinates of preset components of the charging station in a preset coordinate system. The first coordinates are converted to the image coordinate system of the image using a coordinate transformation matrix to obtain second coordinates. The coordinate transformation matrix is ​​then adjusted to obtain a coordinate transformation matrix corresponding to the loop operation. The operation of converting the first coordinates to the image coordinate system using the coordinate transformation matrix to obtain the second coordinates is repeated until a preset loop termination condition is satisfied, resulting in a target coordinate transformation matrix. The second coordinates obtained using the target coordinate transformation matrix meet the preset coordinate transformation requirements, and the charging station pose is obtained based on the target coordinate transformation matrix. This embodiment does not require acquiring and processing radar data to obtain the charging station pose based on radar point cloud data, nor does it require calculating the 3D coordinates of the charging station based on two images obtained by a binocular camera and then calculating the charging station pose based on the 3D coordinates. This embodiment obtains the charging station pose by combining the image and the target coordinate transformation matrix obtained from the preset reference model. Compared to related art methods, this embodiment requires less data processing, has high processing efficiency, low hardware requirements, and a simple method.

[0049] In addition, by converting the first coordinate of the preset component in the preset reference model into the image coordinate system, the second coordinate is obtained, and then the coordinate transformation matrix is ​​adjusted, and the first coordinate is converted back to the image coordinate system through the coordinate transformation matrix to obtain the second coordinate operation until the preset loop termination condition is met and the target coordinate transformation condition is obtained. The coordinate transformation matrix is ​​optimized through multiple loop operations to obtain the target coordinate transformation matrix whose second coordinate meets the preset coordinate transformation requirements. The accuracy of the posture of the charging stand obtained according to the target coordinate transformation matrix is ​​high.

[0050] In one embodiment, Figure 3 The method for acquiring the charging seat posture of this embodiment may include the following steps:

[0051] Step 201 : Acquire an image of a charging base and a preset reference model of the charging base.

[0052] The preset reference model includes the first coordinates of the preset component in the charging base in the preset coordinate system.

[0053] Step 202 : Convert the first coordinates into the image coordinate system of the image through a coordinate transformation matrix to obtain second coordinates.

[0054] Step 203: Obtain the third coordinate of the preset component in the image in the image coordinate system.

[0055] For example, an image is input into the target detection model to obtain the third coordinate of the preset component in the image coordinate system.

[0056] For example, a monocular red-green-blue (RGB) three-channel camera can be used to capture an image containing the charging seat area, which is used as the input of a deep learning-based target detection model. After image preprocessing and feature extraction, a feature map is obtained. Then, model inference and post-processing are performed to obtain the pixel coordinates of the four corner points of the charging seat preset component annotation box as the model output; based on the pixel coordinates of the four corner points of the preset component annotation box, the center point of the annotation box is obtained, and the center point is determined as the third coordinate of the preset component in the image coordinate system.

[0057] For example, the target detection model can be a neural network model based on the target detection algorithm (You Only Look Once, YOLO), and the backbone network of the target detection model includes a Focus module, a basic convolution module, a cross-stage local network, a spatial pyramid pooling module, and a hierarchical downsampling structure. Among them, the target detection model also includes a multi-scale feature extraction layer, and the multi-scale feature extraction layer includes shallow scale features, middle scale features, and high scale features. Furthermore, the input image is enhanced and preprocessed by Mosaic data, and then passed to the backbone network of the target detection model for feature map extraction, and the feature map obtained is then subjected to multi-scale feature fusion of the feature map through the neck network, and the fused feature map is sent to the detection head to generate bounding box coordinates, confidence and category probability, and finally output the detection result; the bounding box coordinates are the annotation box used to mark the preset part.

[0058] For example, the deep learning target detection model can use the CSPDarknet53 network as the backbone network and adopt the Cross Stage Partial (CSP) strategy to reduce redundant gradient information and improve computational efficiency by splitting and merging feature maps. The basic convolution module consists of convolution, batch normalization, and Sigmoid Linear Unit (SiLU) activation function to extract feature maps. The spatial pyramid pooling module expands the receptive field through multi-scale pooling to improve feature expression capabilities while maintaining computational efficiency. The model neck network is a neural network model (Path Aggregation Network, PANet) for target detection and instance segmentation. Based on the PANet network model, the path aggregation capability of the feature map can be enhanced, improving the detection performance of small objects. The deep learning target detection model built based on the YOLO algorithm can detect the preset components in the charging base, improve the robustness of the detection results, and solve the problem of poor lighting conditions and high image noise in the underground garage environment, which leads to unclear charging base features and difficulty in extracting the contour features of the preset components in the charging base. It can also eliminate useless interference information in the image acquisition environment. Based on this model, the feature points of the charging base in the image and their pixel coordinates can be effectively extracted; among them, the feature point is the center point of the area where the preset component is located, and the pixel coordinates of the feature point are the third coordinates in this step.

[0059] A charging seat posture recognition model can be constructed using a neural network's deep learning algorithm. The model includes a model training unit that uses an RGB camera to capture images of the charging seat in parking and charging scenarios for vehicles such as household new energy vehicles. The model training unit identifies the charging seat area in the image, annotates the charging seat detection frame in the image, and then uses the annotated image to train the target detection model. The model also includes a model usage unit that quantizes the trained target detection model to reduce the model size and parameter count, enabling lightweight deployment on edge computing devices. This ensures the performance of the target detection model while reducing the computing pressure on the main control processor, thereby enabling more efficient identification of the charging seat and third coordinates.

[0060] For example, step 203 may include sub-steps A1 and A2:

[0061] Sub-step A1: obtaining a first number of a preset component in a preset reference model and a second number of the preset component in the image.

[0062] The preset reference model is a model generated according to the preset components of the charging base. The first number of the preset components in the preset reference model is obtained by counting the preset components in the preset reference model.

[0063] The image is input into the target detection model. The detection results input by the target detection model include a labeling box used to represent the preset component area, and an identification ID used to indicate that the labeling area of ​​the detection result is the preset component. The number of labeling boxes or identification IDs can be counted and determined as the second number of preset components in the image.

[0064] Sub-step A2: If the first number and the second number are equal, obtaining the third coordinate of the preset component in the image in the image coordinate system.

[0065] Furthermore, if the first number is not equal to the second number of the preset components in the preset reference model, it indicates that the preset components in the image were not fully recognized. This may be caused by the captured image not including the entire area of ​​the charging station, or it may be caused by an error in the model inference process. In this case, you can re-acquire the charging station image or re-predict the preset components in the image based on the object detection model until the first number of the preset components is equal to the second number of the preset components in the preset reference model.

[0066] For example, an image is input into a trained object detection model, and the object detection model outputs a recognition result for a predetermined part. The recognition result includes a box marking the predetermined part and the coordinates of the four corner points of the box in the image coordinate system. The third coordinate of the predetermined part in the image coordinate system can be the center point of the predetermined part, for example, the center point of a terminal.

[0067] Furthermore, the coordinates of the center point of the marked frame in the image coordinate system are obtained according to the coordinates of the four corner points of the marked frame, and the coordinates of the center point of the marked frame are determined as the third coordinates of the preset component in the image coordinate system.

[0068] For example, the obtained charging station image is as follows Figure 4 As shown, the charging base has 7 terminals, which are the preset parts in this embodiment. If the image includes the complete area of ​​the charging base and the detection result of the target detection model is accurate, then based on Figure 4 From the image shown, 7 feature point pixel coordinates can be obtained. One feature point pixel coordinate corresponds to the coordinate of the center of a terminal. The feature point pixel coordinate is the third coordinate of the preset component in the image coordinate system.

[0069] Further, in Figure 4In the illustrated embodiment, a determination is first made as to whether seven circle centers are detected in the image. Specifically, the detection results output by the object detection model have an identification ID. For example, the identification ID of class_id = 1 represents the charging dock terminal. First, the detection boxes with class_id = 1 are screened. If there are seven detection boxes, this indicates that the second number of the preset components detected in the image is equal to the second number of the preset components in the reference model.

[0070] Further, refer to Figure 4 , the annotation box output by the target detection model has four corner points, each of which has pixel coordinates in the image coordinate system. The pixel coordinates are the third coordinates in this embodiment. Figure 4 In the example, the target of detection is the cylindrical area of ​​the terminal inside the charging cradle. The characteristic point of the charging cradle is set as the center of the cross section of the cylindrical terminal. The coordinates of the center are the third coordinates of the preset component in this embodiment. The desired characteristic point coordinates can be calculated based on the coordinates of the four corner points of the annotation box. That is, by calculating the coordinates of the center point of the predicted box of the cylinder, the coordinates of the center of the cross section of the cylindrical terminal are obtained, thereby obtaining the pixel coordinates of the characteristic point (u i ,v i ),u i ,v i are the X-axis coordinates and Y-axis coordinates of the i-th preset component in the image coordinate system; among them, the pixel coordinates of the feature point (u i ,v i ) is the third coordinate in this embodiment.

[0071] In this embodiment, the first number of the preset components in the reference model and the second number of the preset components in the image are obtained. If the first and second numbers are equal, it indicates that the image includes the area of ​​all preset components in the charging station and the recognition result of the preset components is accurate. In this case, the third coordinate of the preset component in the image coordinate system is then obtained based on the image. This ensures that all preset components in the charging station are recognized, avoids missing preset components, and avoids the low accuracy of subsequent processing results caused by missed recognition of preset components. In other words, subsequent processing is performed only after ensuring that all preset components in the image are recognized, thereby improving the accuracy of the processing results.

[0072] For example, the preset component is a position point in the preset component of the charging base; step 203 may include sub-steps B1 to B2:

[0073] Sub-step B1: obtaining a labeling frame for labeling a preset component according to the image.

[0074] The preset component is the terminal of the charging base. The image is input into the trained object detection model to obtain a labeling box for labeling the preset component.

[0075] Sub-step B2: obtaining the third coordinate of the preset component in the image coordinate system according to the marked frame.

[0076] Based on this embodiment, a marking frame for marking the preset component is obtained according to the image, and the third coordinate of the preset component in the image coordinate system is obtained according to the marking frame.

[0077] For example, step 203 may include sub-steps C1 to C2:

[0078] Sub-step C1, obtaining the original coordinates of the preset component in the image coordinate system.

[0079] The original coordinates are the coordinates of the center point of the annotation box directly calculated based on the coordinates of the four corner points of the annotation box output by the object detection model. The annotation box is used to identify the area of ​​the preset component.

[0080] Sub-step C2: correcting the original coordinates according to preset correction parameters to obtain third coordinates.

[0081] The correction parameter may be a distortion parameter of a camera used to acquire the image.

[0082] For example, the distortion parameter may include a 5-dimensional double-precision floating-point vector, where the distortion parameter is used to describe lens distortion and can be divided into three radial distortion coefficients (k1, k2, k3) and two tangential distortion coefficients (p1, p2). The original coordinates can be corrected according to the distortion parameters of these five dimensions to obtain the third coordinate.

[0083] In practical applications, due to imperfections in the camera's optical system, images captured by the camera may exhibit varying degrees of distortion, including radial and tangential distortion. In this embodiment, after obtaining the feature point pixel coordinates (i.e., raw coordinates), the distortion parameters obtained by pre-calibrating the RGB camera are used to compensate for and correct the raw feature point pixel coordinates. Corrected feature point pixel coordinates are then obtained, and the charging station's posture is then resolved.

[0084] For example, the original coordinates of the charging base feature point in the image coordinate system can be corrected using the distortion parameters of the RGB monocular camera itself to obtain the corrected pixel coordinates of the charging base feature point. The corrected pixel coordinates of the charging base feature point are the third coordinates in this step. For example, the distortion correction method used in this embodiment can include: correcting the radial and tangential distortion models; specifically, using the camera's distortion equation and the Newton cycle method to approximate the undistorted coordinate values, thereby obtaining a correction result, and determining the correction result as the third coordinate of the preset component in the image coordinate system. Based on this, the accuracy of the third coordinate can be improved.

[0085] In this embodiment, the entire image is not corrected, but the distortion correction is performed on the original coordinates of the preset components in the image coordinate system. Therefore, the algorithm efficiency can be improved based on the successful optimization of the original coordinates of the preset components to be processed.

[0086] Step 204, obtaining a reprojection error of the second coordinate based on the second coordinate and the third coordinate;

[0087] The preset coordinate transformation requirements include: the reprojection error meets the preset error requirements.

[0088] For example, the deviation between the second coordinate and the third coordinate can be obtained, and the reprojection error of the third coordinate corresponding to the coordinate transformation matrix can be obtained based on the deviation. For example, the Euclidean distance or the square of the Euclidean distance between the second coordinate and the third coordinate can be obtained, and the reprojection error can be obtained based on the Euclidean distance or the square of the Euclidean distance.

[0089] Correspondingly, the preset coordinate transformation requirement in the aforementioned embodiment includes: a reprojection error of the second coordinate corresponding to the coordinate transformation matrix meets a preset error requirement. The preset error requirement may be: the reprojection error is less than or equal to a preset reprojection error threshold, or, during the loop processing of this embodiment, the reprojection error is minimized.

[0090] For example, the coordinate transformation matrix is ​​adjusted to obtain the coordinate transformation matrix corresponding to the loop operation, and the first coordinate is converted to the image coordinate system through the coordinate transformation matrix to obtain the second coordinate operation until the preset loop termination condition is met. Then, the reprojection error of the second coordinate corresponding to the coordinate transformation matrix of each loop operation is obtained, and the minimum reprojection error is obtained from multiple reprojection errors; the coordinate transformation matrix corresponding to the minimum reprojection error is determined as the target coordinate transformation matrix, and the charging seat posture is obtained based on the target coordinate transformation matrix. The loop termination conditions may include: the reprojection error is less than or equal to a preset reprojection error threshold, the reprojection error reaches a minimum, or the number of loops reaches a preset threshold.

[0091] Based on the third coordinate and the second coordinate obtained based on the coordinate transformation matrix, a reprojection error of the second coordinate corresponding to the coordinate transformation matrix is ​​obtained, and the preset coordinate transformation requirement is determined as follows: the reprojection error of the second coordinate corresponding to the coordinate transformation matrix meets the preset error requirement. Thus, the target coordinate transformation matrix obtained by adjusting the coordinate transformation matrix is ​​a target coordinate transformation matrix in which the reprojection error of the second coordinate corresponding to it meets the preset error requirement. Correspondingly, when the first coordinate is transformed based on the target coordinate transformation matrix, the reprojection error of the transformation result meets the preset error requirement. Based on this target coordinate transformation matrix, an accurate charging seat posture solution can be obtained.

[0092] For example, when there are multiple preset components, the reprojection error of the second coordinate is the overall reprojection error of the multiple second coordinates; then step 204 may include sub-steps D1 to D3:

[0093] Sub-step D1, obtaining multiple coordinate pairs according to the first coordinate and the third coordinate.

[0094] The coordinate pair corresponds to the preset component one by one; the coordinate pair includes the first coordinate and the third coordinate of the preset component corresponding to the coordinate pair.

[0095] For example, refer to Figure 2 and Figure 4 There are seven preset components, each of which has a first coordinate in the reference model and a third coordinate in the image. Based on this embodiment, the first coordinate and third coordinate of the first preset component can be determined as a coordinate pair, the first coordinate and third coordinate of the second preset component can be determined as a coordinate pair, and so on, resulting in seven coordinate pairs.

[0096] For example, sub-step D1 may include sub-steps D11 to D13:

[0097] Sub-step D11: obtaining a first sorting result of the first coordinate.

[0098] The first sorting result is a sorting result obtained by sorting the first coordinates according to a preset sorting strategy.

[0099] For example, the reference model can be reused, and the first sorting result of the first coordinates of the preset components in the reference model can be obtained and stored, and then the first sorting result of the first coordinates can be extracted from the storage file.

[0100] For example, the first coordinate is a 3D coordinate, including an X-axis coordinate and a Y-axis coordinate parallel to the end face of the socket in the reference model, and a Z-axis coordinate perpendicular to the end face of the socket. The preset sorting strategy may include: sorting the first coordinate according to the size of the X-axis coordinate and the Y-axis coordinate. Furthermore, the preset sorting strategy may include: arranging the coordinates according to the size of the Y-axis coordinate to obtain an initial arrangement result, the initial sorting result includes at least one row, and each row includes at least one coordinate; for the coordinates in the same row, sorting them according to the size of the X-axis coordinate to obtain the final sorting result. For example, in Figure 2 In the illustrated embodiment, the first sorting result obtained according to the preset sorting strategy is: the first coordinates in the first row are A1 and A2 in sequence; the first coordinates in the second row are A3, A4, and A5 in sequence; and the first coordinates in the third row are A6 and A7 in sequence.

[0101] Sub-step D12, sorting the third coordinates according to a preset sorting strategy to obtain a second sorting result;

[0102] The preset sorting strategy used to obtain the first sorting result is the same as the preset sorting strategy used to obtain the second sorting result.

[0103] Reference Figure 4 According to the preset sorting strategy in the above embodiment, the 7 third coordinates are sorted, and the second sorting result is: the third coordinates in the first row are B1 and B2 in sequence; the third coordinates in the second row are B3, B4, and B5 in sequence; and the third coordinates in the third row are B6 and B7 in sequence.

[0104] Sub-step D13, obtaining a plurality of coordinate pairs according to the first sorting result and the second sorting result.

[0105] For example, the first coordinate of the preset part is the 3D coordinate of the preset part in the reference model; the third coordinate of the preset part is the coordinate of the pixel of the feature point in the image, that is, the 2D coordinate of the preset part in the image. According to the size relationship between the multiple third coordinates, the 2D third coordinates in the detected image are sorted to ensure that the third sorting result matches the order of the first coordinates. In this way, a coordinate pair containing the first coordinate and the third coordinate of the preset component can be obtained, which can ensure that the 2D feature points in the coordinate pair correspond to the preset 3D feature points one by one. Subsequently, the third coordinate in the coordinate pair and the second coordinate corresponding to the first coordinate are reprojected. The error in the solution caused by the mismatch of the coordinate points can be avoided. By obtaining the first sorting result of the first coordinate, sorting the third coordinate according to the preset sorting strategy to obtain the second sorting result, and then according to the first sorting result and the second sorting result, multiple coordinate pairs including the first coordinate and the third coordinate of the preset component corresponding to the coordinate pair can be accurately obtained.

[0106] The first sorting result includes a first arrangement number of the first coordinate, and the second sorting result includes a second arrangement number of the third coordinate; sub-step D13 may include: obtaining a plurality of coordinate pairs according to the first arrangement number and the second arrangement number.

[0107] The first arrangement sequence number of the first coordinate in the coordinate pair is the same as the second arrangement sequence number of the third coordinate in the coordinate pair.

[0108] For example, in Figure 2 and Figure 4 In the embodiment shown, the arrangement sequence of the first coordinates is: A1 is the first in the first row; A2 is the second in the first row; A3 is the first in the second row; A4 is the second in the second row; A5 is the third in the second row;

[0109] A6 is the first in the third row; A7 is the second in the third row. The arrangement order of the third coordinate is: B1 is the first in the first row; B2 is the second in the first row; B3 is the first in the second row; B4 is the second in the second row; B5 is the third in the second row; B6 is the first in the third row; B7 is the second in the third row.

[0110] Furthermore, there are 7 coordinate pairs obtained based on this embodiment, namely, the coordinate pair including A1 and B1; the coordinate pair including A2 and B2; the coordinate pair including A3 and B3; the coordinate pair including A4 and B4; the coordinate pair including A5 and B5; the coordinate pair including A6 and B6; and the coordinate pair including A7 and B7.

[0111] Multiple coordinate pairs are obtained through the first arrangement number and the second arrangement number. Because the first arrangement number in the first sorting result and the second arrangement number in the second sorting result are both obtained by sorting through a preset sorting strategy, the first arrangement number of the first coordinate in the coordinate pair is the same as the second arrangement number of the third coordinate in the coordinate pair. Then, the arrangement position of the preset component corresponding to the first coordinate in the same coordinate pair among the multiple preset components of the charging seat is the same as the arrangement position of the preset component corresponding to the third coordinate in the multiple preset components of the charging seat. This ensures that the preset component corresponding to the first coordinate in the same coordinate pair and the preset component corresponding to the third coordinate are the same. Subsequently, the reprojection error calculation is performed on the third coordinate of the same preset component and the third coordinate corresponding to the first coordinate, which can avoid the reprojection error calculation error caused by coordinate mismatching and avoid the so-called charging seat posture solution error caused by the reprojection error calculation.

[0112] Sub-step D2, obtaining a reprojection error corresponding to the coordinate pair based on the third coordinate in the coordinate pair and the second coordinate corresponding to the first coordinate in the coordinate pair.

[0113] For example, the sum of squared Euclidean distances between the third coordinate of the preset component and the third coordinate of the projection position point can be obtained and used as the reprojection error of the projection coordinate point. Specifically, the sum of squared Euclidean distances Ei can be obtained according to the following method:

[0114]

[0115] in, and are the components of the first coordinate on the two coordinate axes of the image coordinate system, and are the components of the third coordinate on the two coordinate axes of the image coordinate system.

[0116] Sub-step D3, summing the reprojection errors corresponding to the multiple coordinate pairs to obtain the overall reprojection errors of the multiple second coordinates.

[0117] For example, the reprojection error corresponding to each preset component pair is summed to obtain the reprojection error. Specifically, the reprojection error E can be obtained according to the following method:

[0118]

[0119] According to the first coordinate and the third coordinate, a plurality of coordinate pairs are obtained; the coordinate pairs correspond to the preset components one-to-one, and the reprojection error corresponding to the coordinate pair is obtained according to the second coordinate in the coordinate pair and the third coordinate corresponding to the first coordinate in the coordinate pair. Because the coordinate pair includes the first coordinate and the third coordinate of the preset component corresponding to the coordinate pair, the reprojection error corresponding to the coordinate pair obtained based on this embodiment is obtained based on the coordinate data of the same preset component, avoiding the situation where the reprojection error is calculated based on the second coordinate and the third coordinate corresponding to different preset components. The reprojection error thus obtained can more accurately reflect the reprojection error when the coordinate transformation is performed based on the coordinate transformation matrix. In addition, the reprojection errors corresponding to the plurality of coordinate pairs are summed to obtain the overall reprojection error of the plurality of second coordinates. According to the overall reprojection error, the overall accuracy of the second coordinates corresponding to all preset components in the reference model during the coordinate transformation can be accurately reflected. Based on this, the coordinate transformation matrix is ​​adjusted and optimized and the target transformation matrix is ​​screened, and the target coordinate transformation matrix obtained is more accurate.

[0120] Step 205 , adjust the coordinate transformation matrix, and return to the operation of converting the first coordinate into the image coordinate system through the coordinate transformation matrix to obtain the second coordinate, until the preset loop termination condition is met, and the target coordinate transformation matrix is ​​obtained.

[0121] The second coordinates obtained by the target coordinate transformation matrix meet the preset coordinate transformation requirements. Further, the preset coordinate transformation requirements include: if the reprojection error meets the preset error requirement, then the second coordinates obtained by the target coordinate transformation matrix meet the preset coordinate transformation requirements.

[0122] Step 206: Obtain the position and posture of the charging base according to the target coordinate transformation matrix.

[0123] Furthermore, the preset coordinate transformation requirement includes: a reprojection error of the second coordinate corresponding to the coordinate transformation matrix meets a preset error requirement.

[0124] For example, the coordinate transformation matrix includes a rotation matrix and a translation vector; step 207 may include sub-steps E1 to E4:

[0125] Sub-step E1, obtaining an extrinsic parameter matrix according to the rotation matrix and the translation vector.

[0126] The extrinsic parameter matrix includes the rotation matrix R and the translation vector T, which can be expressed as [R|T].

[0127] Sub-step E2: determining an intrinsic parameter matrix of an image acquisition device used to acquire an image.

[0128] Among them, the intrinsic parameter matrix of the image acquisition device is a matrix used to describe the internal geometric and optical characteristics of the image acquisition device.

[0129] Sub-step E3, obtaining the product of the external parameter matrix, the internal parameter matrix, and the first coordinate.

[0130] Sub-step E4, obtaining the second coordinate according to the product.

[0131] For example, the third coordinate is obtained as follows:

[0132]

[0133] Among them, X i 、Y i and Z i is the 3D coordinate of the charging station feature point, i.e. the first coordinate in this step; and are the projected 2D coordinates, i.e., the second coordinates in this step. K is the camera's intrinsic parameter matrix, and [R|T] is the camera's extrinsic parameter matrix, which includes the rotation matrix R and the translation vector T. The rotation matrix and the translation vector are used to represent the position of the charging base in the camera coordinate system.

[0134] For example, by parsing R and T, the 6D pose of the object in the camera coordinate system is extracted from R and T. The 6D pose includes x, y, z coordinates and three Euler angles.

[0135] For example, the target coordinate transformation matrix includes the rotation matrix R:

[0136]

[0137] Among them, r 11 、r 21 and r 31 Respectively represent the components of the X-axis coordinate in the second coordinate on the X-axis, Y-axis, and Z-axis of the original coordinate system; r 12 、r 22 and r 32 Respectively represent the components of the Y-axis coordinate in the second coordinate on the X-axis, Y-axis, and Z-axis of the original coordinate system; r 13 、r 23 and r 33 They represent the components of the Y-axis coordinate in the second coordinate system on the X-axis, Y-axis, and Z-axis of the original coordinate system respectively.

[0138] Among them, the pitch angle of the charging station is θ = sin -1 (-r 31 ), the yaw angle is The roll angle is

[0139] The target coordinate transformation matrix includes the translation vector T: R = (x, y, z), then the coordinates of the charging base along the X-axis, Y-axis and Z-axis are x, y, z respectively, and the 6D pose obtained based on this embodiment is (x, y, z, ψ, θ, ).

[0140] For example, the solvePnP function of the OpenCV algorithm library can be used to automatically generate the initial values ​​of R and T to accelerate the subsequent optimization process of the reprojection error, without manually presetting the initial values ​​of R and T. In addition, in this embodiment, the 3D coordinates and 2D coordinates can be written as homogeneous coordinates to simplify the calculation.

[0141] An extrinsic parameter matrix is ​​obtained using the rotation matrix and the translation vector; an intrinsic parameter matrix of an image acquisition device used to acquire an image of the charging station is obtained; the product of the extrinsic parameter matrix, the intrinsic parameter matrix, and the first coordinate is obtained; and the second coordinate is obtained based on the product. The rotation matrix can represent the rotation angle of the first coordinate during coordinate transformation, and the translation vector can represent the translation amount of the first coordinate during coordinate transformation. The second coordinate obtained based on the rotation matrix and the translation vector can accurately reflect the projection result of the first coordinate in the image coordinate system.

[0142] For example, based on the first coordinates of the preset components in the 3D reference model and the internal parameters of the RGB camera used to capture the charging station image, the 3D first coordinate is converted to the image coordinate system, and the 3D first coordinate is mapped to the 2D pixel coordinate in the image coordinate system. The 2D pixel coordinate is the second coordinate. The third coordinate is obtained after correcting the actual initial coordinate of the charging station. The third coordinate and the 3D first coordinate are mapped to the 2D second coordinate in the image, and the loss function for the RGB camera rotation matrix and translation vector is obtained. The loss function is the reprojection error.

[0143] The reprojection error as the loss function can be cyclically optimized based on the least squares method to obtain the optimal coordinate transformation matrix of the RGB camera. The coordinate transformation matrix can include a rotation matrix and a translation vector. The position of the RGB camera relative to the charging base is obtained according to the rotation matrix and the translation vector, thereby obtaining the estimated position of the charging base with high accuracy.

[0144] With the rapid development of new energy vehicles, a variety of vehicle usage scenarios have emerged with smart travel as the core, forming an intelligent driving ecosystem with electric vehicles, hybrid vehicles and other vehicles that need to be charged as the carriers; with the continuous improvement of the intelligence of new energy vehicles, the demand for automatic vehicle charging is also increasing. However, the insufficient number of charging piles, cumbersome and time-consuming manual operations, and the lack of cost-effective charging robots have limited the efficiency and intelligence of automatic charging. In order to solve these problems, it is necessary to provide a method for accurately locating the charging seat and automatically charging according to the positioning results.

[0145] In practical applications, vehicles can be automatically charged using a charging robot. The charging robot includes components such as a robotic arm for controlling the insertion of the charging gun into the charging base, a drive motor, and a reducer. Software algorithms are used to sense the external environment, obtain the charging base's posture, and control the robotic arm based on the charging base's posture to complete the task of inserting the gun and charging. Among them, the charging base positioning and posture estimation algorithm is one of the key technologies in the charging robot software development process, which is directly related to the accuracy and success rate of the charging gun insertion during automatic charging by the charging robot. Therefore, accurately obtaining the charging base's posture is extremely important for automatic charging technology. However, the charging base posture acquisition methods of related technologies rely on expensive equipment such as radar equipment and binocular cameras, and require processing large amounts of data, resulting in complex methods and low processing efficiency.

[0146] The following is a further exemplary description of the method for acquiring the charging seat posture provided by this embodiment. Figure 5 , the method may include the following steps:

[0147] In step S1, a monocular camera is used to capture an image of the car charging station, which is input into a target detection model to obtain the labeled boxes of the areas where multiple preset components are located, as well as the pixel coordinates of the four corner points of each labeled box.

[0148] Specifically, the backbone network of the YOLOv5 model in the target detection model performs preprocessing and feature extraction, and performs inference classification and post-processing to obtain multiple sets of annotation boxes and the pixel coordinates of the four corner points of each annotation box. For example, refer to Figure 4 , there are 7 annotation boxes.

[0149] For example, this embodiment can acquire the image of the charging station using a monocular RGB camera. Based on the hardware platform composed of the monocular RGB camera, the charging station image is captured based on the monocular RGB camera. For example, a 640p resolution monocular camera can be used to capture the image of the charging station.

[0150] Step S2: Obtain the center coordinates of each marked box according to the pixel coordinates of the four corner points of the marked box, and determine the center coordinates as the third coordinates of the preset component 2D.

[0151] Step S3: Obtain the first 3D coordinate of the preset component in the reference model.

[0152] Among them, the 3D point coordinates are used to represent the initial posture of the charging base.

[0153] Step S4: Match the first 3D coordinate with the third 2D coordinate to obtain multiple coordinate pairs.

[0154] Step S5: Use the coordinate transformation matrix to convert the first coordinate into the image coordinate system to obtain the second coordinate, and cyclically adjust the coordinate transformation matrix. According to the third coordinate in the coordinate pair and the second coordinate corresponding to the first coordinate in the coordinate pair, the reprojection error of the second coordinate is obtained. The coordinate transformation matrix corresponding to the second coordinate whose reprojection error meets the preset error requirement is determined as the target coordinate transformation matrix.

[0155] Step S6: Calculate the posture of the charging base according to the target coordinate transformation matrix to obtain the 6D posture of the charging base.

[0156] Specifically, a standard 3D model of a real charging base is constructed, and the standard 3D model is used as a preset reference model. The fixed first coordinates of the 7 preset components are set as the target physical feature points of the charging base. The image of the charging base is obtained, and the third coordinates of the 7 preset components detected as feature points and the intrinsic parameter matrix of the camera are combined to perform PnP posture solution to obtain the 6-degree-of-freedom posture of the charging base. The 6-degree-of-freedom posture includes 3D coordinates relative to the camera coordinate system and rotation angles around the corresponding coordinate axes. Furthermore, the lightweight deep learning model YOLOv5 and the PnP algorithm can be used. In actual applications, they have low requirements for hardware platform resources and high reliability, and can meet the needs of edge computing platforms for real-time target posture detection.

[0157] This embodiment is suitable for parking and charging scenarios for new energy vehicles, such as family cars. It eliminates the need for point cloud data-based charging seat pose detection, eliminating the need for the charging robot to rely on hardware such as depth cameras and lidar when locating the charging seat. Furthermore, the charging seat pose can be determined based on captured images, providing an effective and reliable visual solution for cost-sensitive charging robot products, eliminating the reliance on hardware such as binocular cameras or expensive structured light cameras for pose detection. Furthermore, by using a deep learning model for vehicle charging seat detection, combined with a preset reference model, this approach addresses the high cost and complexity of hardware such as depth cameras, lidar, and binocular cameras required for charging seat pose detection in related technologies. This embodiment reduces hardware costs and increases efficiency, improving user satisfaction and addressing the need for high-precision visual positioning of the charging seat when the charging robot is plugging in and recharging new energy vehicles. Specifically, this embodiment accurately determines the charging seat pose through feature point matching and coordinate transformation matrix optimization algorithms. Compared to related methods that use binocular cameras to generate scene depth maps and then locate the charging seat based on these depth maps, this approach offers lower computational resource overhead, higher precision, and greater accuracy.

[0158] Reference Figure 6 The present application provides a charging seat posture acquisition device 30, comprising:

[0159] The first acquisition module 301 is used to acquire an image of the charging seat and a preset reference model of the charging seat; the preset reference model includes the first coordinate of the preset component in the preset coordinate system; the second acquisition module 302 is used to convert the first coordinate into the image coordinate system of the image through the coordinate transformation matrix to obtain the second coordinate; the third acquisition module 303 is used to adjust the coordinate transformation matrix and return to the operation of converting the first coordinate into the image coordinate system through the coordinate transformation matrix to obtain the second coordinate until the preset loop termination condition is met to obtain the target coordinate transformation matrix; wherein the second coordinate obtained through the target coordinate transformation matrix meets the preset coordinate transformation requirements; the fourth acquisition module 304 is used to obtain the posture of the charging seat according to the target coordinate transformation matrix.

[0160] For example, the device 30 also includes: a fifth acquisition module, used to obtain the third coordinate of the preset component in the image in the image coordinate system; a sixth acquisition module, used to obtain the reprojection error of the second coordinate based on the second coordinate and the third coordinate; wherein the preset coordinate conversion requirements include: the reprojection error meets the preset error requirements.

[0161] Optionally, the fifth acquisition module is further used to: obtain a first number of the preset component in the preset reference model and a second number of the preset component in the image; if the first number is equal to the second number, then obtain a third coordinate of the preset component in the image in the image coordinate system.

[0162] Optionally, when there are multiple preset components, the reprojection error of the second coordinate is the overall reprojection error of multiple second coordinates; the sixth acquisition module is also used to: obtain multiple coordinate pairs based on the first coordinate and the third coordinate; the coordinate pairs and the preset components correspond one to one; the coordinate pair includes the first coordinate and the third coordinate of the preset component corresponding to the coordinate pair; according to the third coordinate in the coordinate pair and the second coordinate corresponding to the first coordinate in the coordinate pair, obtain the reprojection error corresponding to the coordinate pair; sum the reprojection errors corresponding to the multiple coordinate pairs to obtain the overall reprojection error of the multiple second coordinates.

[0163] Optionally, the sixth acquisition module is also used to: obtain a first sorting result of the first coordinate; the first sorting result is the sorting result obtained by sorting the first coordinate according to a preset sorting strategy; sort the third coordinate according to the preset sorting strategy to obtain a second sorting result; obtain multiple coordinate pairs based on the first sorting result and the second sorting result.

[0164] Optionally, the first sorting result includes the first arrangement number of the first coordinate, and the second sorting result includes the second arrangement number of the third coordinate; the sixth acquisition module is also used to: obtain multiple coordinate pairs according to the first arrangement number and the second arrangement number; wherein the first arrangement number of the first coordinate in the coordinate pair is the same as the second arrangement number of the third coordinate in the coordinate pair.

[0165] Optionally, the coordinate transformation matrix includes a rotation matrix and a translation vector; the second acquisition module 302 is also used to: obtain an external parameter matrix based on the rotation matrix and the translation vector; determine the internal parameter matrix of the image acquisition device used to acquire the image; obtain the product between the external parameter matrix, the internal parameter matrix, and the first coordinate; and obtain the second coordinate based on the product.

[0166] In this embodiment, an image of the charging coordinates and a preset reference model of the charging base are acquired. The preset reference model includes the first coordinates of the preset components of the charging base in a preset coordinate system. The first coordinates are converted to the image coordinate system of the image using a coordinate transformation matrix to obtain the second coordinates. The coordinate transformation matrix is ​​then adjusted to obtain a coordinate transformation matrix corresponding to the loop operation. The operation of converting the first coordinates to the image coordinate system using the coordinate transformation matrix to obtain the second coordinates is then repeated until a preset loop termination condition is met. The target coordinate transformation matrix is ​​obtained. The second coordinates obtained using the target coordinate transformation matrix meet the preset coordinate transformation requirements. The charging seat posture is obtained based on the target coordinate transformation matrix. This embodiment does not require the acquisition and processing of radar data to obtain the charging seat posture. It also does not require the two images obtained by the binocular camera to calculate the 3D coordinates of the charging base based on the two images and then calculate the charging seat posture based on the 3D coordinates. Compared with the methods of the related art, this embodiment requires less data to be processed, has high processing efficiency, low hardware requirements, and a simple method.

[0167] Figure 7 4 is a block diagram of an electronic device 400 according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0168] Reference Figure 7 , electronic device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0169] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 402 may include one or more modules to facilitate interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate interaction between the multimedia component 408 and the processing component 402.

[0170] The memory 404 is used to store various types of data to support operations on the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, multimedia, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0171] The power supply assembly 404 provides power to the various components of the electronic device 400. The power supply assembly 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 400.

[0172] The multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of touch or slide actions, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a multimedia mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0173] The audio component 410 is used to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC) that is used to receive external audio signals when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0174] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0175] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the electronic device 400. For example, the sensor assembly 414 can detect the open / closed state of the electronic device 400, the relative positioning of components, such as the display and keypad of the electronic device 400. The sensor assembly 414 can also detect changes in the position of the electronic device 400 or a component of the electronic device 400, the presence or absence of user contact with the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and temperature changes of the electronic device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0176] The communication component 416 is used to facilitate wired or wireless communication between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0177] In an exemplary embodiment, the electronic device 400 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to implement a charging seat posture acquisition method provided in an embodiment of the present application.

[0178] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to perform the above method. For example, the non-transitory storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0179] Figure 81 is a block diagram of an electronic device 500 according to an exemplary embodiment. For example, the electronic device 500 may be provided as a server. Figure 8 The electronic device 500 includes a processing component 522, which further includes one or more processors, and a memory resource represented by a memory 532 for storing instructions executable by the processing component 522, such as an application. The application stored in the memory 532 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 522 is configured to execute instructions to perform a charging seat posture acquisition method provided in an embodiment of the present application.

[0180] The electronic device 500 may further include a power supply component 526 configured to perform power management of the electronic device 500, a wired or wireless network interface 550 configured to connect the electronic device 500 to a network, and an input / output (I / O) interface 558. The electronic device 500 may operate based on an operating system stored in the memory 532, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0181] An embodiment of the present application also provides a computer program product, including a computer program, and a method for obtaining a charging seat posture implemented when the computer program is executed by a processor.

[0182] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0183] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for acquiring a charging seat posture, characterized in that: include: Acquire an image of a charging base and a preset reference model of the charging base; The preset reference model includes a first coordinate of a preset component of the charging base in a preset coordinate system; Converting the first coordinates into the image coordinate system of the image using a coordinate transformation matrix to obtain second coordinates; Adjusting the coordinate transformation matrix and returning to convert the first coordinates into the image coordinate system through the coordinate transformation matrix to obtain the second coordinates, until a preset loop termination condition is satisfied, thereby obtaining a target coordinate transformation matrix; wherein the second coordinates obtained through the target coordinate transformation matrix meet the preset coordinate transformation requirements; The position and posture of the charging base are obtained according to the target coordinate transformation matrix.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining a third coordinate of a preset component in the image in the image coordinate system; Obtaining a reprojection error of the second coordinate according to the second coordinate and the third coordinate; The preset coordinate transformation requirement includes: the reprojection error meets the preset error requirement.

3. The method according to claim 2, characterized in that Obtaining a third coordinate of a preset component in the image in the image coordinate system includes: Obtaining a first number of the preset component in the preset reference model and a second number of the preset component in the image; If the first number is equal to the second number, the third coordinate of the preset component in the image in the image coordinate system is obtained.

4. The method according to claim 2, characterized in that In the case where there are multiple preset components, the reprojection error of the second coordinate is an overall reprojection error of the multiple second coordinates; and obtaining the reprojection error of the second coordinate according to the second coordinate and the third coordinate includes: According to the first coordinate and the third coordinate, a plurality of coordinate pairs are obtained; the coordinate pairs correspond to the preset components one by one; the coordinate pairs include the first coordinate and the third coordinate of the preset components corresponding to the coordinate pairs; Obtaining a reprojection error corresponding to the coordinate pair based on a third coordinate in the coordinate pair and the second coordinate corresponding to the first coordinate in the coordinate pair; The reprojection errors corresponding to the plurality of coordinate pairs are summed to obtain an overall reprojection error of the plurality of second coordinates.

5. The method according to claim 4, characterized in that The obtaining of a plurality of coordinate pairs according to the first coordinate and the third coordinate includes: Obtaining a first sorting result of the first coordinates; the first sorting result is a sorting result obtained by sorting the first coordinates according to a preset sorting strategy; Sorting the third coordinates according to the preset sorting strategy to obtain a second sorting result; A plurality of coordinate pairs are obtained according to the first sorting result and the second sorting result.

6. The method according to claim 5, characterized in that The first sorting result includes a first arrangement number of the first coordinate, and the second sorting result includes a second arrangement number of the third coordinate; and the plurality of coordinate pairs obtained according to the first sorting result and the second sorting result include: Obtaining a plurality of coordinate pairs according to the first arrangement sequence number and the second arrangement sequence number; The first arrangement sequence number of the first coordinate in the coordinate pair is the same as the second arrangement sequence number of the third coordinate in the coordinate pair.

7. The method according to claim 1, characterized in that The coordinate transformation matrix includes a rotation matrix and a translation vector; the first coordinate is converted into the image coordinate system by the coordinate transformation matrix to obtain the second coordinate, including: Obtaining an extrinsic parameter matrix according to the rotation matrix and the translation vector; determining an intrinsic parameter matrix of an image acquisition device used to acquire the image; Obtaining the product of the extrinsic parameter matrix, the intrinsic parameter matrix, and the first coordinate; The second coordinate is obtained according to the product.

8. A charging seat posture acquisition device, characterized in that: include: A first acquisition module is used to acquire an image of the charging base and a preset reference model of the charging base; The preset reference model includes a first coordinate of a preset component of the charging base in a preset coordinate system; A second acquisition module is configured to transform the first coordinates into an image coordinate system of the image using a coordinate transformation matrix to obtain second coordinates; a third acquisition module, configured to adjust the coordinate transformation matrix and return to the operation of converting the first coordinates into the image coordinate system through the coordinate transformation matrix to obtain the second coordinates, until a preset loop termination condition is satisfied, thereby obtaining a target coordinate transformation matrix; wherein the second coordinates obtained through the target coordinate transformation matrix meet the preset coordinate transformation requirements; The fourth acquisition module is used to obtain the position and posture of the charging base according to the target coordinate transformation matrix.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images