Multi-perception fusion intelligent pile finding and aligning strategy, storage medium and electronic equipment

Abstract

The invention discloses a multi-perception fusion intelligent pile finding and aligning strategy, a storage medium and electronic equipment. The multi-perception fusion intelligent pile finding and aligning strategy comprises the following steps that a camera of a robot collects an image of a charging pile; detecting the image by using the charging pile key point detection model to obtain pixel coordinates of four corners of the charging pile in the image, and calculating pixel coordinates of a central point of the charging pile; obtaining a yaw angle and a pitch angle of the charging pile relative to the position of the camera and a distance between the charging pile and the robot, and calculating a transverse distance between the charging pile and the robot; controlling the robot to translate by a corresponding distance according to the transverse distance between the charging pile and the robot so as to enable the robot to reach the central point of the charging pile; the pose of the robot is adjusted, so that a charging port of the robot faces the charging pile; the robot straightly moves towards the charging pile to carry out pile alignment until a charging signal is received; according to the technical scheme, interference of objects similar to the charging pile in the environment can be effectively eliminated, the misrecognition condition is effectively avoided, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a multi-sensory fusion intelligent pile-finding and pile-matching strategy, a storage medium and an electronic device. Background Art

[0002] With the development of robot technology, robots are gradually replacing some manual work, and there are more and more scenarios for robot application. At present, robots have been widely used in restaurants, hotels, hospitals, government agencies and other scenarios to provide sanitation, delivery, guidance, and other services. Existing robots are usually equipped with rechargeable batteries and use autonomous charging to reduce human intervention. When the robot is in an idle state or the battery power value is less than or equal to a certain power threshold, the robot obtains the location information of the charging pile, and finds the pile to charge the pile according to the current position of the robot and the location of the charging pile. At present, the robot's pile-finding and pile-charging strategy mainly directly sends the target point based on the robot positioning and the location of the charging pile, and mainly relies on the robot's own positioning algorithm, so that the robot can automatically run directly from the current position to the location of the charging pile through the regulation and control algorithm to achieve the purpose of charging the pile. Because its pile-finding strategy needs to rely entirely on the robot's own positioning, it is difficult to meet the robot's positioning without a certain degree of deviation in the existing positioning algorithm.

[0003] See application publication number CN114296467 A for a method of automatically finding and aligning a charging pile with a robot. The robot obtains laser data of scene features of the surrounding environment through a laser radar; converts the collected laser data into point cloud data, and matches the point cloud data with the feature point cloud data of the outer contour of the charging pile to obtain the first relative posture of the charging pile; the robot runs to a preset distance position facing the charging pile; the robot obtains the size values ​​of each feature edge of the charging pile through a laser radar; obtains the second relative posture of the center point of the charging pile relative to the robot according to the size values ​​of each feature edge of the charging pile; plans the alignment path; the robot aligns the charging pile with the charging pile according to the alignment path. The above technical solution controls the posture of the robot through the point cloud features generated by the laser radar, so that the robot can go straight directly after reaching the center of the charging pile to achieve the purpose of charging the pile. Although the deviation caused by positioning is largely discarded from the algorithm level. However, due to the slightly higher complexity of the environment, the algorithm that relies solely on laser radar recognition may cause misidentification, so that the robot cannot accurately find the pile. At the same time, for the subsequent one-machine-multiple-pile search strategy, it is impossible to obtain the uniqueness of a charging pile. Summary of the invention

[0004] To this end, it is necessary to provide a multi-sensory fusion intelligent pile-finding and pile-aligning strategy to solve the problem that the robot cannot accurately find the piles due to misidentification.

[0005] To achieve the above object, the present invention provides a multi-sensory fusion intelligent pile-finding and pile-matching strategy, which includes the following steps:

[0006] The robot’s camera collects images of the charging pile;

[0007] The image is detected using the key point detection model of the charging pile to obtain the four corner pixel coordinates of the charging pile in the image, and the center pixel coordinates of the charging pile are calculated using the four corner pixel coordinates;

[0008] According to the pixel coordinates of the four corners of the charging pile, the pixel coordinates of the center point and the internal parameters of the camera, the yaw angle Yaw and pitch angle Pitch of the charging pile relative to the camera position and the distance between the charging pile and the robot are obtained, and the lateral distance between the charging pile and the robot is calculated by the triangular coordinate system law;

[0009] According to the lateral distance between the charging pile and the robot, control the robot to translate the corresponding distance so that the robot reaches the center point of the charging pile;

[0010] Adjust the robot's posture so that the robot's charging port faces the charging station;

[0011] The robot goes straight to the charging pile until it receives the charging signal.

[0012] Furthermore, in the step of detecting the image using the charging pile key point detection model, obtaining the four corner pixel coordinates of the charging pile, and calculating the center point pixel coordinates of the charging pile through the four corner pixel coordinates, the charging pile key point detection model is a pre-trained deep learning network model.

[0013] Furthermore, the training of the charging pile key point detection model includes the following steps:

[0014] Collect charging pile image samples;

[0015] Label the four corners of the charging pile image sample and use the result as the label of the training data;

[0016] A deep learning network model is established and the charging pile image samples are used as training data to train the deep learning network model. The loss function is defined and the learning network model parameters are updated through back propagation to obtain the charging pile key point detection model.

[0017] Furthermore, the deep learning network model is a YOLOv5 model or a model improved based on the YOLOv5 model.

[0018] Furthermore, the improved model based on the YOLOv5 model includes replacing the Focus layer of the network head with a 6*6 convolutional layer.

[0019] Furthermore, the improved model based on the YOLOv5 model includes replacing the backbone network with a ShufflenetV2 network.

[0020] Furthermore, the loss function is a combination of Wing Loss and the original loss function of YOLOv5.

[0021] Furthermore, the robot moves straight toward the charging pile until receiving the charging signal, which also includes identifying the coding information of the charging pile.

[0022] A storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-mentioned multi-sensory fusion intelligent pile-finding and pile-matching strategy.

[0023] An electronic device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the steps of the above-mentioned multi-sensory fusion intelligent pile-finding and pile-matching strategy are implemented.

[0024] Different from the existing technology, the above technical solution uses a pre-trained visual deep learning model to detect the feature points of the four corners of the charging pile. It can distinguish between the charging pile and objects with similar shapes, effectively eliminate the interference of objects in the environment with similar shapes to the charging pile, effectively avoid misidentification, and improve the accuracy of detection. It can accurately locate even in complex environments, reduce errors, and enhance robustness; then obtain the position of the charging pile relative to the camera based on the camera's internal parameters to control the robot to the front of the charging pile, and by adjusting the robot's posture, make the robot's charging port face the charging pile, and go straight to the charging pile until the charging signal is received, so that the robot can accurately find the pile and align it. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of the multi-sensory fusion intelligent pile-finding and pile-matching strategy of the present invention. DETAILED DESCRIPTION

[0026] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0027] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way to form a corresponding implementable technical solution.

[0028] Unless otherwise defined, the technical terms used in this document have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms in this document is only for describing specific embodiments and is not intended to limit this application.

[0029] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" logical relationship.

[0030] In the present application, terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0031] Without further limitations, in this application, the words "include", "comprises", "has" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0032] Similar to the understanding in the Patent Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0033] In the description of the embodiments of the present application, space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0034] Unless otherwise expressly specified or limited, in the description of the embodiments of the present application, the terms such as "install", "connect", "connect", "fix", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For technicians in the technical field to which the present application belongs, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0035] See also Figure 1 As shown, the present invention provides a multi-sensory fusion intelligent charging pile search and alignment strategy. The visual detection features are more accurate than the laser radar detection features. The pre-trained visual deep learning model is used to detect the feature points of the four corners of the charging pile, which can distinguish the charging pile from objects with similar shapes, and can effectively eliminate the interference of objects in the environment with shapes similar to the charging pile, effectively avoid misidentification, and improve the accuracy of detection. It can accurately locate even in complex environments, reduce errors, and enhance robustness. The position of the charging pile relative to the camera is obtained according to the camera internal parameters to control the robot to the front of the charging pile. By adjusting the robot's posture, the robot's charging port is directed towards the charging pile, and the robot goes straight to the charging pile for alignment until the charging signal is received, so that the robot can accurately find the charging pile and align the pile.

[0036] The following provides an implementation of the multi-sensory fusion intelligent pile-finding and pile-matching strategy of the present invention, which includes the following steps:

[0037] S1, the robot's camera collects images of the charging pile;

[0038] S2. Use the charging pile key point detection model to detect the image, obtain the four corner pixel coordinates of the charging pile in the image, and calculate the center pixel coordinates of the charging pile through the four corner pixel coordinates;

[0039] S3. According to the pixel coordinates of the four corners of the charging pile, the pixel coordinates of the center point and the internal parameters of the camera, the yaw angle Yaw and the pitch angle Pitch of the charging pile relative to the camera position and the distance between the charging pile and the robot are obtained, and the lateral distance between the charging pile and the robot is calculated by the triangular coordinate system rule;

[0040] S4. According to the lateral distance between the charging pile and the robot, control the robot to translate a corresponding distance so that the robot reaches the center point of the charging pile;

[0041] S5. Adjust the posture of the robot so that the charging port of the robot faces the charging pile;

[0042] S6. The robot moves straight toward the charging pile until it receives a charging signal from the charging pile.

[0043] The image of the charging pile captured by the camera of the above-mentioned robot is an image captured in real time by the robot for a certain posture. In order to improve the accuracy of detection of the four corners of the charging pile, the captured image can be preprocessed before the charging pile key point detection model is used to detect the image, such as grayscale, histogram equalization and other operations.

[0044] The above charging pile key point detection model is a pre-trained deep learning network model, which is used to detect the feature points of the four corners of the charging pile in the image, obtain the four corner pixel coordinates of the charging pile in the image, and then calculate the center pixel coordinates of the charging pile through the four corner pixel coordinates of the charging pile (which can be obtained by calculating the average value). Specifically training the charging pile key point detection model may include the following steps:

[0045] Collect charging pile image samples (including image data sets of feature points of the charging pile and its four corners);

[0046] Label the four corners of the charging pile image sample and use the result as the label of the training data;

[0047] A deep learning network model is established and the charging pile image samples are used as training data to train the deep learning network model. The loss function is defined and the learning network model parameters are updated through back propagation to obtain the charging pile key point detection model.

[0048] The above-mentioned deep learning network model can be based on a convolutional neural network model (CNN), a YOLOE series model or other deep learning models, or even a model improved based on the above-mentioned model, etc., and then trained by a large number of images, so that the model can obtain the four corner pixel coordinates of the charging pile in the image according to the image of the charging pile collected by the camera. To further illustrate the charging pile key point detection model of the present invention, the YOLOE series model is taken as an example to further illustrate the deep learning network model. The deep learning network model can be a YOLOv5 model, and the charging pile key point detection model is obtained by training the YOLOv5 model, so that it can output the four corner pixel coordinates of the charging pile in the image according to the image of the charging pile collected by the camera. In order to accurately obtain the outer contour rectangle of the characteristic part of the charging pile, accurately find the four corners on the charging pile in the image, so as to obtain the four corner pixel coordinates of the charging pile in the image, the deep learning network model can also be a model improved based on the YOLOv5 model, and it is trained to obtain the charging pile key point detection model.

[0049] In some embodiments, the improved model based on the YOLOv5 model includes replacing the network head Focus layer with a 6*6 convolutional layer. Replacing the network head Focus layer with a convolutional layer--The basic function of the Focus layer is to split the high-resolution image into multiple images with lower resolutions by performing alternate column sampling and splicing operations. However, the image data sets that need to be processed during the training process are all taken by the camera at a fixed lower resolution, so the Focus layer is not needed to play a role. For this reason, it is changed to a lighter 6*6 convolutional layer to optimize the speed of inference.

[0050] In some embodiments, the improved model based on the YOLOv5 model includes replacing the backbone network with the ShufflenetV2 network. It can speed up the YOLOv5 model in the reasoning process. The backbone network is the main backbone network in the YOLOv5 model, and this backbone network is mainly used to extract information such as image features. With the rapid development of deep learning target detection algorithms, reducing the complexity of the model and improving the detection speed while ensuring the accuracy of target detection have always been regarded as the goals pursued by target detection algorithms. Since the backbone network used in the YOLOv5 model has a good effect on the detection of small objects, but since the charging piles to be detected are roughly between 0.5 meters and 1 meter away from the camera, in order to make the trained model better transplanted to embedded devices, the backbone network is replaced with a more lightweight ShufflenetV2 network.

[0051] In some embodiments, the improved model based on the YOLOv5 model includes adding Wing Loss in the training, that is, the loss function is a combination of Wing Loss and the original loss function of YOLOv5. The accuracy of the key points can be optimized to the maximum, and the outlier problem of the key points can be solved to a certain extent. At the same time, in view of the fact that there are fewer training data sets and most of the data sets are facing the charging pile, only using the official loss training of YOLO will over-fit the front view posture, and will not be able to adapt to and detect the charging pile in the posture with a larger rotation angle. Therefore, the balance strategy of WingLoss can make up for the situation in which there are fewer samples in the training set when the charging pile angle is larger, and keep the training data balanced to a great extent.

[0052] According to the four corner pixel coordinates of the charging pile, the center pixel coordinates and the camera internal parameters, the yaw angle Yaw and the pitch angle Pitch of the charging pile relative to the camera position and the distance between the charging pile and the robot are obtained. That is, according to the camera internal parameters (such as focal length, pixel size, image center position, etc.), the center pixel coordinates and the four corner pixel coordinates of the charging pile are converted into three-dimensional coordinates in the camera coordinate system. In the camera coordinate system, the three-dimensional coordinates of the charging pile and the coordinate origin of the camera form a vector. By calculating the angle between this vector and the axes of the camera coordinate system, the yaw angle Yaw and the pitch angle Pitch of the charging pile relative to the camera and the distance between the charging pile and the robot can be obtained. The above can be achieved through the PNP (Perspective-n-Point) algorithm, such as EPnP (Efficient PNP), DLS (Direct Linear Transformation) or other PNP algorithms. Specifically, you can use the PNP solving function (such as cv2.solvePnP) in image processing libraries such as OpenCV, input the pixel coordinates of the four corners and the center point of the charging pile, and the camera internal parameters to solve the rotation matrix and translation vector of the camera. The rotation matrix contains the attitude information of the camera, including the yaw angle Yaw, the pitch angle Pitch, etc.; and the translation vector represents the three-dimensional displacement from the origin of the camera coordinate system to the center point of the charging pile. If the camera coordinate system coincides with the robot coordinate system, the modulus (i.e., length) of this displacement vector is the distance between the charging pile (the center point of the charging pile) and the robot (or camera). If there is a known fixed offset between the camera and the robot, this offset needs to be added to the calculated distance.

[0053] In the above calculation of the lateral distance between the charging pile and the robot by the triangular coordinate system rule, the lateral distance refers to the straight-line distance of the charging pile on the horizontal projection plane of the robot, that is, perpendicular to the line connecting the robot to the charging pile (on a two-dimensional plane). In the triangular coordinate system rule, according to the yaw angle Yaw, the pitch angle Pitch and the distance between the charging pile and the robot, the lateral distance between the charging pile (the center point of the charging pile) and the robot is usually calculated by trigonometric functions (such as sine, cosine, etc.) and the distance formula in plane geometry. When the robot is at a certain distance to the left of the charging pile, the robot rotates to the left and goes straight until the vehicle reaches the center of the charging pile; when the robot is at a certain distance to the right of the charging pile, the robot rotates to the right and goes straight until the vehicle reaches the center of the charging pile; when the robot is in front of the charging pile, the robot does not need to move. In the above step of controlling the robot to translate a corresponding distance based on the lateral distance between the charging pile and the robot so that the robot reaches the center point of the charging pile, the corresponding translation distance of the robot can be calculated by the odometer, thereby enabling the robot to reach the center point of the charging pile.

[0054] The step of adjusting the posture of the robot so that the charging port of the robot faces the charging pile may include the following steps:

[0055] Determine whether the pixel coordinates of the center point of the charging pile are in the exact center of the image;

[0056] If yes, the charging port of the robot is facing the charging pile; otherwise, control the robot to rotate in place and repeat the above steps.

[0057] After adjusting the posture, the robot can go straight to the charging pile to complete the charging process until it receives the charging signal from the charging pile. The charging signal from the charging pile can be received in a variety of ways. In some embodiments, the charging module of the robot receives the charging signal; in some embodiments, the infrared signal emitted by the charging pile when charging the robot is recognized by the robot. Specifically, the robot moves toward the charging pile until the infrared sensor of the charging pile detects that the robot is in place and correctly docked, and sends a charging signal. When the infrared signal of the charging pile is received to confirm that the robot has been charged, the robot stops moving.

[0058] In some embodiments, the robot goes straight to the charging pile to perform the charging process, and also includes determining whether charging is successful; if successful, reporting that the pile is found successfully; otherwise, reporting that the pile is found unsuccessfully and re-finding the pile. Specifically, when the robot is in the process of aligning with the charging pile, reporting that the pile is found unsuccessfully; the robot moves away from the charging pile by a preset distance, recalculates the lateral distance between the center of the charging pile and the robot, and controls the robot to translate a corresponding distance according to the lateral distance between the charging pile and the robot so that the robot reaches the center of the charging pile, and then re-aligns with the pile.

[0059] In some embodiments, the robot moves straight toward the charging pile until receiving a charging signal, which also includes identifying coded information of the charging pile.

[0060] The coding information of the charging pile is identifiable coding information to obtain the charging pile information (including the location of the charging pile, charging parameters, etc.), which can be a QR code, barcode, RFID tag, etc. In the pile matching process, the coding information of the charging pile is identified and reported to the robot-related module or the background system through the communication module. The robot can clearly know the number and location of the charging pile currently being charged, and can cooperate with repositioning to achieve the purpose of optimizing positioning.

[0061] The present invention also provides a storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned multi-sensory fusion intelligent pile-finding and pile-matching strategy are implemented.

[0062] The computer program involved in the embodiment can be stored in a computer device readable storage medium, which includes but is not limited to a disk, a tape, a magnetic card, a floppy disk, a flash memory, an optical disk, an optical card, a read-only memory (ROM), a random access memory (RAM), an erasable programmable ROM (EPROM) and an electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical or chemical structures that can achieve the same or equivalent functions as the storage media listed above, such as DNA, RNA, protein and other units with information storage capabilities. In a specific embodiment, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiment can be stored in a single medium in a centralized manner, or it can be stored in multiple media in a distributed manner. The memory containing the computer device readable storage medium can be a non-volatile memory or a random access memory. These computer device readable storage media can be built into the device, or they can be connected to the device involved in the embodiment as an external device or a part of an external device. In some embodiments, a memory with a computer device readable storage medium is deployed locally; in other embodiments, a solution of deploying the memory away from the processor may also be adopted, such as a network attached memory accessed via an RF circuit or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), etc., or a suitable combination thereof, as long as the computer device can access the memory. In addition, the computer program involved in the embodiment may be stored in plaintext / ciphertext form, or may be designed as training data, which may be integrated and reorganized through model training and implicitly stored in the parameter state of a deep neural network or other machine learning model.

[0063] The present invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the steps of the above-mentioned multi-sensory fusion intelligent pile-finding and pile-matching strategy are implemented.

[0064] The processor described in the embodiment can be implemented by hardware, firmware, software or a combination thereof, and can use circuits, single or multiple application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), digital signal processing devices (Digital Signal Processing Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field Programmable Gate Array, FPGA), central processing units (Central Processing Unit, CPU), controllers, microcontrollers, at least one of microprocessors, and also include other physical, biological or chemical structures that can achieve similar or equivalent functions to the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer programs or methods involved in the various embodiments of the present application, or any combination of the steps mentioned therein.

[0065] It should be noted that, although the above embodiments have been described in this article, the patent protection scope of the present invention is not limited thereby. Therefore, based on the innovative concept of the present invention, changes and modifications made to the embodiments described herein, or equivalent structures or equivalent process changes made using the contents of the present invention specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included in the patent protection scope of the present invention.

Claims

1. Multi-sensory fusion intelligent pile-finding and pile-matching strategy, characterized by: The following steps are involved: The robot’s camera collects images of the charging pile; The image is detected using the key point detection model of the charging pile to obtain the four corner pixel coordinates of the charging pile in the image, and the center pixel coordinates of the charging pile are calculated using the four corner pixel coordinates; According to the pixel coordinates of the four corners of the charging pile, the pixel coordinates of the center point and the internal parameters of the camera, the yaw angle Yaw and pitch angle Pitch of the charging pile relative to the camera position and the distance between the charging pile and the robot are obtained, and the lateral distance between the charging pile and the robot is calculated by the triangular coordinate system law; According to the lateral distance between the charging pile and the robot, control the robot to translate the corresponding distance so that the robot reaches the center point of the charging pile; Adjust the robot's posture so that the robot's charging port faces the charging station; The robot goes straight to the charging pile until it receives the charging signal.

2. The multi-sensory fusion intelligent pile-finding and pile-matching strategy according to claim 1 is characterized in that: In the step of detecting the image using the charging pile key point detection model, obtaining the four corner pixel coordinates of the charging pile, and calculating the center point pixel coordinates of the charging pile through the four corner pixel coordinates, the charging pile key point detection model is a pre-trained deep learning network model.

3. The multi-sensory fusion intelligent pile-finding and pile-matching strategy according to claim 2 is characterized in that: The training of the charging pile key point detection model includes the following steps: Collect charging pile image samples; Label the four corners of the charging pile image sample and use the result as the label of the training data; A deep learning network model is established and the charging pile image samples are used as training data to train the deep learning network model. The loss function is defined and the learning network model parameters are updated through back propagation to obtain the charging pile key point detection model.

4. The multi-sensory fusion intelligent pile-finding and pile-matching strategy according to claim 3 is characterized in that: The deep learning network model is a YOLOv5 model or a model improved based on the YOLOv5 model.

5. The multi-sensory fusion intelligent pile-finding and pile-matching strategy according to claim 4 is characterized in that: The improved model based on the YOLOv5 model includes replacing the Focus layer of the network head with a 6*6 convolutional layer.

6. The multi-sensory fusion intelligent pile-finding and pile-matching strategy according to claim 4 is characterized in that: The improved model based on the YOLOv5 model includes replacing the backbone network with the ShufflenetV2 network.

7. The multi-sensory fusion intelligent pile-finding and pile-matching strategy according to claim 4 is characterized in that: The loss function is a combination of Wing Loss and the original loss function of YOLOv5.

8. The multi-sensory fusion intelligent pile-finding and pile-matching strategy according to claim 1 is characterized in that: The robot moves straight toward the charging pile until receiving a charging signal, which also includes identifying the coding information of the charging pile.

9. A storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the multi-sensory fusion intelligent pile-finding and pile-matching strategy as described in any one of claims 1 to 8 are implemented.

10. An electronic device, characterized in that It includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the multi-sensory fusion intelligent pile-finding and pile-matching strategy as described in any one of claims 1 to 8 are implemented.

Images