Electric vehicle intelligent charging device based on fast r-cnn target detection and control method thereof

Abstract

The application provides an electric vehicle intelligent charging device based on FastR-CNN target detection and a control method thereof, wherein the electric vehicle intelligent charging device based on FastR-CNN target detection comprises a device body, a first space is arranged in the device body, and a first opening in communication with the first space is arranged on the device body; a charging module, the charging module comprises a charging unit arranged in the first space and a charging plug electrically connected with the charging unit, the charging plug is connectable with a charging port of an electric vehicle; a moving module, the moving module is arranged in the first space, and the moving module comprises a moving arm arranged on an inner wall of the device body; a free end of the moving arm is connected with the charging plug; the moving arm can drive the charging plug to be exposed outside the first opening; a control module, the control module is used for driving the moving arm to drive the charging plug to be connected with the charging port in response to an external charging instruction; the charging plug is intelligently connected with the charging port, and the safety hazard is reduced.

Description

Technical Field

[0001] This application relates to the field of electric vehicle technology, specifically to an intelligent charging device for electric vehicles based on FastR-CNN object detection and its control method. Background Technology

[0002] With the increasing popularity of electric vehicles, charging stations have also emerged. Electric vehicles cannot function without charging stations, but existing manual charging stations have heavy and inconvenient power cords, and unstable plugging and unplugging can easily generate electric arcs, which may cause electric shock to users or damage to equipment, posing a safety hazard to users. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, this application aims to provide an intelligent charging device for electric vehicles based on Fast R-CNN object detection and its control method.

[0004] In a first aspect, this application proposes an intelligent charging device for electric vehicles based on Fast R-CNN object detection, comprising:

[0005] The device body has a first space inside and a first opening communicating with the first space on the device body.

[0006] A charging module, comprising a charging unit disposed in the first space and a charging plug electrically connected to the charging unit, wherein the charging plug can be connected to the charging port of the electric vehicle;

[0007] A mobile module is disposed within the first space. The mobile module includes: a mobile arm disposed on the inner wall of the device body; the free end of the mobile arm is connected to the charging plug; the mobile arm is capable of driving the charging plug to be exposed outside the first opening.

[0008] A control module is provided, which is used to respond to external charging commands and drive the mobile arm to connect the charging plug to the charging port.

[0009] According to the technical solution provided in the embodiments of this application, it further includes a measurement module, which is disposed on the outer wall of the charging plug and configured to obtain the first pressure applied to the charging plug by the inner wall of the charging port when the charging plug is mated with the charging port.

[0010] The control module also includes:

[0011] A first acquisition unit is electrically connected to the measurement module and is configured to acquire the first pressure.

[0012] A first processing unit is electrically connected to the first acquisition unit. The first processing unit is configured to output a first signal when the first pressure is greater than a first preset threshold.

[0013] A first output unit, the input terminal of which is electrically connected to the first arithmetic unit, and the output terminal of which is electrically connected to the moving arm, the first output unit being used to respond to the first signal to adjust the angle of the charging plug.

[0014] According to the technical solution provided in the embodiments of this application, it also includes a first detection module, which is disposed on the outer wall of the device body and is used to acquire first image information of the charging port.

[0015] The control module also includes:

[0016] The second acquisition unit is electrically connected to the first detection module and is configured to acquire the first image information.

[0017] The second processing unit is electrically connected to the second acquisition unit. The second processing unit is configured to obtain the first position information of the charging port using the first image information. The first position information includes the first center coordinates and depth of the charging port. The second processing unit is also configured to obtain the first joint rotation angle of the moving arm using the first position information.

[0018] The second output unit has its input terminal electrically connected to the second arithmetic unit and its output terminal electrically connected to the mobile arm. The second output unit is used to drive the mobile arm to connect the charging plug to the charging port at the first joint rotation angle.

[0019] According to the technical solution provided in the embodiments of this application, it further includes a second detection module, which is located at the end of the charging plug that is relatively far away from the device body. The second detection module is used to acquire second image information of the charging port.

[0020] The control module also includes:

[0021] The third acquisition unit is electrically connected to the second detection module and is configured to acquire the second image information.

[0022] The third processing unit is electrically connected to the third acquisition unit. The second processing unit is configured to acquire the second position information of the charging port using the second image information. The second position information includes the second center coordinates of the charging port. The second processing unit is also configured to acquire the second joint rotation angle of the moving arm using the second position information.

[0023] The third output unit has its input terminal electrically connected to the third arithmetic unit and its output terminal electrically connected to the moving arm. The third output unit is used to drive the moving arm to connect the charging plug to the charging port at the rotation angle of the second joint.

[0024] According to the technical solution provided in the embodiments of this application, the moving module further includes two first adjusting members distributed along a first direction and a second adjusting member bridging the two first adjusting members. The first adjusting members are used to drive the moving arm to move along a second direction, and the second adjusting member is used to drive the moving arm to move along the first direction; the second direction is perpendicular to the first direction.

[0025] Secondly, this application proposes a control method for an electric vehicle intelligent charging device based on Fast R-CNN object detection, comprising the following steps:

[0026] S101. Obtain a first image, which is obtained by the first detection module capturing the charging port.

[0027] S102. Input the first image into the region merging function to obtain a first filtering set. The first filtering set includes multiple first border information. The first border information includes the first center coordinates of the first border.

[0028] S103. Input the first image into the feature function to obtain the first feature matrix;

[0029] S104. Map the first center coordinates to the first feature matrix to obtain a second feature set, the second feature set including multiple second feature matrices;

[0030] S105. Construct the feature vectors of the third feature set to obtain the feature vector set of the third feature set;

[0031] S106. Input the set of feature vectors into the classification model to obtain a first classification sequence. The first classification sequence includes multiple classification results and probability values ​​corresponding to the classification results. Back-reasoning is used to obtain the feature vectors corresponding to the classification results, and then the first center coordinates corresponding to the feature vectors are obtained and set as the final center coordinates.

[0032] S107. Obtain the depth information of the first image, and combine it with the final center coordinates to obtain the first three-dimensional coordinates of the first image;

[0033] S108. Input the first three-dimensional coordinates into the angle conversion function to obtain the first joint angle set of the mobile arm;

[0034] S109. Drive the mobile arm to the charging plug and the charging port using the first joint angle set.

[0035] According to the technical solution provided in the embodiments of this application, when performing step S106, when the classification result corresponding to the multiple feature vectors is the first classification, the feature vector with the largest probability value is obtained, and the first center coordinates corresponding to the feature vector are deduced and set as the final center coordinates.

[0036] According to the technical solution provided in the embodiments of this application, steps S101-S108 are executed to obtain the first set of joint angles;

[0037] The first joint angle set drives the mobile arm to move the charging plug to a first position, where there is a distance between the first position and the charging port;

[0038] A second image is acquired by the second detection module by capturing the charging port.

[0039] Repeat steps S102-S108 to obtain the set of second joint angles of the mobile arm;

[0040] The mobile arm is driven by the second joint angle set to engage with the charging plug and the charging port.

[0041] The technical solution provided in the embodiments of this application further includes the following steps:

[0042] The first pressure between the charging plug and the charging port is obtained;

[0043] When it is determined that the first pressure is greater than the first preset threshold, the angle of the charging plug is adjusted until the first pressure is less than or equal to the first preset threshold.

[0044] According to the technical solution provided in the embodiments of this application, after the first image is labeled as the first category, the control method further includes the following steps:

[0045] By executing steps S101-S107, the first three-dimensional coordinates of the first image are obtained;

[0046] Acquire a third image within a defined area;

[0047] A preset rule shape is set, wherein the area of ​​the preset rule shape is larger than that of the third image;

[0048] The third image is mapped onto the preset rule pattern to obtain the first rule pattern;

[0049] Obtain the boundary coordinate range of the first regular graphic, wherein the boundary coordinate range includes the coordinate information of all pixels of the boundary of the first regular graphic;

[0050] Using the first three-dimensional coordinates as the target point, traverse all joints of the mobile arm to obtain N first sequences, where each first sequence includes path node coordinate information of multiple joints;

[0051] If it is determined that the coordinate information of at least one path node in the first sequence is not within the boundary coordinate range, then it is determined that the moving arm can avoid the obstacle, and the process continues to steps S108-S109.

[0052] In summary, this application proposes an intelligent charging device for electric vehicles based on Fast R-CNN object detection. The device comprises a charging module, a moving module, and a control module. The charging module has a charging plug that connects to the electric vehicle, which is mounted on the moving arm of the moving module. The control module includes mechanisms for responding to external charging commands and driving the charging plug to intelligently connect with the charging port. This application eliminates the need for manual plugging and unplugging of the charging plug. Upon receiving an external charging command, the charging plug and charging port automatically connect, and charging begins, preventing users from touching the charging plug and reducing safety hazards. Attached Figure Description

[0053] Figure 1 A schematic diagram of the structure of an intelligent charging device for electric vehicles (without cabinet door) provided in an embodiment of this application;

[0054] Figure 2 A schematic diagram of the structure of an intelligent charging device for electric vehicles (with cabinet door) provided in an embodiment of this application;

[0055] Figure 3 This is a schematic diagram of the structure of the moving mechanism provided in the embodiments of this application;

[0056] Figure 4 A flowchart of the control method for an intelligent charging device for electric vehicles provided in Embodiment 4 of this application;

[0057] Figure 5 A flowchart of the control method for an intelligent charging device for electric vehicles provided in Embodiment 5 of this application;

[0058] Figure 6 A flowchart of the control method for an intelligent charging device for electric vehicles provided in Embodiment 6 of this application;

[0059] The text labels in the image represent:

[0060] 1. Device body; 11. First space; 12. First opening; 13. Third mounting plate; 14. First mounting plate; 15. Second mounting plate; 21. Charger; 22. Charging plug; 23. Low-voltage component; 3. Moving arm; 31. Servo motor; 32. Base plate; 33. First adjusting component; 331. First slide rail; 332. First slider; 34. Second adjusting component; 341. Second slide rail; 342. Second slider; 41. First controller; 42. Second controller; 43. Third controller; 5. Measurement module; 61. First camera; 62. Infrared sensor; 71. Second camera; 8. Operation screen. Detailed Implementation

[0061] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0062] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0063] Example 1

[0064] As mentioned in the background section, to address the problems in the existing technology, this application proposes an intelligent charging device for electric vehicles based on Fast R-CNN object detection, such as... Figure 1 and Figure 2 As shown, it includes:

[0065] The device body 1 has a first space 11 inside and a first opening 12 communicating with the first space 11 on the device body 1. Preferably, the device body 1 is a cuboid electrical cabinet, the first space 11 is provided inside the electrical cabinet, and the first opening 12 is provided on the cabinet door of the electrical cabinet. The first space 11 is set as a top layer, a middle layer, and a bottom layer along the vertical direction, and a mounting plate is provided at the bottom of each layer.

[0066] The charging module further includes a charging unit disposed within the first space 11 and a charging plug 22 electrically connected to the charging unit. The charging plug 22 can be connected to the charging port of the electric vehicle. The charging unit includes a charger 21 disposed at the bottom layer of the device body 1 and a low-voltage component 23 disposed at the middle layer of the device body 1. The low-voltage component 23 includes circuit protection devices such as power switches, relays, contactors, and circuit breakers. A third mounting plate 13 is provided at the middle layer of the device body 1. The third mounting plate 13 is rectangular and fixedly installed on the inner wall of the device body 1. The low-voltage component 23 is fixed to the third mounting plate 13. A third control unit is integrated on the third mounting plate 13. The third controller 43 includes a third signal receiving unit configured to receive a signal indicating that the charging plug-in 22 and the charging port are paired, and transmit this signal to the low-voltage component 23. The low-voltage component 23 responds to the paired signal to start the charger 21 to charge the charging plug-in 22. Preferably, the model of the third controller 43 is FX3U-16M. The charger 21 is electrically connected to the charging plug-in 22 and can provide power to the charging plug-in 22. The charger 21 can convert the input 220V voltage into the charging voltage required by the electric vehicle, such as 32V. Preferably, the model of the charger 21 is BK-7000VA.

[0067] A mobile module, disposed within the first space 11, includes: a mobile arm 3 disposed on the inner wall of the device body 1; the free end of the mobile arm 3 is connected to the charging plug 22; the mobile arm 3 is capable of driving the charging plug 22 to be exposed outside the first opening 12; optionally, such as Figure 3As shown, the movable arm 3 includes multiple joints distributed along a direction perpendicular to the cabinet door of the device body 1. In some embodiments, it includes four joints, each joint including a servo motor 31 and a base plate 32. The joints are arranged sequentially from the end closest to the device body 1 as the first joint, second joint, third joint, and fourth joint. The first servo motor of the first joint has its end closest to the device body 1 hinged to the first base plate of the first joint, with the hinge axis being the first hinge axis. The other end of the first servo motor is fixed to the second base plate of the second joint on the side closest to the device body 1. The second servo motor of the second joint... One end is hinged to the second base plate, and its hinge axis is the second hinge axis. The end away from the second base plate is fixed to the third base plate of the third joint; the directions of two adjacent hinge axes are perpendicular; preferably, the model of the servo is DS3120; the first opening 12 is a rectangular through hole, and the moving arm 3 can extend from the first opening 12 and change the angle of each joint through the servo 31, so that the charging plug 22 can move within the range of the first opening 12; when not in operation, the charging plug 22 is retracted into the first space 11 to avoid dust contamination of the charging plug 22 socket;

[0068] The control module is used to respond to external charging commands and drive the moving arm 3 to connect the charging plug 22 to the charging port. The intelligent charging device for electric vehicles can realize the intelligent connection between the charging plug 22 and the charging port, avoiding user contact with the charging plug 22 and avoiding safety hazards.

[0069] Furthermore, such as Figure 3 As shown, it also includes a measurement module 5, which is disposed on the outer wall of the charging plug 22 and configured to obtain the first pressure applied to the charging plug 22 by the inner wall of the charging port when the charging plug 22 is mated with the charging port; wherein, the charging plug 22 is cylindrical, the measurement module 5 is a pressure sensor with a sheet-like structure, which can be embedded in the outer wall of the charging plug 22, and the model of the measurement module 5 is MD30-60;

[0070] The control module also includes:

[0071] The first acquisition unit is electrically connected to the measurement module 5 and is configured to acquire the first pressure; wherein, the moving arm 3 is provided with a second mounting plate 15 at the end relatively close to the device body 1, and the first acquisition unit is integrated on the second mounting plate 15.

[0072] A first processing unit is electrically connected to the first acquisition unit. The first processing unit is configured to output a first signal when the first pressure is greater than a first preset threshold. A second controller 42 is integrated on the second mounting plate 15. The second controller 42 includes the second processing unit. The first signal is a pressure non-compliance signal. When the first pressure is less than or equal to the first preset threshold, a second signal is output, which is a pressure compliance signal. The signal of the second controller 42 is: 1690-RaspberryPi4B / 4GB-ND.

[0073] The first output unit has its input terminal electrically connected to the first arithmetic unit and its output terminal electrically connected to the movable arm 3. The first output unit is used to respond to the first signal to adjust the angle of the charging plug 22. During the mating process between the charging plug 22 and the charging port, the first arithmetic unit transmits either the first signal or the second signal to the input terminal of the first output unit in real time. The first output unit is the drive unit of the movable arm 3 and is electrically connected to each of the servo motors 31 of the movable arm 3. When the first signal is received, the drive unit sends a signal to the servo motor 31 of the fourth joint. This servo motor 31 is connected to the charging plug 22, and its rotation angle is changed to avoid damage to the charging plug 22 during mating with the charging port. When the second signal is received, the first output unit does not operate, and the charging plug 22 continues to be mated with the charging port.

[0074] Furthermore, such as Figure 2 As shown, the intelligent charging device further includes a first detection module, which is disposed on the outer wall of the device body 1 and is used to acquire first image information of the charging port. The first detection module is located above the first opening 12 and includes a first camera 61 and an infrared sensor 62. The first camera 61 is model OV7725, and the infrared sensor 62 has a signal of GP2Y0A02. The first image information includes a photograph of the charging port taken by the first camera 61 and depth information of the charging port measured by the infrared sensor 62. The charging port includes multiple charging pins, and the charging plug 22 includes multiple charging holes that mate with the charging pins. The depth information includes a first depth and a second depth. The first depth is the vertical distance from the end of the charging pin furthest from the charging port to the infrared sensor 62, and the second depth is the vertical distance from the charging port excluding the charging pin to the infrared sensor 62.

[0075] The control module also includes:

[0076] The second acquisition unit is electrically connected to the first detection module and is configured to acquire the first image information; wherein, the second acquisition unit is integrated on the second mounting plate 15;

[0077] The second computing unit is electrically connected to the second acquisition unit. The second computing unit is configured to obtain the first position information of the charging port using the first image information. The first position information includes the first center coordinates and depth of the charging port. The second computing unit is also configured to obtain the first joint rotation angle of the mobile arm 3 using the first position information. The second controller 42 includes the second computing unit. The process of the second computing unit obtaining the first position information is an image processing process. The second computing module first determines whether the first image information is the charging port, then obtains the first center coordinates of the charging port through the Fast R-CNN algorithm, combines the depth information to obtain the three-dimensional coordinates of the charging port, and then converts them into the rotation angle of each joint of the mobile arm 3 through a formula, which is the first joint rotation angle.

[0078] The second output unit has its input terminal electrically connected to the second arithmetic unit and its output terminal electrically connected to the movable arm 3. The second output unit is used to drive the movable arm 3 to engage the charging plug 22 with the charging port at the first joint rotation angle. The second output unit is also the driving unit of the movable arm 3. When the second output unit receives the first joint rotation angle, it drives the movable arm 3 to move within the device body 1, exposing the charging plug 22 outside the first opening 12. Then, the second output unit drives the charging plug 22 to engage with the charging port according to the rotation angle of each joint.

[0079] Example 2

[0080] Based on Example 1, further, as Figure 1As shown, the intelligent charging device further includes a second detection module, which is located at the end of the charging plug 22 that is relatively far from the device body 1. The second detection module is used to acquire second image information of the charging port. The second detection module is located on the end face of the charging plug 22 that is relatively far from the device body 1. The second detection module includes a second camera 71, which is used to take a picture of the charging port. The model of the second camera 71 is OV7725. After the second processing unit acquires the first center coordinates and depth information of the charging port, the second output unit drives the moving arm 3 to move until the charging plug 22 reaches the first position, and then the second detection module takes another picture of the charging port. Optionally, the first position is a vertical distance of 10cm from the charging port.

[0081] The control module also includes:

[0082] The third acquisition unit is electrically connected to the second detection module and is configured to acquire the second image information; the third acquisition unit is integrated and installed on the second mounting plate 15, and the second image information is a photo of the charging port taken by the second detection module;

[0083] The third processing unit is electrically connected to the second acquisition unit. The second processing unit is configured to acquire the second position information of the charging port using the second image information. The second position information includes the second center coordinates of the charging port. The second processing unit is also configured to acquire the second joint rotation angle of the mobile arm 3 using the second position information. The second controller 42 includes the third processing unit. The third processing unit acquires the second center coordinates of the charging port using the Fast R-CNN algorithm, and then combines it with the depth information obtained by the first detection module to obtain the three-dimensional coordinates of the charging port. This allows the rotation angle of each joint of the mobile arm 3 to be obtained again, which is the second joint rotation angle.

[0084] The third output unit has its input terminal electrically connected to the third arithmetic unit and its output terminal electrically connected to the mobile arm 3. The third output unit is used to drive the mobile arm 3 to engage the charging plug 22 with the charging port according to the second joint rotation angle. The third output unit is the driving unit for the mobile arm 3. When the third output unit receives the second joint rotation angle, it drives each servo motor 31 of the mobile arm 3 to adjust the angle of each joint according to the second joint rotation angle, thereby driving the charging plug 22 to engage with the charging port. The second detection module is used to obtain the position of the charging port again, improving the accuracy of the movement of the charging plug 22.

[0085] Furthermore, such as Figure 1 and Figure 2 As shown, the intelligent charging device sends charging signals through a human-computer interaction component. The human-computer interaction includes an operation screen 8 located on the cabinet door of the device body 1. Optionally, the operation screen 8 is model SG104-BGCM. Users can use the operation screen 8 to pay, start, and stop charging. The top layer of the first space 11 is also provided with a first mounting plate 14. A fourth acquisition unit is integrated on the first mounting plate 14. The fourth acquisition unit is electrically connected to the operation screen 8 and is configured to receive charging commands. The first mounting plate 14 is also provided with a first controller 41. The first controller 41 includes a fourth arithmetic unit for converting analog charging command signals into digital charging command signals. The first mounting plate is also provided with a fourth output unit. The input terminal of the fourth output unit is electrically connected to the fourth arithmetic unit, and its output terminal is electrically connected to the second detection module. The fourth output unit is configured to send digital charging command signals to the second detection module.

[0086] Furthermore, the first mounting plate 14 is also equipped with a Bluetooth module, which allows users to communicate with the communication unit via Bluetooth through a mobile phone application, enabling the charging function on the mobile phone, or completing payment; optionally, the model of the Bluetooth module is: HC-08.

[0087] Furthermore, the human-computer interaction component also includes a voice prompt module, which issues a voice prompt when the charger 21 charges the charging port, or when there is an obstacle between the device body 1 and the charging port.

[0088] Example 3

[0089] Based on Example 2, further, such as Figure 3As shown, the moving module further includes two first adjusting members 33 distributed along a first direction and a second adjusting member 34 bridging the two first adjusting members 33. The first adjusting members 33 are used to drive the moving arm 3 to move along a second direction, and the second adjusting member 34 is used to drive the moving arm 3 to move along the first direction. The second direction is perpendicular to the first direction. Specifically, the first direction is perpendicular to the direction of the cabinet door of the device body 1, and the second direction is a vertical direction. The first adjusting member 33 and the second adjusting member 34 are electric lead screws. The first adjusting member 33 includes a first slide rail 331 extending parallel to the second direction. The second adjusting member 34 includes a second slide rail 341 extending along a first direction and a second slider 342 sliding along the second slide rail 341. The two ends of the second slide rail 341 are disposed on the two first sliders 332. The moving arm 3 is disposed at one end of the second slider 342 that is relatively far away from the device body 1. The charging plug 22 is disposed at the end of the moving arm 3 that is relatively far away from the second slider 342. Therefore, when the two first sliders 332 move simultaneously, the second slide rail 341 can be driven to move vertically, and the moving arm 3 can be moved in an orthogonal direction.

[0090] Example 4

[0091] This application proposes a control method for an intelligent charging device for electric vehicles based on Fast R-CNN object detection, as described above. Figure 4 As shown, it includes the following steps:

[0092] S101. Obtain a first image, which is obtained by the first detection module capturing the charging port; the first detection module includes the first camera 61, and the first image is obtained by the first camera 61.

[0093] S102. Input the first image into a region merging function to obtain a first filter set. The first filter set includes multiple first border information, and the first border information includes the first center coordinates of the first border. The merging of rows is performed using the Selective Search (SS) algorithm. Obtaining the first filter set using this algorithm includes the following steps:

[0094] The first image includes several pixels, and multiple first regions are generated by merging adjacent pixels based on their similarity. The merging function includes a formula for calculating the similarity between adjacent pixels, where the similarity between two adjacent pixels is the square root of the sum of the squares of the differences between the absolute gray values ​​or the three channel values ​​of the RGB image.

[0095] All pixels are merged repeatedly until the number of pixels in the first region is greater than the set minimum value, and the resulting region is the original segmented region.

[0096] Calculate the similarity between each pair of adjacent original segmented regions, find the pair with the highest similarity and merge them. The merged pair is a new region, known as the second region.

[0097] The second region continues to search for adjacent regions with the highest similarity and merge them, repeating the search until the expanded second regions can cover the entire first image;

[0098] Extract all bounding boxes of the second region as candidate bounding boxes, and remove candidate bounding boxes of the second region with less than 2000 pixels and an aspect ratio greater than 1.2. The remaining candidate bounding boxes are the first bounding boxes. The first image can be represented by a matrix, and the region selected by the candidate bounding box is also a matrix. The width of the candidate bounding box is the number of columns of the matrix, and the height of the candidate bounding box is the number of rows of the matrix. The size of the first candidate bounding box is the number of rows and columns of the first candidate bounding box.

[0099] S103. Input the first image into the feature function to obtain the first feature matrix; wherein, the feature function is a CNN (Convolutional Neural Networks) fully convolutional neural network, optionally, the CNN fully convolutional neural network model is a VGG (Visual Geometry Group) neural network model, the first feature matrix is ​​obtained by performing multiple convolutions on the first image, each convolution sets a filter matrix and a moving step size, starting from the upper left corner of the first image, and performing an inner product of each filter matrix and the first image in a certain order and with a set step size, each inner product yields a value, and the combination yields the first feature matrix, so the first feature matrix is ​​a matrix with multiple rows and columns; in some embodiments, 5 convolutions are performed;

[0100] S104. Map the first center coordinates to the first feature matrix to obtain a second feature set, the second feature set including multiple second feature matrices; wherein, the following steps are included:

[0101] The first center coordinates are mapped to the first feature matrix. First, the first center coordinates are converted into coordinates based on the coordinate system of the first feature matrix. Let the ratio of the size of the first border to the size of the first feature matrix be the first ratio. The second center coordinates are obtained by dividing the first center coordinates by the first ratio.

[0102] The second feature set is obtained by mapping the second center coordinates to the first feature matrix; the second feature set includes multiple second feature matrices; the second feature matrices have different sizes, that is, different numbers of rows and columns. The second border is obtained by mapping the second center coordinates onto the first feature matrix according to the size of the first border. The area selected by the second border is a multi-dimensional matrix, which is the second feature matrix corresponding to the first border; since the size of each first border is different, the size of the second feature matrix is ​​also different.

[0103] S105. Construct the feature vectors of the third feature set to obtain the feature vector set of the third feature set; wherein, firstly, the second feature set is scaled to obtain the third feature set, which includes multiple third feature matrices; the multiple third feature matrices have the same number of rows and columns; in some embodiments, the size of the second feature matrix is ​​set to A1*A2, the size of the third feature matrix is ​​set to B1*B2, K1 = A1 / B1, K2 = A2 / B2, and the third region is selected sequentially on the second feature matrix using K1*K2, and the matrix composed of the maximum value in each region is the third feature matrix; if K1 and K2 are not integers, the smaller integers are used; then, the third feature matrix is ​​fully connected to obtain the feature vector of each third feature matrix, and the feature vector is a matrix with 1 column;

[0104] S106. Input the set of feature vectors into the classification model to obtain a first classification sequence. The first classification sequence includes multiple classification results and probability values ​​corresponding to the classification results. Back-engineer the feature vectors corresponding to the classification results, and then obtain the first center coordinates corresponding to the feature vectors, which are set as the final center coordinates. In one embodiment, the classification model is a Softmax function. The classification model includes two classifications: the first classification is "is a charging port" and the second classification is "is not a charging port". The classification result corresponding to each feature vector includes the first classification and the first probability of the first classification, and the second classification and the second probability of the second classification. The sum of the first probability and the second probability is 100%.

[0105] When the first probability is greater than 50%, it is classified as the first category, meaning that the area selected by the first bounding box is the charging port, and its first center coordinate is the center coordinate of the charging port; when the first probability is less than 50%, it is classified as the second category, and the process returns to the first step to re-acquire the first image of the charging port; therefore, the first classification sequence includes at least one classification result with a first probability greater than 50%; when the first bounding box marked as the first category is obtained, there is a selection error because the value is rounded in step 106. In some embodiments, regression processing is performed on the first bounding box marked as the first category to improve the accuracy of image processing;

[0106] S107. Obtain the depth information of the first image, and combine it with the final center coordinates to obtain the first three-dimensional coordinates of the first image; the first detection module includes an infrared sensor 62, and the depth information is obtained through the infrared sensor 62.

[0107] S108. Input the first three-dimensional coordinates into the angle conversion function to obtain the first joint angle set of the mobile arm 3; after the first detection module obtains the first image and depth information, it inputs them into the second acquisition unit, the second acquisition unit transmits them to the second calculation unit, the second calculation unit executes the above steps to obtain the final center coordinates and depth of the charging port, and the second calculation unit converts them into the rotation angle of each joint through the angle conversion function to obtain the first joint angle set;

[0108] S109. Drive the mobile arm 3 to the charging plug 22 and the charging port according to the first joint angle set; the first output unit drives each servo motor 31 of the mobile arm 3 to move according to the rotation angle of each joint, thereby completing the connection between the charging plug 22 and the charging port.

[0109] Steps S102-S107 involve image processing of the first image using the Fast R-CNN neural network. Before image processing, the Fast R-CNN neural network needs to be trained: First, several photos of the charging port under different angles and lighting conditions are acquired as input, and the center coordinates of the different photos are taken as output. An optimized Fast R-CNN neural network model is obtained through inversion. Then, the first image is input into the Fast R-CNN neural network model to obtain the final center coordinates of the charging port.

[0110] Further, when performing step S106, when the classification result corresponding to multiple feature vectors is the first classification, the feature vector with the highest probability value is obtained, and the first center coordinate corresponding to the feature vector is deduced and set as the final center coordinate; wherein, when the classification result of multiple feature vectors is the first classification, that is, there are multiple labels "is charging port", the feature vector with the highest first probability is selected, and the first center corresponding to it is deduced as the final center coordinate.

[0111] Example 5

[0112] Based on Example 4, further, as Figure 5 As shown, the control method further includes the following steps:

[0113] S201, Execute steps S101-S108 to obtain the first joint angle set;

[0114] S202, Drive the mobile arm 3 to move to the charging plug 22 to a first position using the first joint angle set, wherein there is a distance between the first position and the charging port; in some embodiments, the distance between the first position and the charging port is 10cm;

[0115] S203. Acquire a second image, which is obtained by the second detection module capturing the charging port; or by the second camera 71 of the second detection module capturing the charging port.

[0116] S204. Repeat steps S102-S108 to obtain the second joint angle set of the mobile arm 3; Steps S102-S109 are the process of Fast R-CNN neural network performing image processing on the first image. This step also uses the same method to perform image processing on the second image to obtain the second joint angle set. The second joint angle set includes the rotation angle of each joint. In some embodiments, it is only necessary to rotate the joint corresponding to the charging plug 22.

[0117] S205. Drive the moving arm 3 to engage with the charging plug 22 using the second joint angle set; after the position of the charging port is determined again by the second image, drive the moving arm 3 to engage with the charging port, which improves the engagement accuracy and prevents the charging plug 22 from colliding with the charging port and causing damage.

[0118] Furthermore, during the mating process between the charging plug 22 and the charging port, in some embodiments, there may be an angle between the angle at which the charging plug 22 is inserted into the charging port and the center line of the charging port. If the angle is too large, the charging plug 22 cannot be inserted or may be damaged. Therefore, the control method further includes the following steps:

[0119] The first pressure between the charging plug-in 22 and the charging port is obtained; the first pressure is measured by the measurement module 5 and obtained by the first acquisition unit; the first acquisition unit transmits the first pressure to the first processing unit.

[0120] When it is determined that the first pressure is greater than the first preset threshold, the angle of the charging plug 22 is adjusted until the first pressure is less than or equal to the first preset threshold. This step is executed in the first calculation unit. When the pressure is greater than the first preset threshold, it means that the charging plug 22 is inserted obliquely into the charging port. Then the first calculation unit outputs a first signal, and the first output unit responds to the first signal to drive the joint corresponding to the charging plug 22 to rotate.

[0121] Example 6

[0122] Furthermore, after the first image is labeled as the first category, such as Figure 6 As shown, the control method further includes the following steps:

[0123] S301. Perform steps S101-S107 to obtain the first three-dimensional coordinates of the first image;

[0124] S302. Acquire a third image within a set area; the set area is the region between the infrared sensor 62 and the charging port. By using the depth information of the charging port, the depth of the charging port is excluded from the set area, thus avoiding the charging port being regarded as an obstacle.

[0125] S303. Set a preset regular shape, the area of ​​which is larger than that of the third image; optionally, the preset regular shape can be a cuboid or a cylinder, which can include the obstacle and expand the obstacle.

[0126] S304. Map the third image onto the preset regular graphic to obtain a first regular graphic; the first regular graphic includes the obstacle.

[0127] S305. Obtain the boundary coordinate range of the first regular graphic. The boundary coordinate range includes the coordinate information of all pixels on the boundary of the first regular graphic. The coordinate information is three-dimensional coordinate information. In some embodiments, a point near the first detection module is selected as the origin to establish an XYZ coordinate system. The XY coordinates are obtained through the first camera 61, and the Z coordinates are obtained through the infrared sensor 62.

[0128] S306. Using the first three-dimensional coordinates as the target point, traverse all joints of the mobile arm 3 to obtain N first sequences. The first sequence includes path node coordinate information of multiple joints. Specifically, since the joints can rotate through the hinge axis, different rotation angles correspond to different node coordinate information. In order for the charging plug 22 to reach the final center coordinates, traverse each joint to obtain N paths. Each joint has N node information. In some embodiments, four joints are included, and the possible number of nodes in the four joints are set to a, b, c, and d, respectively. Then the node coordinates of the four joints are:

[0129]

[0130] Among them, X 1i Y 1i Z 1i These represent the X-axis, Y-axis, and Z-axis coordinates of the first joint at different nodes, where i = 1, 2, 3...a;

[0131] X 2j Y2j Z 2j These represent the X, Y, and Z coordinates of the second joint at different nodes, where j = 1, 2, 3...b;

[0132] X 3k Y 3k Z 3k These represent the X, Y, and Z coordinates of the third joint at different nodes, where k = 1, 2, 3...c;

[0133] X 4m Y 4m Z 4m These represent the X-axis, Y-axis, and Z-axis coordinates of the fourth joint at different nodes, where m = 1, 2, 3...d;

[0134] By traversing the node coordinates of the four joints, N first sequences can be obtained, that is, each first sequence includes the node coordinates of the four joints that can reach the final coordinates;

[0135] S307. If it is determined that the coordinate information of at least one path node in the first sequence is not within the boundary coordinate range, then it is determined that the mobile arm 3 can avoid the obstacle, and proceed to steps S108-S109. Specifically, the coordinate information of each path node in the first sequence is combined to form the first path. Therefore, if the coordinate of any path node in the first sequence is within the boundary coordinate range, it means that the joint corresponding to it will reach the boundary coordinate range and there is a possibility of touching the obstacle. Therefore, only when all the coordinate information of the path nodes is not within the boundary coordinate range will the obstacle not be touched. If it is determined that the coordinate information of any path node in all the first sequences is within the boundary coordinate range, then it is determined that the mobile arm 3 cannot avoid the obstacle, and the human-computer interaction component sends a prompt. When the obstacle cannot be avoided, the voice prompt module issues an audio prompt, or sends the prompt to the remote client via the Bluetooth module.

[0136] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are only preferred embodiments of this application. It should be noted that due to the limitations of written expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. An electric vehicle intelligent charging device based on Fast R-CNN target detection, characterized in that, The device comprises: a device body (1) provided with a first space (11) therein, and a first opening (12) on the device body (1) and communicating with the first space (11); a charging module comprising a charging unit arranged in the first space (11) and a charging plug (22) electrically connected with the charging unit, the charging plug (22) being adapted to be connected with a charging port of the electric vehicle; a moving module arranged in the first space (11), the moving module comprising a moving arm (3) arranged on an inner wall of the device body (1), a free end of the moving arm (3) being connected with the charging plug (22), and the moving arm (3) being capable of driving the charging plug (22) to be exposed outside the first opening (12); a control module for driving the moving arm (3) to drive the charging plug (22) to be connected with the charging port in response to an external charging instruction; a first detection module arranged on an outer wall of the device body (1) and used for acquiring first image information of the charging port; The control method of the electric vehicle intelligent charging device based on the Fast R-CNN target detection comprises: S101, acquiring a first image, the first image being obtained by the first detection module capturing the charging port; S102, inputting the first image into a region merging function to obtain a first screening set, the first screening set comprising a plurality of first bounding box information, the first bounding box information comprising a first center coordinate of the first bounding box; S103, inputting the first image into a feature function to obtain a first feature matrix; S104, mapping the first center coordinate to the first feature matrix to obtain a second feature set, the second feature set comprising a plurality of second feature matrices; S105, scaling the second feature set to obtain a third feature set, constructing a feature vector of the third feature set to obtain a feature vector set of the third feature set; S106, inputting the feature vector set into a classification model to obtain a first classification sequence, the first classification sequence comprising a plurality of classification results and probability values corresponding to the classification results; the feature vector corresponding to the classification result is obtained by backstepping, and then the first center coordinate corresponding to the feature vector is obtained, which is defined as a final center coordinate; S107, acquiring depth information of the first image, and combining the final center coordinate to obtain a first three-dimensional coordinate of the first image; S108, inputting the first three-dimensional coordinate into an angle conversion function to obtain a first joint angle set of the moving arm (3); S109, driving the moving arm (3) to the charging plug (22) connected with the charging port by using the first joint angle set; When the first image is labeled as the first classification, the control method further comprises the following steps: performing steps S101-S107 to obtain the first three-dimensional coordinate of the first image; acquiring a third image in a set region; A preset rule pattern is set, an area of the preset rule pattern is greater than the third image; The third image is mapped into the preset rule pattern to obtain a first rule pattern; A boundary coordinate range of the first rule pattern is obtained, the boundary coordinate range including coordinate information of all pixel points of a boundary of the first rule pattern; With the first three-dimensional coordinate as a target point, all joints of the moving arm (3) are traversed to obtain N first sequences, the first sequence including path node coordinate information of a plurality of joints; When the path node coordinate information of at least one first sequence is not within the boundary coordinate range, it is determined that the moving arm (3) can avoid obstacles, and steps S108-S109 are continued. 2.The electric vehicle intelligent charging device based on Fast R-CNN object detection of claim 1, wherein: Further comprising a measurement module (5) arranged on an outer wall of the charging plug (22) and configured to obtain a first pressure applied by an inner wall of the charging port to the charging plug (22) when the charging plug (22) is connected to the charging port; The control module further comprises: A first acquisition unit electrically connected to the measurement module (5) and configured to obtain the first pressure; A first operation unit electrically connected to the first acquisition unit, the first operation unit configured to output a first signal when the first pressure is greater than a first preset threshold; A first output unit having an input end electrically connected to the first operation unit and an output end electrically connected to the moving arm (3), the first output unit configured to adjust an angle of the charging plug (22) in response to the first signal.

3. The electric vehicle intelligent charging device based on Fast R-CNN target detection according to claim 2, wherein: The control module further comprises: A second acquisition unit electrically connected to the first detection module and configured to obtain the first image information; A second operation unit electrically connected to the second acquisition unit, the second operation unit configured to obtain first position information of the charging port based on the first image information, the first position information including a first center coordinate and a depth of the charging port, and the second operation unit further configured to obtain a first joint rotation angle of the moving arm (3) based on the first position information; A second output unit having an input end electrically connected to the second operation unit and an output end electrically connected to the moving arm (3), the second output unit configured to drive the moving arm (3) to connect the charging plug (22) to the charging port based on the first joint rotation angle. 4.The electric vehicle intelligent charging device based on Fast R-CNN object detection of claim 3, characterized in that: Further comprising a second detection module arranged at an end of the charging plug (22) away from the device body (1), the second detection module configured to obtain second image information of the charging port; The control module further comprises: A third acquisition unit is electrically connected with the second detection module and configured to acquire the second image information; A third operation unit is electrically connected with the third acquisition unit, and configured to acquire second position information of the charging port from the second image information, the second position information including second center coordinates of the charging port; the second operation unit is further configured to acquire a second joint rotation angle of the mobile arm (3) from the second position information; A third output unit has an input end electrically connected with the third operation unit and an output end electrically connected with the mobile arm (3), and is configured to drive the mobile arm (3) to drive the charging plug (22) to mate with the charging port according to the second joint rotation angle. 5.The electric vehicle intelligent charging device based on Fast R-CNN object detection of claim 4, characterized in that: The mobile module further includes two first adjusting members (33) distributed along a first direction and a second adjusting member (34) bridging the two first adjusting members (33), the first adjusting members (33) being configured to drive the mobile arm (3) to move along a second direction, and the second adjusting member (34) being configured to drive the mobile arm (3) to move along the first direction; the second direction is perpendicular to the first direction. 6.The control method of the electric vehicle intelligent charging device based on Fast R-CNN object detection of claim 5, characterized in that, The method comprises the following steps: S101, acquiring a first image, the first image being obtained by the first detection module shooting the charging port; S102, inputting the first image into a region merging function to obtain a first screening set, the first screening set including a plurality of first frame information, the first frame information including first center coordinates of the first frame; S103, inputting the first image into a feature function to obtain a first feature matrix; S104, mapping the first center coordinates to the first feature matrix to obtain a second feature set, the second feature set including a plurality of second feature matrices; S105, constructing a feature vector of the third feature set to obtain a feature vector set of the third feature set; S106, inputting the feature vector set into a classification model to obtain a first classification sequence, the first classification sequence including a plurality of classification results and probability values corresponding to the classification results; The feature vector corresponding to the classification result is obtained by backstepping, and then the first center coordinates corresponding to the feature vector are obtained, which are set as final center coordinates; S107, acquiring depth information of the first image, and combining the final center coordinates to obtain first three-dimensional coordinates of the first image; S108, inputting the first three-dimensional coordinates into an angle conversion function to obtain the first joint angle set of the mobile arm (3); S109, driving the mobile arm (3) to mate the charging plug (22) with the charging port according to the first joint angle set; When the first image is labeled as the first classification, the control method further comprises the following steps: Steps S101-S107 are performed to obtain the first three-dimensional coordinates of the first image; A third image in a set region is acquired; A preset regular pattern is set, an area of the preset regular pattern is greater than the third image; The third image is mapped into the preset regular pattern to obtain a first regular pattern; A boundary coordinate range of the first regular pattern is obtained, the boundary coordinate range including coordinate information of all pixel points of a boundary of the first regular pattern; With the first three-dimensional coordinate as a target point, all joints of the mobile arm (3) are traversed to obtain N first sequences, the first sequence including path node coordinate information of a plurality of joints; When the path node coordinate information of at least one first sequence is not in the boundary coordinate range, it is determined that the mobile arm (3) can avoid obstacles, and steps S108-S109 are continued. 7.The control method of the electric vehicle intelligent charging device based on Fast R-CNN object detection of claim 6, characterized in that: When the classification results corresponding to a plurality of feature vectors are the first classification during the execution of step S106, the feature vector with the maximum probability value is obtained, and the first center coordinate corresponding to the feature vector is back calculated to obtain the final center coordinate. 8.The control method of the electric vehicle intelligent charging device based on Fast R-CNN object detection of claim 7, characterized in that, Further comprising the following steps: Steps S101-S108 are executed to obtain the first joint angle set; The mobile arm (3) is driven to the first position by the first joint angle set, and the first position and the charging port have a distance therebetween; A second image is obtained by the second detection module shooting the charging port; Steps S102-S108 are repeatedly executed to obtain a second joint angle set of the mobile arm (3); The mobile arm (3) is driven to the charging plug (22) and the charging port by the second joint angle set. 9.The control method of the electric vehicle intelligent charging device based on Fast R-CNN object detection of claim 8, characterized in that, Further comprising the following steps: The first pressure between the charging plug (22) and the charging port is obtained; When the first pressure is greater than a first preset threshold, the angle of the charging plug (22) is adjusted until the first pressure is less than or equal to the first preset threshold.

Images