Charging control method and device, equipment, storage medium and product

Abstract

The invention relates to the technical field of new energy, in particular to a charging control method and device, equipment, a storage medium and a product, and the method comprises the steps: obtaining multi-mode charging port data through a double-sided vision camera, a laser radar and a force sensor; extracting a multi-modal data feature corresponding to the multi-modal charging port data; and controlling the mobile charging gun to perform staged displacement based on at least one of the multi-modal data characteristics so as to enable the mobile charging gun and the charging port to complete plugging operation, and controlling the mobile charging pile to move in the charging area so as to charge the new energy vehicle when the new energy vehicle needs to supplement energy. And the charging gun is automatically controlled to perform staged displacement, so that the mobile charging gun can be plugged into and pulled out of the charging port, and the energy supplementing waiting time of the new energy vehicle is shortened.

Description

Technical Field

[0001] The present invention relates to the field of new energy technologies, and in particular to charging control methods, devices, equipment, storage media and products. Background Art

[0002] With the development of new energy vehicles, more and more charging stations are being built to support them. However, for the sake of convenience, some cars choose to park in vacant charging spaces, occupying charging quotas. As a result, new energy vehicles have to wait for a long time when they need to recharge. In addition, traditional charging methods rely on manual operation and are less safe.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a charging control method, device, equipment, storage medium and product, aiming to solve the technical problem in the prior art that new energy vehicles need to wait for a long time when recharging and rely on manual operation.

[0005] To achieve the above objectives, the present invention provides a charging control method, which is applied to a charging pile, wherein the charging pile includes at least one mobile charging gun and multiple charging areas. The method includes the following steps:

[0006] Acquiring multimodal charging port data through the double-sided vision camera, the laser radar, and the force sensor;

[0007] Extracting multimodal data features corresponding to the multimodal charging port data;

[0008] Based on at least one of the multimodal data features, the mobile charging gun is controlled to perform a staged displacement so that the mobile charging gun and the charging port complete the plugging and unplugging operation.

[0009] Optionally, the multimodal charging port data includes at least: an RGB image, point cloud data, and force data of the charging port, and the multimodal data features include at least RGB image features, point cloud data features, and force data features;

[0010] The controlling the mobile charging gun to perform staged displacement based on the multimodal data feature so as to complete the plugging and unplugging operation between the mobile charging gun and the charging port includes:

[0011] The RGB image features and point cloud data features are fused through a multimodal feature fusion model to obtain 6D pose estimation data;

[0012] Positioning the charging port of the device to be charged and the mobile charging gun based on the 6D pose estimation data to obtain a relative position between the charging port and the mobile charging gun;

[0013] The mobile charging gun is controlled to perform staged displacement according to at least one of the RGB image features, the point cloud data features, and the force data features, and the relative position, so that the mobile charging gun and the charging port complete the plugging and unplugging operation.

[0014] Optionally, controlling the mobile charging gun to perform staged displacement according to at least one of the RGB image feature, the point cloud data feature, and the force data feature and the relative position includes:

[0015] When the relative position satisfies a first-stage position condition, inputting the RGB image features into a trained visual servoing model to obtain the spatial position of the mobile charging gun, wherein the first-stage position condition includes that the relative distance between the charging port and the mobile charging gun is greater than a preset distance;

[0016] Performing path planning based on the spatial position to obtain a first-stage moving path of the mobile charging gun;

[0017] Based on the first-stage moving path, the mobile charging gun is controlled to perform a first-stage displacement so that the mobile charging gun moves to a preset position, the relative distance between the preset position and the charging port is less than a preset distance, and the preset position is horizontal with the spatial position of the charging port.

[0018] Optionally, the charging control method further includes:

[0019] When the relative position satisfies the second-stage position condition, obtaining the charging port three-dimensional model and target point cloud data of the charging port, wherein the second-stage position condition includes that the spatial position between the charging port and the mobile charging gun is horizontal and the relative distance is less than or equal to a preset distance;

[0020] Matching the target point cloud data and the point cloud data features based on the charging port three-dimensional model;

[0021] The mobile charging gun is controlled to perform a second-stage displacement according to the matching result, so that the mobile charging gun moves to the charging port for alignment.

[0022] Optionally, the charging control method further includes:

[0023] When the relative position satisfies the position condition of the third stage, obtaining contact pressure data fed back by the force sensor in real time;

[0024] The position and movement speed of the mobile charging gun are adjusted according to the contact pressure data so that the mobile charging gun is completely inserted into the charging port.

[0025] Optionally, the step of fusing RGB image features and point cloud data features using a multimodal feature fusion model to obtain 6D pose estimation data includes:

[0026] Performing time alignment on the RGB image features and the point cloud data features to obtain target RGB image features and target point cloud data features;

[0027] Extracting key point cloud data features corresponding to the two-dimensional image features and the target point cloud data features in the target RGB image features;

[0028] Performing spatial coordinate alignment according to the two-dimensional image features and key point cloud data features;

[0029] The 6D pose estimation data of the mobile charging gun is calculated based on the two-dimensional image features and key point cloud data features after spatial coordinate alignment.

[0030] In addition, to achieve the above-mentioned object, the present invention further provides a charging control device, the charging control device comprising:

[0031] an acquisition module, configured to acquire multimodal charging port data through the double-sided vision camera, the laser radar, and the force sensor;

[0032] an extraction module, configured to extract multimodal data features corresponding to the multimodal charging port data;

[0033] A control module is used to control the mobile charging gun to perform staged displacement based on at least one of the multimodal data features, so that the mobile charging gun and the charging port complete the plug-in and unplug operation.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also proposes a charging control device, which includes: a memory, a processor, and a charging control program stored in the memory and executable on the processor, wherein the charging control program is configured to implement the steps of the charging control method described above.

[0035] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which a charging control program is stored. When the charging control program is executed by a processor, the steps of the charging control method described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the charging control method described above are implemented.

[0037] The present invention discloses a charging control method, which is applied to a charging pile, wherein the charging pile includes at least one mobile charging gun and multiple charging areas, and the mobile charging gun is equipped with a double-sided vision camera, a laser radar and a force sensor; the charging control method includes: obtaining multimodal charging port data through the double-sided vision camera, the laser radar and the force sensor; extracting multimodal data features corresponding to the multimodal charging port data; controlling the mobile charging gun to perform staged displacement based on at least one of the multimodal data features, so that the mobile charging gun and the charging port complete the plug-in and unplug operation, and by controlling the mobile charging pile to move within the charging area, when the new energy vehicle needs to be recharged, the charging gun is automatically controlled to perform staged displacement to realize the plug-in and unplugging of the mobile charging gun and the charging port, thereby reducing the waiting time for recharging of the new energy vehicle and avoiding the technical problem in the prior art that new energy vehicles need to wait for a long time when recharging and rely on manual operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0040] Figure 1 This is a flow chart of a first embodiment of a charging control method according to the present invention;

[0041] Figure 2 This is a flow chart of a second embodiment of a charging control method according to the present invention;

[0042] Figure 3 This is a schematic diagram of the connection between a charging gun and a charging port according to an embodiment of a charging control method of the present invention;

[0043] Figure 4 This is a structural block diagram of a first embodiment of a charging control device according to the present invention;

[0044] Figure 5 It is a structural diagram of a charging control device in a hardware operating environment involved in an embodiment of the present invention.

[0045] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0047] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0048] Based on this, the embodiment of the present invention provides a charging control method, referring to Figure 1 , Figure 1 FIG. 1 is a flow chart of a first embodiment of a charging control method according to the present invention.

[0049] In this embodiment, the charging control method includes:

[0050] Step S10: Acquire multimodal charging port data through the double-sided vision camera, the laser radar, and the force sensor.

[0051] Step S20: extracting multimodal data features corresponding to the multimodal charging port data.

[0052] Step S30: Based on at least one of the multimodal data features, the mobile charging gun is controlled to perform a phased displacement, so that the mobile charging gun and the charging port complete the plugging and unplugging operation.

[0053] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of implementing the above functions, a control computer, or a cloud server. The following uses the control computer as an example to illustrate this embodiment and the following embodiments.

[0054] It should be noted that the current traditional charging piles are all one-pile-one-gun mode, that is, one charging pile is equipped with one charging gun. However, when a vehicle that does not need to be recharged is parked in the parking space where the charging pile is located, other vehicles waiting for recharge will need to change to another charging pile or wait for a long time. Therefore, this embodiment proposes that the charging pile is equipped with multiple mobile charging guns. When a parking space is occupied, the remaining charging guns can be moved to an empty parking space for charging. During the charging process, the charging gun and the vehicle charging port are generally plugged in and out manually. Due to the characteristics of high-power charging, the interface between the charging port and the charging gun is relatively tight. Manual operation may result in incorrect plugging and unplugging directions, causing damage to the charging port and affecting user use.

[0055] Based on this, this embodiment proposes a solution to control the movement of the mobile charging pile within the charging area, so that when the new energy vehicle needs to be recharged, the charging gun is automatically controlled to move in stages to realize the plugging and unplugging of the mobile charging gun and the charging port, thereby reducing the waiting time for recharging of the new energy vehicle. This not only shortens the waiting time for charging, but also eliminates the need for manual plugging and unplugging, thereby improving safety.

[0056] It can be understood that the multimodal charging port data in this embodiment refers to at least one of the RGB image, point cloud data and force data of the charging port. Accordingly, the RGB image, point cloud data and force data of the charging port can be collected by a double-sided vision camera, lidar and force sensor installed on the charging robotic arm.

[0057] Multimodal data features can be RGB image features, point cloud data features, and force data features. The specific feature extraction method varies depending on the data category. For example, for RGB image data, an improved ResNet-18 model can be used to extract two-dimensional features such as the edge, color, and QR code of the charging port in the image. For point cloud data, the ICP algorithm model can be used to identify the key point cloud of the charging port, thereby obtaining the key features of the point cloud.

[0058] In a specific implementation, the docking and plugging process of the charging gun and the charging port is divided into three stages. For the sake of convenience, this embodiment takes charging as an example. The control logic of the three stages in the charging process is different. For example: the constraint condition of the first stage is to control the mobile charging gun to move to a relative distance from the charging port that is less than a preset distance and is horizontal with the spatial position of the charging port; the constraint condition of the second stage is to move the mobile charging gun to the charging port and align it; the constraint condition of the third stage is that the mobile charging gun is completely inserted into the charging port.

[0059] This embodiment obtains multimodal charging port data through the double-sided vision camera, the laser radar and the force sensor; extracts multimodal data features corresponding to the multimodal charging port data; and controls the mobile charging gun to perform staged displacement based on at least one of the multimodal data features, so that the mobile charging gun and the charging port can be plugged and unplugged. By controlling the mobile charging pile to move within the charging area, when the new energy vehicle needs to be recharged, the charging gun is automatically controlled to perform staged displacement to achieve plugging and unplugging of the mobile charging gun and the charging port, thereby reducing the waiting time for recharging the new energy vehicle.

[0060] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 Before step S10, the charging control method further includes steps S01 to S02:

[0061] Step S301: RGB image features and point cloud data features are fused using a multimodal feature fusion model to obtain 6D pose estimation data.

[0062] Step S302: Positioning the charging port of the device to be charged and the mobile charging gun based on the 6D pose estimation data to obtain a relative position between the charging port and the mobile charging gun.

[0063] Step S303: controlling the mobile charging gun to perform phased displacement according to at least one of the RGB image feature, the point cloud data feature, and the force data feature and the relative position, so as to complete the plugging and unplugging operation between the mobile charging gun and the charging port.

[0064] It should be noted that the data fusion of RGB image features and point cloud data features can be performed by a multimodal feature fusion network (MMF-Net) based on deep learning to fuse the RGB image features and point cloud data features to output 6D estimation data of the charging port.

[0065] Furthermore, the RGB image features and point cloud data features are fused by the multimodal feature fusion model to obtain 6D pose estimation data, including:

[0066] Performing time alignment on the RGB image features and the point cloud data features to obtain target RGB image features and target point cloud data features;

[0067] Extracting key point cloud data features corresponding to the two-dimensional image features and the target point cloud data features in the target RGB image features;

[0068] Performing spatial coordinate alignment according to the two-dimensional image features and key point cloud data features;

[0069] The 6D pose estimation data of the mobile charging gun is calculated based on the two-dimensional image features and key point cloud data features after spatial coordinate alignment.

[0070] In a specific implementation, the purpose of time-aligning the RGB image features and the point cloud data features is to reduce image loss caused by the movement of the charging gun and improve the reliability of the features.

[0071] In addition, since the point cloud data features are 3D features, a 3D model of the charging port can be first established based on the point cloud data features to improve the efficiency of pose estimation and directly optimize the rotation and translation parameters. By extracting 3D key points from the point cloud features, such as the edges of the charging port and the charging column, and then combining them with the key points in the RGB image features, feature space alignment can be achieved, thereby improving the accuracy of pose estimation.

[0072] Specifically, since RGB image features are 2D features, during the image acquisition process, the image may need to be rotated or translated to align with the 3D key point cloud data. Therefore, the rotation matrix and translation vector of the RGB image features can be established, and the rotation matrix and translation vector of the RGB image features are combined with the matching 2D image features and key point cloud data features for data fusion. The formula for this process is:

[0073]

[0074] Among them, E is the rotation matrix, t is the translation vector, pi and qi are the matched two-dimensional image features and key point cloud data features.

[0075] The target rotation matrix and translation vector in the data fusion process are calculated by minimizing the constraint function, thereby obtaining the data fusion result with the highest confidence.

[0076] Furthermore, controlling the mobile charging gun to perform staged displacement according to at least one of the RGB image feature, the point cloud data feature, and the force data feature and the relative position includes:

[0077] When the relative position satisfies a first-stage position condition, inputting the RGB image features into a trained visual servoing model to obtain the spatial position of the mobile charging gun, wherein the first-stage position condition includes that the relative distance between the charging port and the mobile charging gun is greater than a preset distance;

[0078] Performing path planning based on the spatial position to obtain a first-stage moving path of the mobile charging gun;

[0079] Based on the first-stage moving path, the mobile charging gun is controlled to perform a first-stage displacement so that the mobile charging gun moves to a preset position, the relative distance between the preset position and the charging port is less than a preset distance, and the preset position is horizontal with the spatial position of the charging port.

[0080] Understandably, the reference Figure 3 The first stage refers to moving the charging gun to an area 50 cm horizontally with the vehicle's charging port. During this process, the shortest path planning algorithm can be used to determine the first-stage moving path of the mobile charging gun, and the shortest moving path can be used to achieve rough alignment between the charging gun and the charging port, thereby improving charging efficiency.

[0081] Furthermore, the charging control method further includes:

[0082] When the relative position satisfies the second-stage position condition, obtaining the charging port three-dimensional model and target point cloud data of the charging port, wherein the second-stage position condition includes that the spatial position between the charging port and the mobile charging gun is horizontal and the relative distance is less than or equal to a preset distance;

[0083] Matching the target point cloud data and the point cloud data features based on the charging port three-dimensional model;

[0084] The mobile charging gun is controlled to perform a second-stage displacement according to the matching result, so that the mobile charging gun moves to the charging port for alignment.

[0085] In the specific implementation, after the first stage, since the radar data is more sensitive, in this embodiment, the key features in the point cloud data features can be matched in combination with the pre-built three-dimensional model of the charging port, thereby achieving precise alignment of the charging gun and the charging port.

[0086] Furthermore, the charging control method further includes:

[0087] When the relative position satisfies the position condition of the third stage, obtaining contact pressure data fed back by the force sensor in real time;

[0088] The position and movement speed of the mobile charging gun are adjusted according to the contact pressure data so that the mobile charging gun is completely inserted into the charging port.

[0089] It should be understood that the movement angle of the charging gun entering the charging port will affect the smoothness of plugging and unplugging. Since the charging port and the charging gun are in contact at this time, the radar point cloud data and image data cannot provide high posture adjustment support. Therefore, in order to improve the smoothness and safety of plugging and unplugging the charging gun, this embodiment can detect the contact force around the charging gun in real time, and perform admittance control on the charging gun according to the magnitude of the contact force. For example: when the contact force of the force sensor on one side is greater than 20N, the end direction and end posture of the charging gun are adjusted to make the insertion direction of the charging gun deflect.

[0090] This embodiment obtains 6D pose estimation data by fusing RGB image features and point cloud data features through a multimodal feature fusion model; locates the charging port of the device to be charged and the mobile charging gun based on the 6D pose estimation data to obtain the relative position between the charging port and the mobile charging gun; controls the mobile charging gun to perform staged displacement based on at least one of the RGB image features, point cloud data features, and force data features and the relative position, so that the mobile charging gun and the charging port complete the plug-in and unplug operation, thereby improving the efficiency and safety of charging the charging gun.

[0091] This application also provides a charging control device, please refer to Figure 4 , the charging control device includes:

[0092] The acquisition module 10 is used to acquire multimodal charging port data through the double-sided vision camera, the laser radar and the force sensor.

[0093] The extraction module 20 is configured to extract multimodal data features corresponding to the multimodal charging port data.

[0094] The control module 30 is configured to control the mobile charging gun to perform a phased displacement based on at least one of the multimodal data features, so as to complete the plugging and unplugging operation between the mobile charging gun and the charging port.

[0095] This embodiment obtains multimodal charging port data through the double-sided vision camera, the laser radar and the force sensor; extracts multimodal data features corresponding to the multimodal charging port data; and controls the mobile charging gun to perform staged displacement based on at least one of the multimodal data features, so that the mobile charging gun and the charging port can be plugged and unplugged. By controlling the mobile charging pile to move within the charging area, when the new energy vehicle needs to be recharged, the charging gun is automatically controlled to perform staged displacement to achieve plugging and unplugging of the mobile charging gun and the charging port, thereby reducing the waiting time for recharging the new energy vehicle.

[0096] In one embodiment, the control module 30 is further used to perform data fusion on RGB image features and point cloud data features through a multimodal feature fusion model to obtain 6D pose estimation data; locate the charging port of the device to be charged and the mobile charging gun based on the 6D pose estimation data to obtain the relative position between the charging port and the mobile charging gun; and control the mobile charging gun to perform staged displacement according to at least one of the RGB image features, point cloud data features, and force data features and the relative position, so that the mobile charging gun and the charging port complete the plug-in and unplug operation.

[0097] In one embodiment, the control module 30 is further configured to input the RGB image features into a trained visual servoing model to obtain the spatial position of the mobile charging gun when the relative position satisfies the first-stage position condition, wherein the first-stage position condition includes that the relative distance between the charging port and the mobile charging gun is greater than a preset distance; perform path planning based on the spatial position to obtain the first-stage moving path of the mobile charging gun; and control the mobile charging gun to perform a first-stage displacement based on the first-stage moving path so that the mobile charging gun moves to a preset position, wherein the relative distance between the preset position and the charging port is less than a preset distance, and the preset position is level with the spatial position of the charging port.

[0098] In one embodiment, the control module 30 is further used to obtain the charging port three-dimensional model and target point cloud data of the charging port when the relative position meets the second-stage position conditions, and the second-stage position conditions include that the spatial position between the charging port and the mobile charging gun is horizontal, and the relative distance is less than or equal to a preset distance; match the target point cloud data and the point cloud data features based on the charging port three-dimensional model; and control the mobile charging gun to perform a second-stage displacement according to the matching result, so that the mobile charging gun moves to the charging port for alignment.

[0099] In one embodiment, the control module 30 is further used to obtain contact pressure data fed back by the force sensor in real time when the relative position meets the third stage position condition; and adjust the posture and moving speed of the mobile charging gun according to the contact pressure data so that the mobile charging gun can be fully inserted into the charging port.

[0100] In one embodiment, the control module 30 is further used to perform time alignment on the RGB image features and the point cloud data features to obtain target RGB image features and target point cloud data features; extract the two-dimensional image features in the target RGB image features and the key point cloud data features corresponding to the target point cloud data features; perform spatial coordinate alignment according to the two-dimensional image features and the key point cloud data features; and calculate the 6D pose estimation data of the mobile charging gun based on the two-dimensional image features and the key point cloud data features after spatial coordinate alignment.

[0101] The present application provides a charging control device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the charging control method in the above-mentioned embodiment one.

[0102] Reference below Figure 5 , which shows a schematic diagram of the structure of a charging control device suitable for implementing the embodiments of the present application. The charging control device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5The charging control device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0103] like Figure 5 As shown, the charging control device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the charging control device. Processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and communication device 1009. Communication device 1009 can allow the charging control device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a charging control device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or provided instead.

[0104] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0105] The charging control device provided in this application utilizes the charging control method described in the above-mentioned embodiment to solve the technical problem of charging control. Compared with the prior art, the beneficial effects of the charging control device provided in this application are the same as those of the charging control method described in the above-mentioned embodiment. The other technical features of the charging control device are the same as those disclosed in the above-mentioned embodiment and are not further described here.

[0106] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0107] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0108] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the charging control method in the above embodiment.

[0109] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0110] The computer-readable storage medium may be included in the charging control device, or may exist independently without being incorporated into the charging control device.

[0111] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the charging control device, the charging control device performs charging control.

[0112] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0114] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0115] The computer-readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned charging control method, thereby resolving the technical issues surrounding charging control. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the charging control method provided in the aforementioned embodiment, and are not further elaborated here.

[0116] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned charging control method when executed by a processor.

[0117] The computer program product provided in this application can solve the technical problem of charging control. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the charging control method provided in the above embodiment, and will not be repeated here.

[0118] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A charging control method, characterized in that: The charging control method is applied to a charging pile, which includes at least one mobile charging gun and multiple charging areas. The mobile charging gun is equipped with a double-sided vision camera, a laser radar, and a force sensor. The charging control method includes: Acquiring multimodal charging port data through the double-sided vision camera, the laser radar, and the force sensor; Extracting multimodal data features corresponding to the multimodal charging port data; Based on at least one of the multimodal data features, the mobile charging gun is controlled to perform a staged displacement so that the mobile charging gun and the charging port complete the plugging and unplugging operation.

2. The charging control method according to claim 1, wherein: The multimodal charging port data includes at least: an RGB image, point cloud data, and force data of the charging port; the multimodal data features include at least RGB image features, point cloud data features, and force data features; The controlling the mobile charging gun to perform staged displacement based on the multimodal data feature so as to complete the plugging and unplugging operation between the mobile charging gun and the charging port includes: The RGB image features and point cloud data features are fused through a multimodal feature fusion model to obtain 6D pose estimation data; Positioning the charging port of the device to be charged and the mobile charging gun based on the 6D pose estimation data to obtain a relative position between the charging port and the mobile charging gun; The mobile charging gun is controlled to perform staged displacement according to at least one of the RGB image features, the point cloud data features, and the force data features, and the relative position, so that the mobile charging gun and the charging port complete the plugging and unplugging operation.

3. The charging control method according to claim 2, wherein: The controlling the mobile charging gun to perform staged displacement according to at least one of the RGB image feature, the point cloud data feature, and the force data feature and the relative position includes: When the relative position satisfies a first-stage position condition, inputting the RGB image features into a trained visual servoing model to obtain the spatial position of the mobile charging gun, wherein the first-stage position condition includes that the relative distance between the charging port and the mobile charging gun is greater than a preset distance; Performing path planning based on the spatial position to obtain a first-stage moving path of the mobile charging gun; Based on the first-stage moving path, the mobile charging gun is controlled to perform a first-stage displacement so that the mobile charging gun moves to a preset position, the relative distance between the preset position and the charging port is less than a preset distance, and the preset position and the charging port are spatially level.

4. The charging control method according to claim 2, wherein: The charging control method further includes: When the relative position satisfies the second-stage position condition, obtaining the charging port three-dimensional model and target point cloud data of the charging port, wherein the second-stage position condition includes that the spatial position between the charging port and the mobile charging gun is horizontal and the relative distance is less than or equal to a preset distance; Matching the target point cloud data and the point cloud data features based on the charging port three-dimensional model; The mobile charging gun is controlled to perform a second-stage displacement according to the matching result, so that the mobile charging gun moves to the charging port for alignment.

5. The charging control method according to claim 2, wherein: The charging control method further includes: When the relative position satisfies the position condition of the third stage, obtaining contact pressure data fed back by the force sensor in real time; The position and movement speed of the mobile charging gun are adjusted according to the contact pressure data so that the mobile charging gun is completely inserted into the charging port.

6. The charging control method according to claim 2, wherein: The multimodal feature fusion model is used to fuse RGB image features and point cloud data features to obtain 6D pose estimation data, including: Performing time alignment on the RGB image features and the point cloud data features to obtain target RGB image features and target point cloud data features; Extracting key point cloud data features corresponding to the two-dimensional image features and the target point cloud data features in the target RGB image features; Performing spatial coordinate alignment according to the two-dimensional image features and key point cloud data features; The 6D pose estimation data of the mobile charging gun is calculated based on the two-dimensional image features and key point cloud data features after spatial coordinate alignment.

7. A charging control device, characterized in that: The charging control device includes: an acquisition module, configured to acquire multimodal charging port data through the double-sided vision camera, the laser radar, and the force sensor; an extraction module, configured to extract multimodal data features corresponding to the multimodal charging port data; A control module is used to control the mobile charging gun to perform staged displacement based on at least one of the multimodal data features, so that the mobile charging gun and the charging port complete the plug-in and unplug operation.

8. A charging control device, characterized in that: The charging control device includes: a memory, a processor, and a charging control program stored in the memory and executable on the processor, wherein the charging control program is configured to implement the charging control method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a charging control program, and when the charging control program is executed by the processor, the charging control method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the charging control method according to any one of claims 1 to 6 are implemented.

Images