Charging gun intelligent plugging method and device under image identification

Abstract

The invention discloses an intelligent charging gun plugging method and device under image recognition, and relates to the technical field of image processing. The method comprises the following steps: identifying a target vehicle model based on a plurality of vehicle images; retrieving a three-dimensional model of the target vehicle; modeling the target vehicle to generate a target vehicle parking model; obtaining a charging port coordinate point; planning a motion route of the charging gun from the original point to the coordinate point of the charging port; controlling the charging gun to move to the coordinate point of the charging port through the mechanical arm, aligning the charging gun to the charging port, and inserting the charging gun into the charging port for charging; and after charging is completed, the charging gun is pulled out through the mechanical arm and moves reversely according to the movement route, and the charging gun is placed back to the original position. The technical problems that in the prior art, manual charging gun plugging is low in efficiency and inaccurate in positioning, and a vehicle charging port is prone to being damaged are solved, intelligent charging gun plugging is achieved through the image recognition and three-dimensional modeling technology, and the technical effects that the charging efficiency is improved, the charging port is accurately aligned, and the vehicle charging port is effectively protected are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for intelligently plugging and unplugging a charging gun under image recognition. Background Art

[0002] With the popularity of electric vehicles, the demand for electric vehicle charging equipment has increased significantly. Existing charging systems usually rely on manual operation to insert the charging gun into the charging port of the vehicle. This mode of operation is not only inefficient, but also has the problem of inaccurate positioning, which can easily cause the charging gun and the charging port to be misaligned, thereby causing damage to the charging equipment or the vehicle charging port. In addition, due to differences in vehicle models and charging port locations, manual operation is difficult to adapt to diverse charging needs, especially when the charging port is relatively hidden or the vehicle is not parked in an ideal position, which further increases the complexity of the charging operation. Therefore, how to realize the intelligent plugging and unplugging of the charging gun and improve charging efficiency and safety has become a problem that needs to be solved urgently in the current technical field. Summary of the invention

[0003] The present application provides a method and device for intelligent plugging and unplugging of a charging gun under image recognition, which solves the technical problems in the prior art of manual plugging and unplugging of a charging gun, such as low efficiency, inaccurate positioning, and easy damage to the vehicle charging port.

[0004] In view of the above problems, the present application provides a method and device for intelligent plugging and unplugging of a charging gun under image recognition.

[0005] In a first aspect of the present application, a method for intelligent plugging and unplugging of a charging gun under image recognition is provided, the method comprising: Acquire multiple vehicle images of a target vehicle entering a charging parking space, and identify the target vehicle model based on the multiple vehicle images; retrieve a target vehicle three-dimensional model from a three-dimensional vehicle model library according to the target vehicle model, and the target vehicle three-dimensional model has a charging port identification; take the charging gun of the charging parking space as the origin, combine the multiple vehicle images and the target vehicle three-dimensional model, model the actual parking posture of the target vehicle in the charging parking space, and generate a target vehicle parking model; determine the charging port identification in the target vehicle parking model, obtain the position of the charging port identification in the target vehicle parking model, and obtain the charging port coordinate point; plan the movement route of the charging gun from the origin to the charging port coordinate point according to the target vehicle parking model; control the charging gun to move to the charging port coordinate point according to the movement route through a robotic arm, align the charging gun with the charging port of the target vehicle, and insert it into the charging port for charging; after charging is completed, pull out the charging gun through the robotic arm, and reverse along the movement route to place the charging gun back to its original position.

[0006] In a second aspect of the present application, a charging gun intelligent plugging and unplugging device under image recognition is provided, the device comprising: An identification module, the identification module is used to obtain multiple vehicle images of the target vehicle entering the charging parking space, and identify the target vehicle model based on the multiple vehicle images; a vehicle model retrieval module, the vehicle model retrieval module is used to retrieve the target vehicle three-dimensional model from the three-dimensional vehicle model library according to the target vehicle model, and the target vehicle three-dimensional model has a charging port identification; a vehicle posture modeling module, the vehicle posture modeling module is used to take the charging gun of the charging parking space as the origin, combine the multiple vehicle images and the target vehicle three-dimensional model, model the actual parking posture of the target vehicle in the charging parking space, and generate a target vehicle parking model; a charging port positioning module, the charging port positioning module is used to locate the target vehicle in the parking model Determine the charging port identifier, obtain the position of the charging port identifier in the target vehicle parking model, and obtain the charging port coordinate point; a charging route planning module, the charging route planning module is used to plan the movement route of the charging gun from the origin to the charging port coordinate point according to the target vehicle parking model; a charging execution module, the charging execution module is used to control the charging gun to move to the charging port coordinate point according to the movement route through a mechanical arm, align the charging gun with the charging port of the target vehicle, and insert the charging gun into the charging port for charging; a charging end processing module, the charging end processing module is used to pull out the charging gun through the mechanical arm after charging is completed, and reverse along the movement route to put the charging gun back to its original position.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: First, multiple vehicle images of the target vehicle entering the charging parking space are obtained, and the target vehicle model is identified based on the multiple vehicle images. Then, according to the target vehicle model, the target vehicle three-dimensional model is retrieved from the three-dimensional vehicle model library, and the target vehicle three-dimensional model has a charging port identification. Then, taking the charging gun of the charging parking space as the origin, combining multiple vehicle images and the target vehicle three-dimensional model, the actual parking posture of the target vehicle in the charging parking space is modeled to generate the target vehicle parking model. Further, the charging port identification is determined in the target vehicle parking model, the position of the charging port identification in the target vehicle parking model is obtained, and the charging port coordinate point is obtained. Next, according to the target vehicle parking model, the movement route of the charging gun from the origin to the charging port coordinate point is planned. Finally, the charging gun is controlled by the mechanical arm to move to the charging port coordinate point according to the movement route, the charging gun is aligned with the charging port of the target vehicle, and inserted into the charging port for charging; after charging is completed, the charging gun is pulled out by the mechanical arm, and the charging gun is placed back to its original position in reverse according to the movement route. The technical problems of low efficiency, inaccurate positioning and easy damage to the vehicle charging port caused by manual plugging and unplugging of charging guns in the prior art are solved. Intelligent plugging and unplugging of charging guns is realized through image recognition and three-dimensional modeling technology, which achieves the technical effect of improving charging efficiency, accurately aligning the charging port and effectively protecting the vehicle charging port. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0009] Figure 1 A schematic diagram of the process flow of the intelligent plugging and unplugging method of a charging gun under image recognition provided in an embodiment of the present application; Figure 2 This is a schematic diagram of the structure of the intelligent plug-in and unplug-out device for a charging gun under image recognition provided in an embodiment of the present application.

[0010] Explanation of the reference numerals: identification module 11 , vehicle type retrieval module 12 , vehicle posture modeling module 13 , charging port positioning module 14 , charging route planning module 15 , charging execution module 16 , charging end processing module 17 . DETAILED DESCRIPTION

[0011] The present application solves the technical problems in the prior art of manual plugging and unplugging of charging guns, such as low efficiency, inaccurate positioning and easy damage to the vehicle charging port, by providing a method and device for intelligent plugging and unplugging of charging guns under image recognition.

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0013] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or are inherent to these processes, methods, products or devices.

[0014] Embodiment 1, as Figure 1 As shown, the present application provides a method for intelligent plugging and unplugging of a charging gun under image recognition, wherein the method includes: A plurality of vehicle images of a target vehicle entering a charging parking space are acquired, and a model of the target vehicle is identified based on the plurality of vehicle images.

[0015] When the target vehicle enters the charging parking space, the vehicle is first photographed by multiple cameras to obtain images of the vehicle from multiple angles. Then, based on these images, the image recognition algorithm is used to analyze and match the appearance features of the vehicle to identify the target vehicle model.

[0016] Further, acquiring multiple vehicle images of a target vehicle entering a charging parking space, and identifying the target vehicle model based on the multiple vehicle images, includes: A plurality of surveillance cameras are arranged around the charging parking space, and when a target vehicle enters the charging parking space, a plurality of vehicle images of the target vehicle at different angles are acquired; based on the plurality of vehicle images, a plurality of vehicle features of the target vehicle are extracted using a pre-constructed vehicle feature set, wherein the vehicle feature set at least includes a vehicle logo style, a vehicle logo position, a vehicle body contour, and a vehicle body size; based on the plurality of vehicle features, a search is performed in a vehicle database to determine the model of the target vehicle.

[0017] Specifically, multiple surveillance cameras are arranged around the charging parking space to ensure that all-round images of the vehicle can be captured from different angles; when the target vehicle enters the charging parking space, multiple cameras shoot synchronously to obtain multiple vehicle images of the vehicle at different angles; based on the multiple vehicle images obtained, a pre-built vehicle feature set is used to extract features of the target vehicle in the image, wherein the vehicle feature set contains a variety of vehicle identification elements, including but not limited to vehicle logo style, vehicle logo position, body contour, and body size; after extracting multiple key features of the vehicle, these features are matched with a pre-established vehicle database, and the vehicle database is retrieved through the vehicle feature set to ultimately determine the specific model of the target vehicle.

[0018] According to the target vehicle model, a target vehicle three-dimensional model is retrieved from a three-dimensional vehicle model library, and the target vehicle three-dimensional model has a charging port identifier.

[0019] The 3D vehicle model library refers to a database containing a variety of 3D models of vehicles. The 3D vehicle model library includes 3D structural data of various types of vehicles, and each model is marked with specific location information of key parts, especially the location identification of the vehicle's charging port. After the specific model of the target vehicle is successfully identified, the model information will be used to retrieve a 3D model that matches the target vehicle from the 3D vehicle model library. The 3D model contains the vehicle's overall shape, size ratio, detailed features, and identifies the location of the vehicle's charging port. Each 3D vehicle model contains the vehicle's charging port identification, which clearly indicates the specific location and direction of the charging port. The charging port identification is a key data point for the subsequent positioning of the charging gun, ensuring that the charging gun can be accurately aligned and inserted into the vehicle's charging port.

[0020] Taking the charging gun of the charging parking space as the origin, combining the multiple vehicle images and the target vehicle three-dimensional model, the actual parking posture of the target vehicle in the charging parking space is modeled to generate a parking model of the target vehicle.

[0021] Taking the position of the charging gun in the charging parking space as the origin of the coordinate system, based on vehicle images from multiple angles, the image processing algorithm is used to analyze the actual parking posture of the target vehicle, including the position of the vehicle in the parking space, the vehicle's rotation angle (such as whether there is any deviation), height, etc. By combining these data with the standard three-dimensional model of the vehicle, the actual position and angle of the vehicle in the charging parking space can be obtained. Specifically, the external contour of the vehicle is extracted from the image at each angle through image processing algorithms (such as edge detection and contour tracking); based on the extracted vehicle contour, key feature points on the vehicle are selected, such as the roof, wheel position, front and rear bumpers, etc., and these feature points are matched with the three-dimensional model of the target vehicle to calculate the actual spatial posture of the vehicle in the charging parking space; combining images from multiple angles, using three-dimensional reconstruction algorithms (such as stereo vision technology or multi-view geometry) to analyze and model the actual parking posture of the vehicle, through these reconstruction algorithms, the three-dimensional posture information of the vehicle can be generated, including the pitch angle, roll angle, rotation angle, position offset, etc. of the vehicle; combining the origin position of the charging gun, the vehicle image and the three-dimensional model of the target vehicle, a parking model of the target vehicle is constructed, which accurately reflects the actual spatial position, angle and posture of the vehicle in the charging parking space.

[0022] A charging port identifier is determined in the target vehicle parking model, a position of the charging port identifier in the target vehicle parking model is acquired, and a charging port coordinate point is obtained.

[0023] In the three-dimensional model of the target vehicle, the specific location of the charging port has been marked as the charging port logo; after generating the parking model of the target vehicle, the area of ​​the vehicle charging port is located by the charging port logo, so as to obtain the charging port coordinate point, which represents the specific location of the charging port in three-dimensional space.

[0024] According to the target vehicle parking model, a movement route of the charging gun from the origin to the charging port coordinate point is planned.

[0025] The origin of the charging gun (the position of the charging gun is the origin) and the coordinate point of the charging port constitute the starting point and end point of the motion route planning; the path planning algorithm (such as Dijkstra algorithm, A* algorithm, Bezier curve, etc.) is used for route calculation. These algorithms can plan the best motion route of the charging gun according to the obstacles in the three-dimensional space, the actual size of the charging parking space and the parking posture of the vehicle; based on the parking model of the target vehicle, a three-dimensional motion route from the origin of the charging gun to the coordinate point of the charging port will be generated. This route will not only consider the horizontal displacement (X, Y axis direction), but also the height difference between the charging gun and the charging port (Z axis direction) to ensure that the charging gun is aligned with the charging port at the correct height.

[0026] The charging gun is controlled by a robotic arm to move to the charging port coordinate point along the motion route, the charging gun is aligned with the charging port of the target vehicle, and is inserted into the charging port for charging.

[0027] After the path planning is completed, the planned movement route is input into the control system of the robotic arm. The robotic arm usually has multiple degrees of freedom (for example, 6 degrees of freedom) and can move flexibly in three-dimensional space. Through the control parameters of path planning, the robotic arm begins to move from the initial position (origin) of the charging gun to the coordinate point of the vehicle charging port according to the specified route. When the charging gun moves to the vicinity of the vehicle charging port, secondary positioning is performed through visual sensors, laser ranging and other technologies to ensure that the charging gun is aligned at the exact position of the charging port to avoid insertion deviation. After aligning the charging port, the robotic arm gradually reduces the movement speed of the charging gun to ensure that the charging gun is inserted into the vehicle's charging port with appropriate force and angle; after the charging gun is successfully inserted into the charging port, it starts charging the vehicle.

[0028] Further, aligning the charging gun with the charging port of the target vehicle includes: A plurality of micro cameras in a circular array are arranged at the gun head of the charging gun, the number of the micro cameras is not less than 4, and the angle between two adjacent micro cameras is not greater than 90°; when the charging gun moves to the coordinate point of the charging port, the plurality of micro cameras are started to capture images of the charging port to obtain a plurality of real-time images of the charging port; according to the plurality of real-time images of the charging port, the charging gun is controlled to align with the charging port of the target vehicle.

[0029] No less than 4 micro cameras are installed at the tip of the charging gun to form a circular array; the angle between two adjacent micro cameras is no more than 90°, ensuring that the camera can cover the full range of vision around the charging gun; these micro cameras are used to collect real-time images of the charging port and provide precise visual feedback during the alignment process.

[0030] When the charging gun moves close to the coordinate point of the charging port, the micro camera on the head of the charging gun will be automatically started. These cameras will capture images of the charging port of the target vehicle from different angles to ensure comprehensive acquisition of the environment and space information around the charging port; through real-time image acquisition by multiple cameras, the relative position of the charging gun and the charging port can be monitored in real time from different perspectives to obtain multiple real-time images of the charging port; based on the real-time images of multiple charging ports, the position and angle of the charging gun can be adjusted in real time to ensure that the tip of the charging gun is aligned with the center point of the charging port to avoid deviation or poor contact during insertion.

[0031] Furthermore, according to the real-time images of the multiple charging ports, controlling the charging gun to align with the charging port of the target vehicle includes: According to the target vehicle model, multiple standard images of the charging port are obtained, wherein the multiple standard images of the charging port are the images of the charging port captured by each micro camera when the charging gun head is completely aligned with the charging port of the target vehicle; based on the camera numbers of the multiple micro cameras, it is determined whether the multiple real-time images of the charging ports are consistent with the multiple standard images of the charging ports; if the multiple real-time images of the charging ports are consistent with the multiple standard images of the charging ports, it is determined that the charging gun and the charging port of the target vehicle are in an aligned state; if the multiple real-time images of the charging ports are inconsistent with the multiple standard images of the charging ports, the posture of the charging gun is controlled to be adjusted until the charging gun is aligned with the charging port of the target vehicle.

[0032] Specifically, according to the target vehicle model, a standard image of the charging port that matches the model of vehicle is retrieved from a pre-established standard image library of the charging port. The standard image of the charging port is an image of the charging port captured by each micro camera when the charging gun head is completely aligned with the charging port of the target vehicle. Each standard image accurately matches the shape, position, angle and other information of the charging port of a specific vehicle. When comparing the real-time captured charging port image with the standard image, it is necessary to ensure that each real-time image and the corresponding standard image come from the same micro camera (that is, the camera number must be consistent). Through image processing algorithms (such as feature matching, edge detection, template matching, etc.), the consistency of multiple real-time images and corresponding standard images is judged. If the two are highly similar or the feature matching degree reaches a preset threshold, the images are considered to be consistent.

[0033] If the real-time images collected by multiple micro cameras are completely consistent with the standard images, it means that the tip of the charging gun has been completely aligned with the charging port of the target vehicle. At this time, the charging gun will be judged to be in the correct alignment state and ready to perform the charging gun insertion operation. If deviations are detected between the real-time images of multiple charging ports and the standard images (such as inconsistent positions, angles or shapes), it means that the charging gun has not been fully aligned with the charging port; based on these deviations, the posture (i.e., position and posture) of the charging gun is dynamically adjusted to correct its motion trajectory; by fine-tuning the displacement and rotation of the charging gun, ensure that the charging gun can be gradually aligned with the charging port; in the process of adjusting the posture of the charging gun, new real-time images will be continuously collected and compared until the real-time image is consistent with the standard image.

[0034] Further, controlling and adjusting the posture of the charging gun until the charging gun is aligned with the charging port of the target vehicle includes: A micro camera is randomly selected from the multiple micro cameras as the first micro camera, and the corresponding real-time image of the first charging port and the standard image of the first charging port are extracted according to the first micro camera; two feature points are randomly obtained in the standard image of the first charging port as the first standard feature point and the second standard feature point; in the real-time image of the first charging port, a first real-time feature point is obtained according to the first standard feature point, and a second real-time feature point is obtained according to the second standard feature point; the first standard feature point and the second standard feature point are connected to obtain a first standard feature line, and the first real-time feature point and the second real-time feature point are connected to obtain a first real-time feature line; with the goal of making the first real-time feature line overlap the first standard feature line, the posture of the charging gun is controlled and adjusted until the charging gun is aligned with the charging port of the target vehicle.

[0035] Optionally, among the multiple micro cameras in the circular array, a micro camera is randomly selected as the first micro camera; the corresponding real-time image of the first charging port is extracted from the image captured by the first micro camera, and the standard image of the first charging port corresponding to the camera is obtained from the pre-stored standard image set, and the two images respectively represent the current position of the charging gun and the standard reference when the charging port is aligned; in the standard image of the first charging port, two feature points are randomly selected as the first standard feature point and the second standard feature point; in the real-time image of the first charging port, the same method as the standard image is used to locate the feature points corresponding to the first standard feature point and the second standard feature point, which are respectively called the first real-time feature point and the second real-time feature point; in the standard image and the real-time image, the two feature points are connected. The line between the first standard feature point and the second standard feature point is called the first standard feature line, and the line between the first real-time feature point and the second real-time feature point is called the first real-time feature line. These feature lines can reflect the relative position and angle of the charging port; by adjusting the displacement and rotation parameters (i.e., posture) of the charging gun, the first real-time feature line is completely overlapped with the first standard feature line, thereby ensuring that the posture of the charging gun is consistent with the standard posture of the charging port; by gradually adjusting the posture of the charging gun, the first real-time feature line finally overlaps with the first standard feature line, which means that the alignment state of the charging gun and the charging port has met the requirements.

[0036] Further, controlling and adjusting the posture of the charging gun until the charging gun is aligned with the charging port of the target vehicle includes: Extract the image center points of multiple standard images of charging ports to obtain multiple standard center points; extract the graphic center points of multiple real-time images of charging ports to obtain multiple real-time center points; connect the multiple standard center points in sequence to obtain a standard feature surface, and connect the multiple real-time center points in sequence to obtain a real-time feature surface; with the goal of making the real-time feature surface overlap the standard feature surface, control and adjust the posture of the charging gun until the charging gun is aligned with the charging port of the target vehicle.

[0037] Optionally, for each standard image of the charging port, use an image processing algorithm (such as a centroid algorithm, calculating the center after edge detection, etc.) to extract the center point of the charging port in the image to form multiple standard center points; for each real-time image of the charging port, also use the above algorithm to extract the center point of the charging port in the image to form multiple real-time center points; connect the multiple standard center points in sequence to form a feature surface representing the standard position of the charging port, and the standard feature surface describes the posture of the charging port in space under standard conditions, including position and angle information; similarly, connect the multiple real-time center points in sequence to form a feature surface representing the real-time position of the current charging port, and the real-time feature surface represents the spatial position and posture of the real-time charging port under the current viewing angle of the charging gun; determine the position adjustment direction and amplitude of the charging gun by comparing the deviation between the real-time feature surface and the standard feature surface; gradually overlap the real-time feature surface and the standard feature surface by adjusting the displacement and posture of the charging gun. When the real-time feature surface overlaps with the standard feature surface, it is determined that the charging gun has successfully aligned with the charging port of the target vehicle.

[0038] Furthermore, the method further comprises: Match the multiple real-time center points with the multiple standard center points one by one to obtain multiple real-time standard center distances; eliminate outliers in the multiple real-time standard center distances, as well as the corresponding real-time center points and standard center points, to obtain multiple updated real-time center points and multiple updated standard center points; connect the multiple updated real-time center points in sequence to obtain standard feature surfaces, and connect the multiple updated standard center points in sequence to obtain real-time feature surfaces.

[0039] Preferably, multiple real-time center points are matched one by one with multiple standard center points. For each pair of corresponding real-time center points and standard center points, the distance between them is calculated to obtain multiple real-time standard center distances; the calculated multiple real-time standard center distances are analyzed to identify and eliminate abnormal values, which may be due to calculation deviations caused by camera angles, changes in ambient light or other reasons. Specifically, according to the actual application scenario and accuracy requirements, a reasonable distance threshold is set to determine which distances are abnormal, that is, beyond the normal range; each real-time standard center distance is compared with the set threshold. If the distance is greater than the threshold, it is considered that the distance is abnormal. For abnormal distances, the corresponding real-time center point and standard center point are eliminated from the set. After eliminating the abnormal values, multiple updated real-time center points and multiple updated standard center points are obtained. These updated center points are more accurate and can reflect the real position of the charging port. Use the updated center points to construct real-time feature surfaces and standard feature surfaces respectively. The real-time feature surface is formed by connecting the updated real-time center points in sequence, and the standard feature surface is formed by connecting the updated standard center points in sequence.

[0040] After charging is completed, the charging gun is pulled out by the robot arm, and the charging gun is placed back to its original position by moving in reverse along the movement route.

[0041] After charging is completed, the robotic arm will pull out the charging gun. The robotic arm will pull the charging gun out of the vehicle's charging port according to the predetermined force and speed. After the charging gun is pulled out, the robotic arm will reverse according to the insertion route to ensure that the charging gun returns accurately along the previous path. When the charging gun returns to its original position, the robotic arm will safely place the charging gun back to the designated storage location.

[0042] In summary, the embodiments of the present application have at least the following technical effects: First, multiple vehicle images of the target vehicle entering the charging parking space are obtained, and the target vehicle model is identified based on the multiple vehicle images. Then, according to the target vehicle model, the target vehicle three-dimensional model is retrieved from the three-dimensional vehicle model library, and the target vehicle three-dimensional model has a charging port identification. Then, taking the charging gun of the charging parking space as the origin, combining multiple vehicle images and the target vehicle three-dimensional model, the actual parking posture of the target vehicle in the charging parking space is modeled to generate the target vehicle parking model. Further, the charging port identification is determined in the target vehicle parking model, the position of the charging port identification in the target vehicle parking model is obtained, and the charging port coordinate point is obtained. Next, according to the target vehicle parking model, the movement route of the charging gun from the origin to the charging port coordinate point is planned. Finally, the charging gun is controlled by the mechanical arm to move to the charging port coordinate point according to the movement route, the charging gun is aligned with the charging port of the target vehicle, and inserted into the charging port for charging; after charging is completed, the charging gun is pulled out by the mechanical arm, and the charging gun is placed back to its original position in reverse according to the movement route. The technical problems of low efficiency, inaccurate positioning and easy damage to the vehicle charging port caused by manual plugging and unplugging of charging guns in the prior art are solved. Intelligent plugging and unplugging of charging guns is realized through image recognition and three-dimensional modeling technology, which achieves the technical effect of improving charging efficiency, accurately aligning the charging port and effectively protecting the vehicle charging port.

[0043] Embodiment 2 is based on the same inventive concept as the intelligent plugging and unplugging method of a charging gun under image recognition in the above embodiment. Figure 2 As shown, the present application provides a charging gun intelligent plugging and unplugging device under image recognition, wherein the device includes: An identification module 11, the identification module 11 is used to obtain multiple vehicle images of the target vehicle entering the charging parking space, and identify the target vehicle model based on the multiple vehicle images; a vehicle model retrieval module 12, the vehicle model retrieval module 12 is used to retrieve the target vehicle three-dimensional model from the three-dimensional vehicle model library according to the target vehicle model, and the target vehicle three-dimensional model has a charging port identification; a vehicle posture modeling module 13, the vehicle posture modeling module 13 is used to take the charging gun of the charging parking space as the origin, combine the multiple vehicle images and the target vehicle three-dimensional model, model the actual parking posture of the target vehicle in the charging parking space, and generate a target vehicle parking model; a charging port positioning module 14, the charging port positioning module 14 is used to park the target vehicle Determine the charging port identifier in the model, obtain the position of the charging port identifier in the target vehicle parking model, and obtain the charging port coordinate point; a charging route planning module 15, the charging route planning module 15 is used to plan the movement route of the charging gun from the origin to the charging port coordinate point according to the target vehicle parking model; a charging execution module 16, the charging execution module 16 is used to control the charging gun to move to the charging port coordinate point along the movement route through a mechanical arm, align the charging gun with the charging port of the target vehicle, and insert the charging gun into the charging port for charging; a charging end processing module 17, the charging end processing module 17 is used to pull out the charging gun through the mechanical arm after charging is completed, and reverse along the movement route to put the charging gun back to its original position.

[0044] Furthermore, the identification module 11 is used to execute the following method: A plurality of surveillance cameras are arranged around the charging parking space, and when a target vehicle enters the charging parking space, a plurality of vehicle images of the target vehicle at different angles are acquired; based on the plurality of vehicle images, a plurality of vehicle features of the target vehicle are extracted using a pre-constructed vehicle feature set, wherein the vehicle feature set at least includes a vehicle logo style, a vehicle logo position, a vehicle body contour, and a vehicle body size; based on the plurality of vehicle features, a search is performed in a vehicle database to determine the model of the target vehicle.

[0045] Furthermore, the charging execution module 16 is used to execute the following method: A plurality of micro cameras in a circular array are arranged at the gun head of the charging gun, the number of the micro cameras is not less than 4, and the angle between two adjacent micro cameras is not greater than 90°; when the charging gun moves to the coordinate point of the charging port, the plurality of micro cameras are started to capture images of the charging port to obtain a plurality of real-time images of the charging port; according to the plurality of real-time images of the charging port, the charging gun is controlled to align with the charging port of the target vehicle.

[0046] Furthermore, the charging execution module 16 is used to execute the following method: According to the target vehicle model, multiple standard images of the charging port are obtained, wherein the multiple standard images of the charging port are the images of the charging port captured by each micro camera when the charging gun head is completely aligned with the charging port of the target vehicle; based on the camera numbers of the multiple micro cameras, it is determined whether the multiple real-time images of the charging ports are consistent with the multiple standard images of the charging ports; if the multiple real-time images of the charging ports are consistent with the multiple standard images of the charging ports, it is determined that the charging gun and the charging port of the target vehicle are in an aligned state; if the multiple real-time images of the charging ports are inconsistent with the multiple standard images of the charging ports, the posture of the charging gun is controlled to be adjusted until the charging gun is aligned with the charging port of the target vehicle.

[0047] Furthermore, the charging execution module 16 is used to execute the following method: A micro camera is randomly selected from the multiple micro cameras as the first micro camera, and the corresponding real-time image of the first charging port and the standard image of the first charging port are extracted according to the first micro camera; two feature points are randomly obtained in the standard image of the first charging port as the first standard feature point and the second standard feature point; in the real-time image of the first charging port, a first real-time feature point is obtained according to the first standard feature point, and a second real-time feature point is obtained according to the second standard feature point; the first standard feature point and the second standard feature point are connected to obtain a first standard feature line, and the first real-time feature point and the second real-time feature point are connected to obtain a first real-time feature line; with the goal of making the first real-time feature line overlap the first standard feature line, the posture of the charging gun is controlled and adjusted until the charging gun is aligned with the charging port of the target vehicle.

[0048] Furthermore, the charging execution module 16 is used to execute the following method: Extract the image center points of multiple standard images of charging ports to obtain multiple standard center points; extract the graphic center points of multiple real-time images of charging ports to obtain multiple real-time center points; connect the multiple standard center points in sequence to obtain a standard feature surface, and connect the multiple real-time center points in sequence to obtain a real-time feature surface; with the goal of making the real-time feature surface overlap the standard feature surface, control and adjust the posture of the charging gun until the charging gun is aligned with the charging port of the target vehicle.

[0049] Furthermore, the charging execution module 16 is used to execute the following method: Match the multiple real-time center points with the multiple standard center points one by one to obtain multiple real-time standard center distances; eliminate outliers in the multiple real-time standard center distances, as well as the corresponding real-time center points and standard center points, to obtain multiple updated real-time center points and multiple updated standard center points; connect the multiple updated real-time center points in sequence to obtain standard feature surfaces, and connect the multiple updated standard center points in sequence to obtain real-time feature surfaces.

[0050] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0051] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0052] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. Intelligent plug-in and unplug method of charging gun under image recognition, It is characterized in that The method comprises: Acquire multiple vehicle images of a target vehicle entering a charging parking space, and identify a model of the target vehicle based on the multiple vehicle images; According to the target vehicle model, a target vehicle three-dimensional model is retrieved from a three-dimensional vehicle model library, wherein the target vehicle three-dimensional model has a charging port identifier; Taking the charging gun of the charging parking space as the origin, combining the multiple vehicle images and the three-dimensional model of the target vehicle, modeling the actual parking posture of the target vehicle in the charging parking space, and generating a parking model of the target vehicle; Determine a charging port identifier in the target vehicle parking model, obtain a position of the charging port identifier in the target vehicle parking model, and obtain a charging port coordinate point; According to the target vehicle parking model, planning the movement route of the charging gun from the origin to the charging port coordinate point; Controlling the charging gun to move to the charging port coordinate point along the motion route through a robotic arm, aligning the charging gun with the charging port of a target vehicle, and inserting the charging gun into the charging port for charging; After charging is completed, the charging gun is pulled out by the robot arm, and the charging gun is placed back to its original position in reverse along the movement route; Aligning the charging gun with the charging port of the target vehicle, including: A plurality of micro cameras in a circular array are arranged on the gun head of the charging gun, the number of the micro cameras is not less than 4, and the angle between two adjacent micro cameras is not greater than 90°; When the charging gun moves to the coordinate point of the charging port, the multiple micro cameras are started to collect images of the charging port to obtain multiple real-time images of the charging port; According to the real-time images of the multiple charging ports, controlling the charging gun to align with the charging port of the target vehicle; Wherein, according to the real-time images of the multiple charging ports, controlling the charging gun to align with the charging port of the target vehicle comprises: According to the target vehicle model, a plurality of standard images of the charging port are obtained, wherein the plurality of standard images of the charging port are images of the charging port captured by each micro camera when the charging gun head is completely aligned with the charging port of the target vehicle; Based on the camera numbers of the multiple micro cameras, determining whether the multiple real-time images of the charging ports are consistent with the multiple standard images of the charging ports; If the real-time images of the multiple charging ports are consistent with the multiple standard images of the charging ports, it is determined that the charging gun and the charging port of the target vehicle are aligned; If the real-time images of the multiple charging ports are inconsistent with the standard images of the multiple charging ports, the position and posture of the charging gun are controlled to be adjusted until the charging gun is aligned with the charging port of the target vehicle.

2. According to the image recognition method of claim 1, It is characterized in that Acquiring multiple vehicle images of a target vehicle entering a charging parking space, and identifying a target vehicle model based on the multiple vehicle images, including: A plurality of surveillance cameras are arranged around the charging parking space, and when a target vehicle enters the charging parking space, a plurality of vehicle images of the target vehicle at different angles are obtained; Based on the multiple vehicle images, a pre-built vehicle feature set is used to extract multiple vehicle features of the target vehicle, wherein the vehicle feature set at least includes a vehicle logo style, a vehicle logo position, a vehicle body contour, and a vehicle body size; According to the multiple vehicle features, a search is performed in a vehicle database to determine the target vehicle model.

3. According to the image recognition method of claim 1, It is characterized in that Controlling and adjusting the posture of the charging gun until the charging gun is aligned with the charging port of the target vehicle includes: Randomly select a micro camera from the multiple micro cameras as a first micro camera, and extract the corresponding first charging port real-time image and the first charging port standard image according to the first micro camera; Randomly acquiring two feature points in the first charging port standard image as first standard feature points and second standard feature points; In the first charging port real-time image, a first real-time feature point is acquired according to the first standard feature point, and a second real-time feature point is acquired according to the second standard feature point; Connecting the first standard feature point and the second standard feature point to obtain a first standard feature line, and connecting the first real-time feature point and the second real-time feature point to obtain a first real-time feature line; With the goal of making the first real-time characteristic line overlap the first standard characteristic line, the posture of the charging gun is controlled and adjusted until the charging gun is aligned with the charging port of the target vehicle.

4. According to the method for intelligent plugging and unplugging of charging guns under image recognition according to claim 1, It is characterized in that Controlling and adjusting the posture of the charging gun until the charging gun is aligned with the charging port of the target vehicle includes: Extract image center points of multiple charging port standard images to obtain multiple standard center points; Extracting the center points of the graphics of multiple real-time images of charging ports to obtain multiple real-time center points; Connecting multiple standard center points in sequence to obtain a standard feature surface, and connecting multiple real-time center points in sequence to obtain a real-time feature surface; With the goal of making the real-time feature surface overlap the standard feature surface, the position and posture of the charging gun is controlled and adjusted until the charging gun is aligned with the charging port of the target vehicle.

5. According to the method for intelligent plugging and unplugging of charging guns under image recognition according to claim 4, It is characterized in that The method further comprises: Match the multiple real-time center points with the multiple standard center points one by one to obtain multiple real-time standard center distances; Eliminate outliers in multiple real-time standard center distances, as well as corresponding real-time center points and standard center points, and obtain multiple updated real-time center points and multiple updated standard center points; Connect multiple updated real-time center points in sequence to obtain a standard feature surface, and connect multiple updated standard center points in sequence to obtain a real-time feature surface.

6. Intelligent plug-in and unplug device for charging guns under image recognition, It is characterized in that The device is used to implement the intelligent plugging and unplugging method of a charging gun under image recognition according to any one of claims 1 to 5, and comprises: an identification module, the identification module being used to obtain multiple vehicle images of a target vehicle entering a charging parking space, and to identify a model of the target vehicle based on the multiple vehicle images; A vehicle model retrieval module, the vehicle model retrieval module is used to retrieve a target vehicle three-dimensional model from a three-dimensional vehicle model library according to the target vehicle model, the target vehicle three-dimensional model having a charging port identification; A vehicle posture modeling module, wherein the vehicle posture modeling module is used to model the actual parking posture of the target vehicle in the charging parking space by taking the charging gun of the charging parking space as the origin, combining the multiple vehicle images and the target vehicle three-dimensional model, and generating a parking model of the target vehicle; A charging port positioning module, the charging port positioning module is used to determine the charging port identifier in the target vehicle parking model, obtain the position of the charging port identifier in the target vehicle parking model, and obtain the charging port coordinate point; A charging route planning module, the charging route planning module is used to plan the movement route of the charging gun from the origin to the charging port coordinate point according to the target vehicle parking model; A charging execution module, the charging execution module is used to control the charging gun to move to the charging port coordinate point according to the motion route through a mechanical arm, align the charging gun with the charging port of the target vehicle, and insert the charging gun into the charging port for charging; A charging end processing module, wherein after charging is completed, the charging end processing module is used to pull out the charging gun through the mechanical arm, and reverse along the movement route to place the charging gun back to its original position; The charging execution module is used to execute the following method: A plurality of micro cameras in a circular array are arranged on the gun head of the charging gun, the number of the micro cameras is not less than 4, and the angle between two adjacent micro cameras is not greater than 90°; When the charging gun moves to the coordinate point of the charging port, the multiple micro cameras are started to collect images of the charging port to obtain multiple real-time images of the charging port; According to the real-time images of the multiple charging ports, controlling the charging gun to align with the charging port of the target vehicle; Wherein, the charging execution module is also used to execute the following method: According to the target vehicle model, a plurality of standard images of the charging port are obtained, wherein the plurality of standard images of the charging port are images of the charging port captured by each micro camera when the charging gun head is completely aligned with the charging port of the target vehicle; Based on the camera numbers of the multiple micro cameras, determining whether the multiple real-time images of the charging ports are consistent with the multiple standard images of the charging ports; If the real-time images of the multiple charging ports are consistent with the multiple standard images of the charging ports, it is determined that the charging gun and the charging port of the target vehicle are aligned; If the real-time images of the multiple charging ports are inconsistent with the standard images of the multiple charging ports, the position and posture of the charging gun are controlled to be adjusted until the charging gun is aligned with the charging port of the target vehicle.

Images