Charging port key point marking method, equipment and medium

Abstract

The present application provides a method, device and medium for key point labeling of a charging port, and relates to the field of computer vision technology. The method obtains a first image data set and a second image data set of a vehicle charging port, and performs coverage processing on the key point positions of the charging port in the first image data set, and does not perform coverage processing on the key point positions of the charging port in the second image data set; performs batch detection on the first image data set, and generates a key point information set and a corresponding original image set based on the batch detection results and the second image data set, and each key point information file in the key point information set stores the key point information of the image in the corresponding storage order in the original image set; generates a key point labeling file for the charging port based on the key point information set and the original image set. The method of the present application improves the work efficiency and accuracy of key point labeling of the charging port.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device and medium for marking key points of a charging port. Background Art

[0002] Key point annotation is a task in computer vision technology, which is mainly used to annotate the key points of target objects in images, such as facial features, human joints, hand key points, etc.

[0003] In the related art, the key point annotation of the charging port is usually completed by manually operating the annotation tool, that is, manually collecting images of the vehicle charging port at different angles and manually marking the key points of interest on each charging port image. For example, the operator can use tools such as LabelMe and CVAT to manually select the key point position of the charging port in each image data.

[0004] However, the manual marking of key points is inefficient, and there are large errors in the subjective selection of the key point positions of the charging port, resulting in low accuracy in the marking of the key points of the charging port. Summary of the invention

[0005] The embodiments of the present application provide a method, device and medium for marking key points of a charging port, which can improve the work efficiency and accuracy of marking key points of a charging port.

[0006] In a first aspect, an embodiment of the present application provides a method for marking key points of a charging port, including:

[0007] Acquire a first image data set and a second image data set of a charging port of a vehicle, wherein key point positions of the charging port in the first image data set are covered, and key point positions of the charging port in the second image data set are not covered, and images in the same storage order in the two image data sets are collected under the same camera posture;

[0008] Performing a labeled batch detection on the first image data set, and generating a key point information set and a corresponding original image set according to the batch detection result and the second image data set, wherein each key point information file in the key point information set stores key point information of images in a corresponding storage order in the original image set;

[0009] A key point annotation file of the charging port is generated according to the key point information set and the original image set.

[0010] In a possible implementation, performing a labeled batch detection on the first image data set, and generating a key point information set and a corresponding original image set according to the batch detection result and the second image data set, includes:

[0011] Performing labeled batch detection on the first image in the first image data set;

[0012] For each first image, determine whether a mark detection result of the first image is qualified; if the mark detection result is qualified, generate a corresponding key point information file according to the mark detection result and the storage order of the first image; if the mark detection result is unqualified, delete the second image in the second image data set that has the same storage order as the first image;

[0013] A key point information set is generated according to each of the key point information files, and an original image set is generated according to the second images in the second image data set that correspond one-to-one to the storage order of each key point information file.

[0014] In a possible implementation manner, the key point positions are corner points of the upper end, the lower end, the left end, and the right end of the connection terminal in the charging port, and the connection terminal includes a positive terminal, a negative terminal, and a protective grounding terminal;

[0015] The key point positions of the charging port in the first image data set are covered with a two-dimensional code, and the corner point positions of the two-dimensional code coincide with the corner point positions of the connecting terminal, so that the two-dimensional code covers the key point positions of the charging port.

[0016] In a possible implementation manner, determining whether the mark detection result of the first image is qualified includes:

[0017] Determine whether the first image detects pixel coordinates of a preset number of corner points, where the preset number is the number of corner points of the QR code covered by the charging port;

[0018] If the pixel coordinates of a preset number of corner points are detected, it is determined that the mark detection result of the first image is qualified;

[0019] If the pixel coordinates of the preset number of corner points are not detected, it is determined that the mark detection result of the first image is unqualified.

[0020] In a possible implementation manner, generating a corresponding key point information file according to the mark detection result and the storage order of the first image includes:

[0021] A corresponding key point information file is generated according to the pixel coordinates of the preset number of corner points and the storage order of the first image.

[0022] In a possible implementation manner, acquiring the first image data set and the second image data set of the vehicle charging port includes:

[0023] According to a preset data acquisition path of the robotic arm and a camera disposed at the end of the robotic arm, a first image of the charging port under different camera postures when the charging port is covered is acquired;

[0024] The first image is saved to a first storage location in the order in which the images are generated, so as to generate a first image data set when the key point position of the charging port is covered;

[0025] After detecting that the key point position of the charging port is uncovered, obtaining a second image of the charging port in different camera postures when the charging port is not covered according to a preset robotic arm data acquisition path and a camera disposed at the end of the robotic arm;

[0026] The second image is saved to a second storage location in the order in which the images are generated, so as to generate a second image data set when the key point position of the charging port is not covered.

[0027] In a possible implementation, acquiring the first image / the second image according to a preset data acquisition path of the robotic arm and a camera disposed at the end of the robotic arm includes:

[0028] Control the robotic arm to move to the starting point of the robotic arm data acquisition path, and control the robotic arm to operate according to the robotic arm data acquisition path and a preset movement speed, and during the operation of the robotic arm, control the camera to acquire the first image / second image of the charging port in different camera postures according to a preset image acquisition time and image acquisition frame rate.

[0029] In a second aspect, an embodiment of the present application provides a computing device, including:

[0030] an acquisition module, configured to acquire a first image data set and a second image data set of a vehicle charging port, wherein key point positions of the charging port in the first image data set are covered, and key point positions of the charging port in the second image data set are not covered, and images in the same storage order in the two image data sets are collected under the same camera posture;

[0031] A processing module is used to perform batch detection on the first image data set, and generate a key point information set and a corresponding original image set based on the batch detection results and the second image data set, each key point information file in the key point information set storing key point information of images in the original image set in a corresponding storage order; based on the key point information set and the original image set, a key point annotation file of the charging port is generated.

[0032] In a third aspect, an embodiment of the present application provides another computing device, including:

[0033] A processor, and a memory communicatively connected to the processor;

[0034] Memory is used to store computer executable instructions;

[0035] The processor is used to execute the computer-executable instructions stored in the memory, so that the processor executes the above-mentioned first aspect and / or various possible implementations of the first aspect.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above-mentioned first aspect and / or various possible implementation methods of the first aspect.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the above-mentioned first aspect and / or various possible implementation methods of the first aspect.

[0038] The embodiment of the present application provides a method, device and medium for marking key points of a charging port, which can respectively obtain a first image data set after the key point position of the vehicle charging port is covered and a second image data set without covering the key point position, and mark the first image data set for batch detection. According to the batch detection results and the second image data set, a key point information set and a corresponding original image set can be obtained, so as to generate a key point marking file of the charging port according to the key point information set and the original image set. Through such a setting, a semi-automatic marking method for key points of a charging port is provided, and the operator only needs to manually cover the key point position of the charging port when collecting the first image data set, and remove the coverage of the key point position when collecting the second image data set. Other key point marking processes can be performed automatically, which not only greatly improves the work efficiency of key point marking of the charging port, but also ensures the consistency of extraction of each key point of the charging port, greatly reduces the key point selection error caused by manual marking, and improves the precision and accuracy of key point marking of the charging port. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] Figure 1 A system architecture diagram of an embodiment of the present application;

[0041] Figure 2 This is a flow chart of a method for marking key points of a charging port according to an embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the key points of the charging port according to an embodiment of the present application;

[0043] Figure 4 This is a schematic diagram of key point position coverage in one embodiment of the present application;

[0044] Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the structure of a computing device according to another embodiment of the present application.

[0046] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0048] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0050] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0051] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0052] The charging port key point marking method, device and medium of the present application can be used in the field of computer vision technology, and can also be used in any field outside the field of computer vision technology, such as the field of target detection, the field of key point marking, etc. The application field of the charging port key point marking method, device and medium of the present application is not limited.

[0053] The charging port key point marking method, device and medium of the present application can be applied to the scenario of key point marking of the vehicle charging port. Any scenario involving key point marking of the charging port during model training can apply the charging port key point marking method, device and medium of the present application.

[0054] First, the terms involved in this application are explained:

[0055] Aruco is a binary visual marker-based library that is widely used in computer vision, especially in robot positioning, camera pose estimation, and augmented reality. Aruco markers are patterns similar to QR codes. By detecting these markers, the position and pose of the camera relative to the marker can be calculated.

[0056] Key point annotation is a task in computer vision technology, which is mainly used to annotate the key points of target objects in images, such as facial features, human joints, hand key points, etc., to obtain key point annotation data, so that the model trained according to the key point annotation data has the ability to detect and identify the key points of target objects in images.

[0057] In the related art, key point annotation is usually completed by manually operating annotation tools, that is, manually collecting images of the vehicle charging port at different angles and manually marking the key points of interest on each target object. For example, operators can use tools such as LabelMe and CVAT to manually select the key point positions of the target object in each image data.

[0058] Since the manual selection of the key points of each charging port image data is a subjective operation, there is a large error in the position selection of each charging port key point, resulting in low accuracy of the charging port key point marking and low efficiency of manual key point marking.

[0059] Based on the above technical problems, the inventive concept of the present application is: how to provide a charging port key point marking solution that can improve the work efficiency and accuracy of charging port key point marking.

[0060] The embodiments of the present application provide a method, device and medium for marking key points of a charging port, and can provide a semi-automatic marking process for key points of a charging port. The operator only needs to manually cover the key point positions of the charging port when collecting the first image data set, and remove the coverage of the key point positions when collecting the second image data set. The computing device can automatically obtain the first image data set and the second image data set, and mark the first image data set for batch detection. According to the batch detection results and the second image data set, the key point information set and the corresponding original image set can be obtained, thereby automatically generating a key point marking file for the charging port according to the key point information set and the original image set, thereby greatly improving the work efficiency and accuracy of the key point marking of the charging port.

[0061] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0062] Figure 1 This is a system architecture diagram of an embodiment of the present application, such as Figure 1As shown, the computing device is connected to the mechanical arm in communication, and the computing device can send a control signal to the mechanical arm to control the mechanical arm to operate according to the set mechanical arm data acquisition path. A camera is provided at the end of the mechanical arm, and the computing device is also connected to the camera in communication. The computing device can send a control signal to the camera to control the camera to collect image data of the charging port during the operation of the mechanical arm, and the camera can send the collected image to the computing device. According to the preset mechanical arm data acquisition path and the camera set at the end of the mechanical arm, the computing device can respectively obtain the first image data set and the second image data set of the vehicle charging port, the key point position of the charging port in the first image data set is covered, the key point position of the charging port in the second image data set is not covered, and the images with the same storage order in the two image data sets are collected under the same camera posture; the first image data set is marked and batch detected, and a key point information set and a corresponding original image set are generated according to the batch detection results and the second image data set, and each key point information file in the key point information set stores the key point information of the image with the corresponding storage order in the original image set; according to the key point information set and the original image set, a key point annotation file of the charging port is generated.

[0063] Figure 2 This is a flow chart of a method for marking key points of a charging port according to an embodiment of the present application. This embodiment uses a computing device as an execution subject to illustrate the method for marking key points of a charging port. Figure 2 As shown, the charging port key point marking method may include the following steps:

[0064] S201: Acquire a first image data set and a second image data set of a vehicle charging port.

[0065] In this embodiment, the key point positions of the charging port in the first image data set are overwritten, and the key point positions of the charging port in the second image data set are not overwritten, and the images with the same storage order in the two image data sets are collected under the same camera posture.

[0066] In this embodiment, the computing device can be a terminal device such as a computer or a mobile phone, or a server device such as a physical server or a cloud server, or a terminal-server integrated device. No restrictions are imposed here as long as the computing device can implement the charging port key point marking method of this application.

[0067] In this embodiment, the operator can first cover the key point position of the vehicle charging port, and then set up a robotic arm data acquisition path script and a camera image data acquisition script in the computing device. After that, the computing device is used to control the robotic arm to run the preset robotic arm data acquisition path with the above script, and the camera set at the end of the robotic arm performs image acquisition, so as to obtain the first image of the charging port under different camera postures when the coverage processing is performed, and the first image data set is stored in the order of generation of the first image.

[0068] In this embodiment, after the acquisition of the first image data set is completed, the operator can remove the coverage of the key point position of the charging port, and run the robotic arm to the starting point of the robotic arm data acquisition path, and use the computing device to control the robotic arm to run the pre-set robotic arm data acquisition path with the same script. The camera set at the end of the robotic arm performs image acquisition, and the second image with the same camera posture as the first image when the charging port is not covered can be obtained, and the second image data set is stored in the order in which the second images are generated.

[0069] In this embodiment, the first image in the first image data set is stored in the order of image generation, and the second image in the second image data set is also stored in the order of image generation. Since the first image data set and the second image data set are acquired using the same robotic arm data acquisition path, the first image corresponds to the second image one-to-one, and the images in the same storage order in the two image data sets are acquired under the same camera posture, and are exactly the same except for whether the charging port is covered.

[0070] In this embodiment, the key point positions of the charging port can be covered with a two-dimensional code such as an Aruco code or an Apriltag code. Of course, other covering processing methods can also be used, and no limitation is made here.

[0071] S202: Performing a labeled batch detection on the first image data set, and generating a key point information set and a corresponding original image set according to the batch detection result and the second image data set.

[0072] In this embodiment, each key point information file in the key point information set stores key point information of images in a corresponding storage order in the original image set.

[0073] In this embodiment, the key point positions of the charging port in the first image data set are overwritten. Therefore, the first image data set can be marked for batch detection to obtain the key point information in each image. Based on the batch detection results and the second image data set, a key point information set and the corresponding original image set can be generated.

[0074] In this embodiment, the key point information files in the key point information set correspond one-to-one to the images in the original image set, and the key point information files in the same storage order store the key point information of the images in the corresponding storage order.

[0075] S203: Generate a key point annotation file of the charging port according to the key point information set and the original image set.

[0076] In this embodiment, the original image set includes original images of the charging port in different camera postures, and the key point information set stores the key point information of each original image. Therefore, a key point annotation file of the charging port can be generated based on the key point information set and the original image set.

[0077] In this embodiment, after completing the data processing of the above steps S201-S202, the images folder (original image set) storing the original image of the charging port and the labels folder (key point information set) can be obtained. The labels folder contains all txt files (key point labeling files) with the same prefix as the original image. Each txt file stores the key point information of each corresponding original image of the charging port. Finally, all the data that can be used for key point training is obtained to complete semi-automatic labeling.

[0078] In this embodiment, each image / file in the first image data set, the second image data set, the key point information set, and the original image set may be named according to the corresponding image generation order / storage order.

[0079] In this embodiment, the first image data set after the key point position of the vehicle charging port is covered and the second image data set without covering can be obtained respectively, and the first image data set is marked for batch detection. According to the batch detection results and the second image data set, the key point information set and the corresponding original image set can be obtained, so as to generate the key point annotation file of the charging port according to the key point information set and the original image set. Through such a setting, a semi-automatic annotation method for the key points of the charging port is provided. The operator only needs to manually cover the key point position of the charging port when collecting the first image data set, and remove the coverage of the key point position when collecting the second image data set. The other key point annotation processes can be automatically performed, which not only greatly improves the work efficiency of the key point annotation of the charging port, but also ensures the consistency of the extraction of each key point of the charging port, greatly reduces the key point selection error caused by manual annotation, and improves the precision and accuracy of the key point annotation of the charging port.

[0080] In a possible implementation, the above step S201 of acquiring the first image data set and the second image data set of the vehicle charging port may include:

[0081] S11: According to a preset data acquisition path of the robotic arm and a camera disposed at the end of the robotic arm, a first image of the charging port under different camera postures when the charging port is covered is obtained.

[0082] S12: Saving the first image to a first storage location in the order in which the images are generated, so as to generate a first image data set for covering the key point position of the charging port.

[0083] S13: After detecting that the key point position of the charging port is uncovered, a second image of the charging port under different camera postures when the charging port is not covered is obtained according to a preset robotic arm data acquisition path and a camera disposed at the end of the robotic arm.

[0084] S14: Saving the second image to a second storage location in the order in which the images are generated, so as to generate a second image data set when the key point position of the charging port is not covered.

[0085] In this embodiment, the operator can set up a robotic arm data acquisition path script and a camera image data acquisition script in the computing device, and then use the computing device to control the robotic arm to run the preset robotic arm data acquisition path with the above script, and the camera set at the end of the robotic arm performs image acquisition, so as to obtain the first image of the charging port under different camera postures when the covering process is performed.

[0086] In this embodiment, the preset robot arm data collection path can be flexibly set by those skilled in the art. For example, the robot arm data collection path can be a circular path or an arc path, and no limitation is made here.

[0087] In this embodiment, the first storage location and the second storage location can be flexibly set by those skilled in the art, and no limitation is imposed herein, as long as the first storage location and the second storage location are not the same.

[0088] Exemplarily, the computing device can run a robot arm data acquisition path script and a camera image data acquisition script to obtain a set of first images of charging ports covered with Aruco codes, and save the first images in a folder image_with_qr according to sequential indexing to obtain a first image data set.

[0089] Afterwards, the computing device can run the robotic arm to the starting point of the path, and the operator can remove the Aruco code covering the charging port. After detecting the key point position of the charging port and removing the cover, the computing device can run the same robotic arm data acquisition path script and camera image data acquisition script again to obtain the second image of the corresponding index at the same position, and save it to the folder images to obtain the second image data set.

[0090] In this embodiment, the first image data set and the second image data set are acquired according to the same robotic arm data acquisition path and a camera arranged at the end of the robotic arm, and are stored in the order of image generation, so that the first image in the first image data set corresponds one-to-one to the second image in the second image data set, and the images in the same storage order in the two image data sets are acquired under the same camera posture, and are exactly the same except whether the charging port is covered.

[0091] In a possible implementation, the step S11 / S13 of acquiring the first image / the second image according to the preset robot arm data acquisition path and the camera disposed at the end of the robot arm may include:

[0092] Control the robotic arm to move to the starting point of the robotic arm data acquisition path, and control the robotic arm to run according to the robotic arm data acquisition path and the preset movement speed. During the operation of the robotic arm, control the camera to collect the first image / second image of the charging port under different camera postures according to the preset image acquisition time and image acquisition frame rate.

[0093] In this embodiment, the movement speed of the robot arm, the image acquisition time of the camera, and the image acquisition frame rate can be flexibly set by those skilled in the art according to actual conditions, and no limitation is made here.

[0094] In this embodiment, when acquiring the first image / the second image, the robotic arm operates according to the robotic arm data acquisition path and the preset movement speed, and the camera also acquires according to the preset image acquisition time and image acquisition frame rate, ensuring that the starting position and speed of the robotic arm movement are the same during the two acquisitions, and the camera acquisition start time point and frame rate are also the same, so that the first image / the second image with the same storage order is acquired under the same camera posture, and the two sets of data differ only in whether there is coverage or not at the key point position of the charging port.

[0095] In a possible implementation, the above step S202 performs batch detection on the first image data set and generates a key point information set and a corresponding original image set according to the batch detection result and the second image data set, which may include:

[0096] S21: Performing label batch detection on the first image in the first image data set.

[0097] S22: For each first image, determine whether the mark detection result of the first image is qualified; if the mark detection result is qualified, generate a corresponding key point information file based on the mark detection result and the storage order of the first image; if the mark detection result is unqualified, delete the second image in the second image data set with the same storage order as the first image.

[0098] S23: Generate a key point information set according to each key point information file, and generate an original image set according to the second images in the second image data set that correspond one-to-one to the storage order of each key point information file.

[0099] In this embodiment, the Aruco code batch detection script can be run, and the label batch detection can be performed on each first image in the first image data set to detect whether the label in each first image is qualified.

[0100] In this embodiment, if the mark detection result of a first image is unqualified, the second image in the second image data set that is stored in the same order as the first image can be deleted, and the mark detection result is not saved.

[0101] In this embodiment, the key point information file and the key point information set can be stored in a third storage location. The third storage location can be flexibly set by technical personnel in this field without any limitation, as long as the third storage location is different from the first storage location and the second storage location.

[0102] Exemplarily, if the mark detection result of the first image with sequence number 0003 in the image_with_qr folder is qualified, a txt file with sequence number 0003 is generated according to the mark detection result, and the txt file is stored in the labels folder.

[0103] If the mark detection result of the first image with sequence number 0005 in the image_with_qr folder fails, the mark detection result of the first image with sequence number 0005 is not saved, and the second image with sequence number 0005 in the images folder is deleted.

[0104] In this embodiment, the first image data set that is covered with the key point position of the charging port can be subjected to batch detection of markings to detect whether the markings in each first image are qualified. If the marking detection result of the first image is qualified, the corresponding key point information file can be generated according to the marking detection result and the storage order of the first image. Through such a setting, the consistency of the key point position extraction of each first image can be guaranteed, the validity of the key point information file can be guaranteed, the key point selection error caused by manual marking can be greatly reduced, and the precision and accuracy of the key point marking of the charging port can be improved.

[0105] In a possible implementation, the key point positions may be the corner points of the upper end, lower end, left end, and right end of the connection terminals in the charging port, and the connection terminals include a positive terminal, a negative terminal, and a protective ground terminal.

[0106] The key point positions of the charging port in the first image data set are covered with a QR code, and the corner point positions of the QR code coincide with the corner point positions of the connecting terminal, so that the QR code covers the key point positions of the charging port.

[0107] For example, Figure 3 Schematic diagram of key points of the charging port according to an embodiment of the present application. Figure 3 As shown, the key point positions can be the top, leftmost, rightmost and bottommost of the DC+, DC- and PE ports in the charging port, for a total of 12 corner point positions.

[0108] Figure 4 This is a schematic diagram of key point position coverage in an embodiment of the present application. Figure 4 As shown, the key point position of the charging port in the first image data set can be covered by three QR codes, covering DC+, DC-, and PE ports respectively, and the corner points of each QR code coincide with the corner point positions of the DC+, DC-, and PE ports respectively.

[0109] In this embodiment, the key point positions of the charging port can be covered by using a two-dimensional code such as an Aruco code or an Apriltag code.

[0110] In this embodiment, the key point positions of the charging port can be covered by a QR code, so as to facilitate batch detection of the first image data set based on the QR code. The corner point positions of the QR code coincide with the corner point positions of the connection terminal, so that the QR code covers the key point positions of the charging port, ensuring the consistency of the key point position extraction of each first image, and further avoiding key point position selection errors.

[0111] In a possible implementation, determining whether the mark detection result of the first image is qualified in the above step S22 may include:

[0112] S31: Determine whether the first image detects pixel coordinates of a preset number of corner points, where the preset number is the number of corner points of the QR code covered by the charging port.

[0113] S32: If the pixel coordinates of a preset number of corner points are detected, it is determined that the mark detection result of the first image is qualified.

[0114] S33: If the pixel coordinates of the preset number of corner points are not detected, it is determined that the mark detection result of the first image is unqualified.

[0115] In this embodiment, since the key points of the charging port are the top, left, right and bottom corners of the DC+, DC- and PE ports, the preset number is 12. That is, if the pixel coordinates of the 12 corner points of the three QR codes are detected, it can be determined that the mark detection result of the first image is qualified.

[0116] In this embodiment, since the key point positions of the charging port are the corner points of the upper end, lower end, left end and right end of each connecting terminal in the charging port, when the first image data set is subjected to batch marking detection, QR code corner point detection can be performed on each first image in the first image data set. Based on the number of detected corner point pixel coordinates, it is possible to simply and accurately determine whether the marking detection result of the first image is qualified.

[0117] In a possible implementation, the step S22 of generating a corresponding key point information file according to the mark detection result and the storage order of the first image may include:

[0118] A corresponding key point information file is generated according to the pixel coordinates of a preset number of corner points and the storage order of the first image.

[0119] In this embodiment, if a first image detects pixel coordinates of a preset number of corner points, the mark detection result of the first image is determined to be qualified, and a key point information file corresponding to the first image can be generated based on the pixel coordinates of the preset number of corner points, and the key point information file can be stored in a third storage location according to the storage order of the first image.

[0120] For example, if the pixel coordinates of 12 corner points of 3 QR codes are detected, the pixels of the 12 corner points can be written in sequence as key point information into a txt file with the same name as the first image, and all txt files can be saved in the labels folder.

[0121] In this embodiment, after performing QR code corner point detection on each first image in the first image data set, if the mark detection result is qualified, the corresponding key point information file can be generated according to the pixel coordinates of a preset number of corner points and the storage order of the first image, so that each key point information file in the key point information set stores the key point information of the first image / second image in the corresponding storage order.

[0122] The following is an explanation of the key point marking method of the charging port of the present application with reference to a specific embodiment.

[0123] In a specific embodiment, a technician needs to train an object detection model that can detect and identify the key point positions of a charging port in an image. First, the key points of the charging port need to be annotated. The specific charging port key point annotation process is as follows:

[0124] In the first step, the technicians set up a robotic arm data acquisition path script and a camera image data acquisition script in the computing device, and overlap the corner points of the printed Aruco code of appropriate size with the top, leftmost, rightmost and bottom of the DC+, DC- and PE ports in the charging port, so that the three Aruco codes cover the key point positions of the charging port respectively.

[0125] In the second step, the technician inputs the image acquisition start signal to the computing device. After receiving the signal, the computing device runs the robot arm data acquisition path script and the camera image data acquisition script to control the robot arm to run according to the preset robot arm data acquisition path and preset movement speed. During the operation of the robot arm, the camera is controlled to collect the first image under different camera postures when the charging port is covered with the Aruco code according to the preset image acquisition time and image acquisition frame rate.

[0126] In the third step, the computing device saves each first image into the folder image_with_qr according to the order in which the images are generated, and obtains a first image data set.

[0127] In the fourth step, the technician removes the Aruco code covering the charging port and inputs the image acquisition start signal to the computing device again. After receiving the signal, the computing device runs the robotic arm data acquisition path script and the camera image data acquisition script again to control the robotic arm to run according to the preset robotic arm data acquisition path and preset movement speed. During the operation of the robotic arm, the camera is controlled to collect the second image at different camera postures when the charging port is not covered with the Aruco code according to the preset image acquisition time and image acquisition frame rate.

[0128] In the fifth step, the computing device saves each second image into the folder images according to the order in which the images are generated, and obtains a second image data set.

[0129] In the sixth step, the computing device performs labeled batch detection on the first images in the first image data set. For each first image, it is determined whether the pixel coordinates of the 12 corner points are detected in the first image; if the pixel coordinates of the 12 corner points are detected, the corresponding key point information files are generated according to the pixel coordinates of the 12 corner points and the storage order of the first image, and each key point information file is saved in the folder labels; if the pixel coordinates of the 12 corner points are not detected, the second image in the second image data set with the same storage order as the first image is deleted.

[0130] In the seventh step, the computing device generates a key point information set according to each key point information file, and generates an original image set according to the second images in the second image data set that correspond one-to-one to the storage order of each key point information file.

[0131] In the eighth step, the computing device generates a key point annotation file for the charging port based on the key point information set and the original image set, and completes the key point annotation of the charging port.

[0132] Figure 5 This is a schematic diagram of the structure of a computing device according to an embodiment of the present application. Figure 5 As shown, the computing device includes: an acquisition module 51, which is used to acquire a first image data set and a second image data set of a vehicle charging port, wherein the key point positions of the charging port in the first image data set are covered, and the key point positions of the charging port in the second image data set are not covered, and the images of the same storage order in the two image data sets are collected under the same camera posture; a processing module 52, which is used to mark batch detection of the first image data set, and generate a key point information set and a corresponding original image set based on the batch detection results and the second image data set, wherein each key point information file in the key point information set stores the key point information of the images of the corresponding storage order in the original image set; and a key point annotation file of the charging port is generated based on the key point information set and the original image set.

[0133] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0134] In a possible implementation, the acquisition module 51 may also be used to:

[0135] According to a preset data acquisition path of the robotic arm and a camera arranged at the end of the robotic arm, a first image of the charging port under different camera postures when the charging port is covered is obtained;

[0136] The first image is saved to a first storage location in the order in which the images are generated, so as to generate a first image data set when the key point position of the charging port is covered;

[0137] After detecting the key point position of the charging port and removing the cover, according to the preset robot arm data acquisition path and the camera set at the end of the robot arm, obtain the second image of the charging port under different camera postures when the charging port is not covered;

[0138] The second image is saved to a second storage location in the order in which the images are generated, so as to generate a second image data set when the key point position of the charging port is not covered.

[0139] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0140] In a possible implementation, the acquisition module 51 may also be used to:

[0141] Control the robotic arm to move to the starting point of the robotic arm data acquisition path, and control the robotic arm to run according to the robotic arm data acquisition path and the preset movement speed. During the operation of the robotic arm, control the camera to collect the first image / second image of the charging port under different camera postures according to the preset image acquisition time and image acquisition frame rate.

[0142] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0143] In a possible implementation, the processing module 52 may also be used to:

[0144] Performing labeled batch detection on a first image in the first image data set;

[0145] For each first image, determine whether the mark detection result of the first image is qualified; if the mark detection result is qualified, generate a corresponding key point information file according to the mark detection result and the storage order of the first image; if the mark detection result is unqualified, delete the second image in the second image data set that has the same storage order as the first image;

[0146] A key point information set is generated according to each key point information file, and an original image set is generated according to the second images in the second image data set that correspond one-to-one to the storage order of each key point information file.

[0147] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0148] In a possible implementation, the processing module 52 may also be used to:

[0149] Determine whether the first image detects pixel coordinates of a preset number of corner points, where the preset number is the number of corner points of the QR code covered by the charging port;

[0150] If the pixel coordinates of a preset number of corner points are detected, it is determined that the mark detection result of the first image is qualified;

[0151] If the pixel coordinates of the preset number of corner points are not detected, it is determined that the mark detection result of the first image is unqualified.

[0152] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0153] In a possible implementation, the processing module 52 may also be used to:

[0154] A corresponding key point information file is generated according to the pixel coordinates of a preset number of corner points and the storage order of the first image.

[0155] The computing device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0156] Figure 6 This is a schematic diagram of the structure of a computing device according to an embodiment of the present application. Figure 6 As shown, the computing device includes: a processor 601, and a memory 602 communicatively connected to the processor 601; the memory 602 stores computer execution instructions; the processor 601 executes the computer execution instructions stored in the memory 602 to implement the steps of the charging port key point marking method in the above-mentioned method embodiments.

[0157] In the above computing device, the memory 602 and the processor 601 are electrically connected directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines, such as through a bus connection. The memory 602 stores computer-executable instructions for implementing the data access control method, including at least one software function module that can be stored in the memory 602 in the form of software or firmware. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602.

[0158] The memory 602 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory 602 is used to store programs, and the processor 601 executes the programs after receiving the execution instruction. Furthermore, the software programs and modules in the above-mentioned memory 602 may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.

[0159] The processor 601 may be an integrated circuit chip having the ability to process signals. The processor 601 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps, and logic diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0160] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the steps of each method embodiment of the present application.

[0161] An embodiment of the present application further provides a computer program product, including a computer program, which implements the steps of each method embodiment of the present application when executed by a processor.

[0162] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0163] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0164] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0165] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0166] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

[0168] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for marking key points of a charging port, characterized in that: include: Acquire a first image data set and a second image data set of a charging port of a vehicle, wherein key point positions of the charging port in the first image data set are covered, and key point positions of the charging port in the second image data set are not covered, and images in the same storage order in the two image data sets are collected under the same camera posture; Performing a labeled batch detection on the first image data set, and generating a key point information set and a corresponding original image set according to the batch detection result and the second image data set, wherein each key point information file in the key point information set stores key point information of images in a corresponding storage order in the original image set; A key point annotation file of the charging port is generated according to the key point information set and the original image set.

2. The charging port key point marking method according to claim 1, characterized in that: The step of performing batch detection on the first image data set and generating a key point information set and a corresponding original image set according to the batch detection result and the second image data set includes: Performing labeled batch detection on the first image in the first image data set; For each first image, determine whether a mark detection result of the first image is qualified; if the mark detection result is qualified, generate a corresponding key point information file according to the mark detection result and the storage order of the first image; if the mark detection result is unqualified, delete the second image in the second image data set that has the same storage order as the first image; A key point information set is generated according to each of the key point information files, and an original image set is generated according to the second images in the second image data set that correspond one-to-one to the storage order of each key point information file.

3. The charging port key point marking method according to claim 2, characterized in that: The key point positions are the corner points of the upper end, lower end, left end, and right end of the connection terminal in the charging port, and the connection terminal includes a positive terminal, a negative terminal, and a protective grounding terminal; The key point positions of the charging port in the first image data set are covered with a two-dimensional code, and the corner point positions of the two-dimensional code coincide with the corner point positions of the connecting terminal, so that the two-dimensional code covers the key point positions of the charging port.

4. The charging port key point marking method according to claim 3, characterized in that: The determining whether the mark detection result of the first image is qualified includes: Determine whether the first image detects pixel coordinates of a preset number of corner points, where the preset number is the number of corner points of the QR code covered by the charging port; If the pixel coordinates of a preset number of corner points are detected, it is determined that the mark detection result of the first image is qualified; If the pixel coordinates of the preset number of corner points are not detected, it is determined that the mark detection result of the first image is unqualified.

5. The charging port key point marking method according to claim 4, characterized in that: The step of generating a corresponding key point information file according to the mark detection result and the storage order of the first image comprises: A corresponding key point information file is generated according to the pixel coordinates of the preset number of corner points and the storage order of the first image.

6. The method for marking key points of a charging port according to any one of claims 1 to 5, characterized in that: The step of acquiring a first image data set and a second image data set of a vehicle charging port includes: According to a preset data acquisition path of the robotic arm and a camera disposed at the end of the robotic arm, a first image of the charging port under different camera postures when the charging port is covered is acquired; The first image is saved to a first storage location in the order in which the images are generated, so as to generate a first image data set when the key point position of the charging port is covered; After detecting that the key point position of the charging port is uncovered, obtaining a second image of the charging port in different camera postures when the charging port is not covered according to a preset robotic arm data acquisition path and a camera disposed at the end of the robotic arm; The second image is saved to a second storage location in the order in which the images are generated, so as to generate a second image data set when the key point position of the charging port is not covered.

7. The charging port key point marking method according to claim 6, characterized in that: According to a preset data acquisition path of the robotic arm and a camera disposed at the end of the robotic arm, a first image / a second image is acquired, including: Control the robotic arm to move to the starting point of the robotic arm data acquisition path, and control the robotic arm to operate according to the robotic arm data acquisition path and a preset movement speed, and during the operation of the robotic arm, control the camera to acquire the first image / second image of the charging port in different camera postures according to a preset image acquisition time and image acquisition frame rate.

8. A computing device, characterized in that include: an acquisition module, configured to acquire a first image data set and a second image data set of a vehicle charging port, wherein key point positions of the charging port in the first image data set are covered, and key point positions of the charging port in the second image data set are not covered, and images in the same storage order in the two image data sets are collected under the same camera posture; A processing module, configured to perform a batch detection on the first image data set, and generate a key point information set and a corresponding original image set according to the batch detection result and the second image data set, wherein each key point information file in the key point information set stores key point information of images in a corresponding storage order in the original image set; A key point annotation file of the charging port is generated according to the key point information set and the original image set.

9. A computing device, characterized in that include: A processor, and a memory communicatively connected to the processor; The memory is used to store computer-executable instructions; The processor is used to execute the computer-executable instructions stored in the memory, so that the processor executes the charging port key point marking method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the charging port key point marking method according to any one of claims 1 to 7.

Images