Charging equipment inclination state detection method and device, medium and program product

Abstract

The invention provides a charging equipment inclination state detection method and device, a medium and a program product, and the method comprises the steps: determining a first region according to all first bounding boxes of each initial image; determining a second area according to all second bounding boxes of the current image; comparing the second area with each first area, and screening out the first area matched with the second area; and determining the matched first region as a reference, screening out a second bounding box which does not meet a preset screening condition from the second region, identifying the second bounding box as an abnormal bounding box, determining a relative inclination angle of the abnormal bounding box, and when the relative inclination angle is greater than or equal to an inclination angle threshold value, determining that the abnormal bounding box does not meet the preset screening condition. And identifying that the charging equipment corresponding to the abnormal bounding box is inclined. According to the method, interference caused by image distortion is avoided, and the detection precision of the inclination of the charging equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, medium, and program product for detecting the tilt state of a charging device. Background Art

[0002] In the prior art, the detection of the tipping of charging equipment mainly adopts the following two methods: First, the image data set of the tipping equipment is used to train the deep learning model, so that it can directly identify the charging pile that looks like it is tipping on the way. Second, based on the deep learning model, the side profile of the charging pile is identified, the slope of the side straight line is obtained, and then the inclination angle of the charging pile is determined, and then the tilt state is determined by the inclination angle.

[0003] However, in reality, the collected images will be distorted, and image distortion will cause the objects in the image to be distorted. Image distortion exists objectively, so the vertical charging pile may appear tilted in the picture. Therefore, for the above two methods, there will be misjudgment of tilt. For example, the tilt angle determined by the straight line edge in method 2 is not the actual tilt angle, which leads to misjudgment of tilt and affects the accuracy of recognition. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a method, device, medium, and program product for detecting the tilt state of a charging device. The method avoids interference caused by image distortion and improves the detection accuracy of the tilt of the charging device.

[0005] One aspect of the present invention provides a method for detecting a tilt state of a charging device, the method comprising:

[0006] Acquire an initial image set of the target station, the initial image set comprising a plurality of initial images, and identify a first bounding box of each charging device in each of the initial images;

[0007] Acquire a current image of the target station, and identify a second bounding box of each charging device in the current image;

[0008] Determine a first region according to all the first bounding boxes of each initial image, the first region being a union of all the first bounding boxes of the corresponding initial image; determine a second region according to the second bounding box, the second region being a union of all the second bounding boxes of the current image;

[0009] Determine whether the second area matches each of the first areas; determine the matched first area as a reference, filter out second bounding boxes that do not meet a preset filtering condition from the second area and identify them as abnormal bounding boxes, determine a relative tilt angle of the abnormal bounding box, and when the relative tilt angle is greater than or equal to a tilt angle threshold, identify that the charging device corresponding to the abnormal bounding box is tilted;

[0010] The relative tilt angle is the tilt angle of the abnormal bounding box relative to the matched first bounding box.

[0011] In one embodiment of the present invention, comparing the second region with each of the first regions to determine whether the second region matches each of the first regions includes:

[0012] Determine a third area according to an overlapping area between the second area and each of the first areas, wherein each of the third areas is an intersection of the second area and the corresponding first area;

[0013] Determine a fourth region according to a set of the second region and each of the first regions, wherein each of the fourth regions is a union of the second region and the corresponding first region;

[0014] For each of the first regions, determining the first ratio according to the ratio of the third region to the fourth region;

[0015] If the first ratio is greater than or equal to a ratio threshold, it is determined that the first region matches the second region.

[0016] In one embodiment of the present invention, determining the matched first region as a reference, and screening out second bounding boxes that do not meet a preset screening condition from the second region and identifying them as abnormal bounding boxes includes:

[0017] The matched first region is determined as a reference, and a difference set between the second region and the first region is calculated to obtain a fifth region, wherein the fifth region includes a plurality of disconnected sub-regions;

[0018] According to the preset screening condition, screening out a target sub-region from the multiple sub-regions in the fifth region, and determining a center point of the target sub-region;

[0019] According to the coordinates of the center point, a second bounding box and a first bounding box closest to the center point are respectively identified from the second area and the matched first area, the second bounding box is identified as the abnormal bounding box, and the first bounding box is a bounding box that matches the abnormal bounding box.

[0020] In one embodiment of the present invention, the preset screening condition includes a first condition, the first condition is set to be greater than or equal to a pixel area threshold, and the pixel area threshold is set according to the image size ratio of the current image;

[0021] The area of ​​each sub-region is determined, and if the area of ​​a sub-region does not meet the first condition, the sub-region is screened out.

[0022] In one embodiment of the present invention, the preset screening condition includes a second condition, the second condition is set to be greater than or equal to a size threshold, and the size threshold is set according to the ratio of the long side to the short side of the bounding box;

[0023] The size ratio of the long side to the short side of each sub-region is determined, and if the size ratio of a sub-region does not meet the second condition, the sub-region is screened out.

[0024] In one embodiment of the present invention, calculating the difference between the second region and the first region to obtain the fifth region includes:

[0025] If the second area and the first area have completely overlapping bounding boxes, deleting the completely overlapping bounding boxes;

[0026] If the second area partially overlaps with a boundary box of the first area, deleting the overlapping area and retaining the non-overlapping portion of the boundary box of the second area;

[0027] If there is a bounding box in the second region that does not overlap with the first region at all, the bounding box of the second region is retained.

[0028] In one embodiment of the present invention, in the coordinate system of the initial image, the relative tilt angle is obtained according to the difference between the tilt angle of the abnormal bounding box and the tilt angle of the matching first bounding box.

[0029] In one embodiment of the present invention, a first bounding box of each charging device in each of the initial images is identified by a deep learning model, and / or a second bounding box of each charging device in the current image is identified;

[0030] Before the step of identifying the first bounding box of each charging device in each of the initial images and / or identifying the second bounding box of each charging device in the current image by using the deep learning model, the method further includes:

[0031] The boundary box adopts a rotated rectangular box;

[0032] Acquire a charging field image dataset, and annotate a bounding box for a charging device in each of the charging field images, and adjust the bounding box angle so that the boundary of the bounding box fits the outer contour of the charging device and completely includes the charging device, so as to obtain a training set;

[0033] The initial deep learning model is trained using the training set.

[0034] In one embodiment of the present invention, the initial image is collected when the target station is initially built, or after an offline or cloud-based manual inspection confirms that the charging equipment is not damaged.

[0035] Another aspect of the present invention further provides a device for detecting the tilt state of a charging device, comprising:

[0036] a bounding box identification module, configured to obtain an initial image set of a target station, the initial image set comprising a plurality of initial images, identify a first bounding box of each charging device in each of the initial images; and obtain a current image of the target station, identify a second bounding box of each charging device in the current image;

[0037] a bounding box set module, configured to determine a first region according to all the first bounding boxes of each initial image, the first region being the union of all the first bounding boxes of the corresponding initial image; and determine a second region according to the second bounding box, the second region being the union of all the second bounding boxes of the current image;

[0038] A matching module is used to determine whether the second area matches each of the first areas, and identify the first area that matches the second area, indicating that the current image corresponding to the second area and the initial image corresponding to the matched first area are acquired from the same camera.

[0039] a screening module, configured to determine the matched first area as a reference, screen out second bounding boxes that do not meet a preset screening condition from the second area and identify them as abnormal bounding boxes, and determine a relative tilt angle of the abnormal bounding box;

[0040] a tilt determination module, configured to identify the tilt of the charging device corresponding to the abnormal boundary box when the relative tilt angle is greater than or equal to a tilt angle threshold;

[0041] The relative tilt angle is the tilt angle of the abnormal bounding box relative to the matched first bounding box.

[0042] In another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned method for detecting the tilt state of a charging device when the computer program is executed by a processor.

[0043] Another aspect of the present invention provides a computer program product, including a computer program, which implements the steps of the above-mentioned charging device tilt state detection method when executed by a processor.

[0044] It can be seen from the above scheme that the advantages of the present invention are:

[0045] The present invention discloses a method for detecting the tilt state of a charging device, which obtains a first area by combining all the first bounding boxes of each initial image; combines all the second bounding boxes of the current target station to determine a second area; and then compares the second area with each of the first areas by the intersection-and-union ratio (i.e., the first ratio) of the first area and the second area. If it is determined that the second area matches a first area, it is identified that the current image corresponding to the second area and the initial image corresponding to the matched first area are acquired from the same camera. Then, the matched first area is determined as a reference, and the second bounding boxes that do not meet the preset screening conditions are screened out from the second area and identified as abnormal bounding boxes, and the relative tilt angle of the abnormal bounding box is determined. In this method, by matching the images acquired by the same camera, the image distortion effects of the two images will be exactly the same. By comparing the charging devices of the two images, the tilt state can be accurately determined to prevent the images acquired by different cameras from misjudging the tilt curve caused by distortion as tilt due to different distortion effects, thereby eliminating the visual tilt interference caused by image distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The image effect diagram caused by image distortion is shown;

[0047] Figure 2 The overall flow chart of the method for detecting the tilt state of a charging device is shown;

[0048] FIG. 3( a ) shows an effect diagram of an initial image identification boundary box according to an embodiment;

[0049] FIG3( b ) shows an effect diagram of a current image identification boundary box according to an embodiment;

[0050] Figure 4 The effect diagram of the union of all bounding boxes in an image is shown;

[0051] Figure 5 Shows Figure 2 Detailed flow diagram of step S3;

[0052] Figure 6 The effect diagram of the fifth area is obtained by performing a difference operation between the second area determined by FIG. 3(b) and the first area determined by FIG. 3(a);

[0053] FIG. 7( a ) shows an effect diagram of an initial image identification boundary box according to another embodiment;

[0054] FIG. 7( b ) shows an effect diagram of a current image identification boundary box according to another embodiment;

[0055] Figure 8 Schematic diagram for bounding box tilt angle calculation;

[0056] Fig. 9 A schematic diagram of the structure of a device for detecting the tilt state of a charging device is shown;

[0057] 10, 20: area;

[0058] 301, 302, 30: bounding box;

[0059] 40: Union area;

[0060] 50: target sub-region;

[0061] 600: Charging device tilt state detection device;

[0062] 610: Bounding box identification module;

[0063] 620: Bounding box collection module;

[0064] 630: matching module;

[0065] 640: screening module;

[0066] 650: Tilt determination module. DETAILED DESCRIPTION

[0067] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0068] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0069] In the prior art, based on the deep learning model, the side profile of the charging pile is identified to obtain the slope of the side straight line, and then the inclination angle of the charging pile is determined to determine the tilt state. However, due to the existence of image distortion, the image distortion causes the objects in the image to be distorted, which leads to the method misjudging the tilt state of the charging device. For example, Figure 1As shown in , it is obvious that the columns in area 10 are all arc-shaped and appear to be tilted; while in reality, the tilted charging piles in area 20 that are a little further away all look upright. Image distortion is determined by the intrinsic parameters of the camera, and subtle differences in the manufacturing process make the intrinsic parameters of different cameras different. If you want to remove the distortion, you need to obtain the intrinsic parameters of the camera that took the picture, which is unrealistic for image data sets collected by several cameras. Therefore, in order to overcome the above technical problems, the present invention improves the prior art, identifies images collected by the same camera for comparison, so as to avoid errors caused by different distortions of images from different cameras, so as to accurately realize the tilt detection of the charging device.

[0070] Please refer to Figure 2 As shown in Figure 2 The overall flow chart of the method for detecting the tilt state of a charging device disclosed in an embodiment of the present invention is shown. The method for detecting the tilt state of a charging device specifically comprises the following steps:

[0071] Step S1: Acquire an initial image set of a target station, the initial image set comprising a plurality of initial images, identify a first bounding box of each charging device in each of the initial images, acquire a current image of the target station, and identify a second bounding box of each charging device in the current image.

[0072] In this embodiment, a number of cameras and a monitoring device are arranged in the charging station. One monitoring device corresponds to one station. A single monitoring device is connected to multiple cameras. The collected monitoring images are stored locally or in the cloud according to the specified file path and file name format. The file path information contains information such as shooting time and station ID. Since the same station ID often corresponds to multiple cameras in actual applications, this embodiment is expected to distinguish different cameras under the same ID based on the union area of ​​the bounding box of each image.

[0073] Specifically, in this embodiment, for the target station that needs to monitor the external damage of the charging equipment, it is first necessary to collect several initial images when the station is initially built, or when the charging equipment is confirmed to be free of damage after offline or cloud-based manual inspections, to obtain an initial image set, which characterizes the situation where the charging equipment in the station has no external damage. These initial images may be collected by different cameras. For each initial image, a deep learning model is used to identify the first bounding box of each charging device in the image. Similarly, for the current image of the target station obtained, a deep learning model is also used to identify the second bounding box of each charging device in the current image.

[0074] It should be noted that, in this embodiment, the first bounding box and the second bounding box are only used to distinguish the initial image from the current image, and the specific marking method of the bounding boxes is the same, rather than two different types of bounding boxes.

[0075] In this embodiment, the initial image set is used as a reference, and these images are images captured when the charging device has no external damage. The captured current image is matched with each of the initial images to identify images captured by the same camera in the same state. By comparing the matched images, the tilt is detected to avoid misjudgment of tilt due to different distortions of different cameras.

[0076] In addition, in this embodiment, before using the deep learning model for bounding box recognition, the deep learning model needs to be trained in advance. In this embodiment, a charging field image dataset is obtained, and a bounding box is annotated for the charging device in each of the charging field images. The charging device usually exists in the form of a charging pile. For example, the four corners of the pile body and the obvious feature lines in the contour of the charging pile (such as straight lines, curves, etc.) can be selected. The intersection of these feature lines is used as the key point to mark the bounding box. These images of the charging field image dataset can come from different scenes, angles and lighting conditions to ensure the generalization ability of the model.

[0077] In this embodiment, the bounding box adopts a rotated rectangular box, and the angle of the bounding box is adjusted during annotation so that the boundary of the bounding box fits the outer contour of the charging device and completely includes the charging device to obtain a training set, and then the training set trains the initial deep learning model. Finally, the bounding boxes of the initial image and the current image are marked using the trained deep learning model. As shown in Figure 3(a) and Figure 3(b), Figure 3(a) is a rendering of the bounding box marked for the initial image, and Figure 3(b) is a rendering of the bounding box marked for the current image, and the figure shows several marked bounding boxes 30.

[0078] In one embodiment, the deep learning model may be an algorithm model such as YOLO series v8, v10, v11, or other neural network models that support OBB (Oriented Bounding Box).

[0079] Step S2: determine a first region based on all the first bounding boxes of each initial image, where the first region is the union of all the first bounding boxes of the corresponding initial image; and determine a second region based on all the second bounding boxes, where the second region is the union of all the second bounding boxes of the current image.

[0080] In this embodiment, the set of all the first bounding boxes of each initial image is counted, and a union operation is performed on all the first bounding boxes of the initial image to determine the first region. Similarly, the set of all the second bounding boxes of the current image is counted, and a union operation is performed on all the second bounding boxes of the initial image to determine the second region. In this embodiment, the first region and the second region are both the union regions of the bounding box sets. For example, for the current image, it contains n second bounding boxes, and all the second bounding boxes B j The union of is represented as That is the second area. Figure 4 As shown in Figure 4 The figure shows the union of all bounding boxes in an image, with the union region 40 being displayed.

[0081] Step S3, judging the second area and each of the first areas, if it is determined that the second area matches one of the first areas, identifying that the current image corresponding to the second area and the initial image corresponding to the matching first area are acquired from the same camera.

[0082] In this embodiment, the second region is compared with each of the first regions to determine whether the second region matches each of the first regions. Figure 5 As shown in Figure 5 A detailed flow chart of step S3 is shown. First, in step S31, for each first area, determine the overlapping area of ​​the second area and the first area, that is, perform an intersection operation on the second area and the first area to obtain a third area. At the same time, in step S32, count the sets of the second area and the first area, perform a union operation on the second area and the first area, and determine the fourth area. Then, in step S33, determine a first ratio based on the ratio of the third area to the fourth area, and the first ratio is the ratio of the third area to the fourth area. Furthermore, if the first ratio is greater than or equal to a ratio threshold, it is determined that the first area matches the second area.

[0083] For example, for the current image, it contains n second bounding boxes, all the second bounding boxes B j The union of is represented as That is, the second area; for an initial image, it contains m first bounding boxes, and the union of all first bounding boxes is expressed as That is the first area. Then the third area is expressed as The fourth region is represented by The first ratio represents the IOU is expressed as:

[0084] By comparing the second area with each of the first areas in the above manner, a first area matching the second area is identified, and the current image corresponding to the second area and the initial image corresponding to the matching first area are acquired by the same camera. As shown in FIG3(a) and FIG3(b), the image distortion effects of the two images are exactly the same. By comparing the charging devices of the two images, the tilt state can be accurately determined to prevent the tilt curve caused by the distortion from being misjudged as tilt due to the different distortion effects of the images acquired by different cameras, thereby eliminating the visual tilt interference caused by image distortion.

[0085] Step S4, determining the matched first area as a reference, screening out a second bounding box that does not meet a preset screening condition from the second area and identifying it as an abnormal bounding box, determining a relative tilt angle of the abnormal bounding box, and when the relative tilt angle is greater than or equal to a tilt angle threshold, identifying that the charging device corresponding to the abnormal bounding box is tilted; wherein the relative tilt angle is a tilt angle of the abnormal bounding box relative to the matched first bounding box.

[0086] In this embodiment, in the matching process of step S3, the first area of ​​the matched initial image is used as a reference, and the second bounding box that does not meet the preset screening condition is screened out from the second area and identified as an abnormal bounding box, and then the relative tilt angle of the abnormal bounding box is determined. Here, the relative tilt angle is the tilt angle of the abnormal bounding box relative to the matched first bounding box, that is, the current image is compared with the same charging device identified in the initial image, and the tilt angle of the charging device in the current image relative to the charging device in the initial image, that is, the relative tilt angle, is determined. If the relative tilt angle exceeds a tilt angle threshold, such as 10°, it is determined that the charging device in the current image is tilted.

[0087] Specifically, in one embodiment, the matched first region is determined as a reference, and a difference operation is performed on the second region and the first region to obtain a fifth region. During the difference operation, if the second region and the first region have completely overlapping bounding boxes, the completely overlapping bounding boxes are deleted; if the second region partially overlaps with a bounding box of the first region, the overlapping region is deleted, and the non-overlapping portion of the bounding box of the second region is retained; if the second region has a bounding box that does not overlap with the first region at all, the bounding box of the second region is retained, so as to obtain the fifth region, and the first region is composed of multiple non-connected sub-regions, which are the retained second bounding boxes, the second bounding boxes displayed in the non-overlapping parts, etc. In this way, through the difference between the first region and the first region, the bounding boxes in the fifth region are some bounding boxes in the second region that have a certain deviation from the bounding boxes of the first region. Figure 6The effect diagram of the fifth area is obtained by performing a difference operation between the second area determined in FIG. 3( b ) and the first area determined in FIG. 3( a ).

[0088] Then, according to the preset screening conditions, the target sub-region 50 is screened out from the multiple sub-regions in the fifth region, and the center point of the target sub-region is determined. In one embodiment, the screening conditions are preset by multiple thresholds to exclude occasional false detections and small targets (small targets mean that they are too far away from the camera, or are falsely detected objects nearby). These thresholds include but are not limited to the detection frame pixel area threshold, the detection frame size threshold, etc.

[0089] Specifically, in one embodiment, the preset screening condition includes a first condition, the first condition is set to be greater than or equal to a pixel area threshold, and the pixel area threshold is set according to the image size ratio of the current image; the area of ​​each sub-region is determined, and if the area of ​​a sub-region does not meet the first condition, the sub-region is filtered out, and the sub-region corresponds to a small target that is misdetected nearby or a detected target that is far away. For example, if the threshold ratio is set to 0.016 and the image size is 1920x1080, the pixel area threshold is 1920*0.016*1080*0.016=530, that is, targets with a pixel area less than 530 are filtered out.

[0090] In one embodiment, the preset screening condition includes a second condition, and the second condition is set to be greater than or equal to a size threshold, and the size threshold is set according to the ratio of the long side to the short side of the standard bounding box. Determine the size ratio of the long side to the short side of each sub-area, and if the size ratio of a sub-area does not meet the second condition, the sub-area is filtered out. When the size threshold is passed, small targets smaller than the size threshold are filtered out, and the tilt angle is not determined. For detected targets that are far away or obstructed, this can avoid angle misjudgment and filter out distant or obstructed targets. As shown in Figure 7 (a) and Figure 7 (b), Figure 7 (a) is the initial image and Figure 7 (b) is the current image, both of which are collected by the same camera. The current image of Figure 7 (b) detects two charging piles in the distance (at the upper left corner of the image) and marks the bounding box 302. However, in the initial image of Figure 7 (a), the two charging piles at the upper left corner of the image are not detected due to being blocked by the vehicle, and the bounding box is not marked. Then, when the difference operation is performed between the first area and the second area of ​​the two images, the two bounding boxes at the upper left corner of the image will exist in the fifth area. At this time, the size threshold or the area threshold can be used to filter out the two bounding boxes, and the inclination change is not determined, so as to prevent the two bounding boxes 302 from being inconsistent with each other. Figure 6 Matching the bounding box 301 identified in (a) may result in misjudgment of tilt.

[0091] Thus, through the above process, after the target sub-region is screened out, the center point of the target sub-region is determined. Then, according to the coordinates of the center point, the second bounding box and the first bounding box closest to the center point are identified from the second region and the first matched region, respectively. Then, the second bounding box will be identified as the abnormal bounding box, and the first bounding box is a bounding box that matches the abnormal bounding box. Figure 6 As shown in , after the fifth area is screened, a target sub-area 50 is screened out, and the target sub-area 50 corresponds to the bounding boxes 301 and 302 marked in green in FIG. 3( a ) and FIG. 3( b ).

[0092] After determining the abnormal bounding box 301 and the corresponding first bounding box 302, the tilt angle θ1 of the abnormal bounding box 301 and the tilt angle γ2 of the first bounding box 302 can be determined in the coordinate system of the initial image, and then the relative tilt angle of the abnormal bounding box is obtained as θ1-θ2. If the relative tilt angle is greater than or equal to the tilt angle threshold, it will be identified that the charging device corresponding to the abnormal bounding box is tilted.

[0093] In addition, in one embodiment, the calculation of the tilt angle of the bounding box is performed by Figure 8 For example, the bounding box shown in the figure is the four corner points of the bounding box [p1, p2, p3, p4]. If any three adjacent points are taken, the lengths of the two adjacent sides can be compared to determine the long side. Then the slope of the long side is taken to obtain the inclination angle. For example, if the three points p1, p2, and p3 are taken, since the distance between the two points p2p3 is greater than the distance between the two points p1p2, p2p3 is identified as the long side. Therefore, the inclination angle can be obtained from the slope of the long side p2p3.

[0094] The method for detecting the tilt state of a charging device disclosed in the present invention can be applied to a station or to the cloud. For example, the intelligent algorithm box (AIbox) applied to the station can detect in real time by calling the camera of the station to determine whether there is an abnormally tilted charging device in the current target charging field. When an abnormality is detected, an early warning is triggered, and the early warning information is pushed to the cloud, and the cloud pushes it to the administrator. In stations without intelligent algorithm boxes, the cameras of the station can be called in time periods to capture pictures, and the pictures can be uploaded to the cloud and then recognized by the model. The cloud has a higher computing power and uses a high-precision recognition model; the intelligent algorithm box is measured at the edge, and the device computing power is lower, so the recognition model used can be a lightweight version of the model.

[0095] In summary, the method for detecting the tilt state of a charging device disclosed in the present invention obtains a first area by combining all the first bounding boxes of each initial image; determines a second area by combining all the second bounding boxes of the current target station; and then compares the second area with each of the first areas by the intersection-and-union ratio (i.e., the first ratio) of the first area and the second area. If it is determined that the second area matches a first area, it is identified that the current image corresponding to the second area and the initial image corresponding to the matched first area are acquired from the same camera. Then, the matched first area is determined as a reference, and the second bounding boxes that do not meet the preset screening conditions are screened out from the second area and identified as abnormal bounding boxes, and the relative tilt angle of the abnormal bounding box is determined. In this method, by matching the images acquired by the same camera, the image distortion effects of the two images will be exactly the same. By comparing the charging devices of the two images, the tilt state can be accurately determined to prevent the images acquired by different cameras from misjudging the tilt curve caused by distortion as tilt due to different distortion effects, thereby eliminating the visual tilt interference caused by image distortion. In addition, the present invention also performs a difference operation between the first area and the second area, and sets filtering conditions in combination with area, size, etc., to filter out misidentified or small-sized boundary boxes in the distance, so as to accurately identify the tilted charging device, further improving the accuracy of tilt judgment.

[0096] Correspondingly, corresponding to the above-mentioned charging device tilt state detection method, the present invention also provides a charging device tilt state detection device. Fig. 9 As shown, Fig. 9 The schematic diagram of the structure of the device for detecting the tilt state of the charging device is shown. The device for detecting the tilt state of the charging device 600 at least comprises:

[0097] The bounding box identification module 610 is used to obtain an initial image set of a target station, the initial image set including a plurality of initial images, identify a first bounding box of each charging device in each of the initial images; and obtain a current image of the target station, and identify a second bounding box of each charging device in the current image.

[0098] The bounding box set module 620 is used to determine a first area according to all the first bounding boxes of each initial image, and the first area is the union of all the first bounding boxes of the corresponding initial image; determine a second area according to the second bounding box, and the second area is the union of all the second bounding boxes of the current image.

[0099] The matching module 630 is used to determine whether the second area matches each of the first areas, and identify the first area that matches the second area, indicating that the current image corresponding to the second area and the initial image corresponding to the matched first area are acquired from the same camera.

[0100] The screening module 640 is used to determine the matched first area as a reference, screen out second bounding boxes that do not meet the preset screening condition from the second area and identify them as abnormal bounding boxes, and determine the relative tilt angle of the abnormal bounding box.

[0101] The tilt determination module 650 is used to identify that the charging device corresponding to the abnormal boundary box is tilted when the relative tilt angle is greater than or equal to a tilt angle threshold, wherein the relative tilt angle is the tilt angle of the abnormal boundary box relative to the matching first boundary box.

[0102] In addition, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the charging device tilt state detection device can refer to the corresponding process in the aforementioned charging device tilt state detection method embodiment, and will not be repeated here.

[0103] In addition, the present invention also proposes an electronic device in another embodiment, comprising: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the above-mentioned charging device tilt state detection method.

[0104] In another embodiment, the present invention further provides a storage medium for storing a method for executing the above Figure 2 , Figure 5A computer program for detecting the tilt state of any of the charging devices shown. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0105] In addition, the present invention also provides a computer program product in one embodiment, the computer program product includes a computer program, the computer program can be stored in a readable storage medium, and when the computer program is executed by a processor, the computer can perform the above Figure 2 , Figure 5 Any charging device tilt state detection method shown.

[0106] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A method for detecting the tilt state of a charging device, characterized in that: Include: Acquire an initial image set of the target station, the initial image set comprising a plurality of initial images, and identify a first bounding box of each charging device in each of the initial images; Acquire a current image of the target station, and identify a second bounding box of each charging device in the current image; Determine a first region according to all the first bounding boxes of each initial image, where the first region is a union of all the first bounding boxes of the corresponding initial image; Determine a second region according to the second bounding box, where the second region is a union of all second bounding boxes of the current image; Determine whether the second area matches each of the first areas; determine the matched first area as a reference, filter out second bounding boxes that do not meet a preset filtering condition from the second area and identify them as abnormal bounding boxes, determine a relative tilt angle of the abnormal bounding box, and when the relative tilt angle is greater than or equal to a tilt angle threshold, identify that the charging device corresponding to the abnormal bounding box is tilted; The relative tilt angle is the tilt angle of the abnormal bounding box relative to the matched first bounding box.

2. The method according to claim 1, characterized in that Determining whether the second region matches each of the first regions includes: Determine a third area according to the second area and each of the first areas, wherein each of the third areas is an intersection of the second area and the corresponding first area; Determine a fourth area according to the second area and each of the first areas, wherein each of the fourth areas is a union of the second area and the corresponding first area; For each of the first regions, determining a first ratio according to a ratio of the third region to the fourth region; If the first ratio is greater than or equal to a ratio threshold, it is determined that the first region matches the second region.

3. The method according to claim 1, characterized in that Determining the matched first region as a reference, and screening out second bounding boxes that do not meet a preset screening condition from the second region as abnormal bounding boxes, comprises: The matched first region is determined as a reference, and a difference set between the second region and the first region is calculated to obtain a fifth region, wherein the fifth region includes a plurality of disconnected sub-regions; According to the preset screening condition, screening out a target sub-region from the multiple sub-regions in the fifth region, and determining a center point of the target sub-region; According to the coordinates of the center point, a second bounding box and a first bounding box closest to the center point are respectively identified from the second area and the matched first area, the second bounding box is identified as the abnormal bounding box, and the first bounding box is a bounding box that matches the abnormal bounding box.

4. The method according to claim 3, characterized in that The preset screening condition includes a first condition, the first condition is set to be greater than or equal to a pixel area threshold, and the pixel area threshold is set according to the image size ratio of the current image; The area of ​​each sub-region is determined, and if the area of ​​a sub-region does not meet the first condition, the sub-region is screened out.

5. The method according to claim 3, characterized in that: The preset screening condition includes a second condition, the second condition is set to be greater than or equal to a size threshold, and the size threshold is set according to the ratio of the long side to the short side of the bounding box; The size ratio of the long side to the short side of each sub-region is determined, and if the size ratio of a sub-region does not meet the second condition, the sub-region is screened out.

6. The method according to claim 3, characterized in that The step of calculating the difference between the second region and the first region to obtain a fifth region includes: If the second area and the first area have completely overlapping bounding boxes, deleting the completely overlapping bounding boxes; If the second area partially overlaps with a boundary box of the first area, deleting the overlapping area and retaining the non-overlapping portion of the boundary box of the second area; If there is a bounding box in the second region that does not overlap with the first region at all, the bounding box of the second region is retained.

7. The method according to claim 3, characterized in that In the coordinate system of the initial image, the relative tilt angle is obtained according to the difference between the tilt angle of the abnormal bounding box and the tilt angle of the matched first bounding box.

8. The method according to claim 1, characterized in that Identifying a first bounding box of each charging device in each of the initial images and / or identifying a second bounding box of each charging device in the current image by a deep learning model; Before the step of identifying the first bounding box of each charging device in each of the initial images and / or identifying the second bounding box of each charging device in the current image by using the deep learning model, the method further includes: The boundary box adopts a rotated rectangular box; Acquire a charging field image dataset, and annotate a bounding box for a charging device in each of the charging field images, and adjust the bounding box angle so that the boundary of the bounding box fits the outer contour of the charging device and completely includes the charging device, so as to obtain a training set; The initial deep learning model is trained using the training set.

9. The method according to claim 1, characterized in that: The initial image is collected when the target station is first built, or when an offline or cloud-based manual inspection confirms that the charging equipment is not damaged.

10. A device for detecting the tilt state of a charging device, characterized in that: Include: a bounding box identification module, configured to obtain an initial image set of a target station, the initial image set comprising a plurality of initial images, identify a first bounding box of each charging device in each of the initial images; and obtain a current image of the target station, identify a second bounding box of each charging device in the current image; a bounding box set module, configured to determine a first region according to all the first bounding boxes of each initial image, the first region being a union of all the first bounding boxes of the corresponding initial image; Determine a second region according to the second bounding box, where the second region is a union of all second bounding boxes of the current image; a matching module, configured to determine whether the second region matches each of the first regions, and identify the first regions that match the second regions; a screening module, configured to determine the matched first area as a reference, screen out second bounding boxes that do not meet a preset screening condition from the second area and identify them as abnormal bounding boxes, and determine a relative tilt angle of the abnormal bounding box; a tilt determination module, configured to identify the tilt of the charging device corresponding to the abnormal boundary box when the relative tilt angle is greater than or equal to a tilt angle threshold; The relative tilt angle is the tilt angle of the abnormal bounding box relative to the matched first bounding box.

11. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images