Unstructured data processing method, device, equipment and storage medium of smart pole

Abstract

The present invention discloses a method, device, equipment and storage medium for processing unstructured data of a smart pole, and relates to the technical field of smart multifunctional pole towers, including: establishing an adjacent visual vector relationship between a local smart pole and a target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole; obtaining a first photo image file of the target smart pole determined according to the adjacent visual vector relationship before construction, and marking the area to be constructed in the first photo image file; according to the adjacent visual vector relationship, using the local smart pole to shoot the target smart pole after construction to obtain a second photo image file; comparing the first photo image file and the second photo image file, as well as the area to be constructed and its corresponding area in the second photo image file, to determine the construction result of the target smart pole. The present invention avoids the problem that images taken at random cannot be matched and analyzed, and the unstructured data of the image is damaged.

Description

Technical Field

[0001] The present invention relates to the technical field of smart multifunctional pole towers, and in particular to a method, device, equipment and storage medium for processing unstructured data of a smart pole. Background Art

[0002] Unstructured data is data with irregular or incomplete structure, no predefined data model, and is not convenient to be represented by a two-dimensional logical table in a database. It includes office documents of all formats, text, pictures, HTML, various reports, images, and audio / video information, etc.

[0003] In the field of smart poles, unstructured data mainly includes the data of photographed files. During the construction phase of the smart pole project, construction personnel use the smart pole management APP to take photos of the completed smart poles and upload them to the management platform; during the operation and maintenance phase of the smart poles, when the smart devices are incrementally installed, disassembled or routinely inspected on the completed smart poles, the staff will continue to take photos and upload them to the management platform after the specific operation is completed; due to the large number of photo files, whether the smart poles are installed, disassembled or inspected depends on personal understanding, so administrators often use spot checks or post-inspection methods to analyze the file corresponding to the on-site situation of the smart poles.

[0004] However, in the existing smart pole management system, the collection of photo file data is affected by subjective factors and the update timeliness is poor; manual shooting from random perspectives is not convenient, the matching degree of the previous and next versions of the image is low, and the integrity of unstructured data is low, which is not convenient for automatic analysis.

[0005] In addition, the existing operating mode is not compatible with the need for automatic analysis and review of unstructured data of photo files during operation and maintenance; manual analysis and review of unstructured data of photos has a large workload, high error rate, and incomplete coverage; especially the integration degree of manually analyzed unstructured data and existing structured data is low and the reusability is poor. Summary of the invention

[0006] In view of the defects existing in the prior art, the first aspect of the present invention provides a method for processing unstructured data of a smart pole, which collects photo images of target smart poles with the same perspective, target and specification, thereby avoiding the problem that randomly photographed images cannot be matched and analyzed, and the unstructured data of the images is damaged.

[0007] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0008] A method for processing unstructured data of a smart pole, the method comprising the following steps:

[0009] Establishing an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters, and shooting conditions between the local smart pole and the target smart pole;

[0010] Acquire a first photo image file of the target smart pole determined according to the adjacent visible vector relationship before construction, and mark the area to be constructed in the first photo image file;

[0011] According to the adjacent visible vector relationship, using the local smart pole to photograph the target smart pole after construction to obtain a second photo image file;

[0012] The first photo image file and the second photo image file, as well as the area to be constructed and its corresponding area in the second photo image file are compared to determine the construction result of the target smart pole.

[0013] In some embodiments, the establishing of the adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole, includes:

[0014] Establish a corresponding mapping relationship based on the names of the local smart pole and the target smart pole;

[0015] Set the shooting conditions of the local smart pole to shoot the target smart pole;

[0016] According to the coordinates of the target smart pole, adjust the imaging parameters of the camera of the local smart pole to meet the shooting conditions, wherein the imaging parameters include the monitoring direction, the focal length value and the pan / tilt angle value;

[0017] The adjacent visible vector relationship is generated according to the mapping relationship, imaging parameters and shooting conditions.

[0018] In some embodiments, the step of setting the shooting conditions for the local smart pole to shoot the target smart pole includes:

[0019] Generate a rectangular outline of the target smart pole using the minimum rectangle that is tangent to the projection of the outer edges of each component of the target smart pole as the boundary;

[0020] The rectangular outline of the target smart pole is used as the upper layer for rendering, and the video display component of the smart pole at the local end when shooting is used as the lower layer for rendering;

[0021] The shooting condition is when the outer edge projections of the various components of the target smart pole shot in the video display assembly are inscribed in the rectangular outline respectively.

[0022] In some embodiments, the step of photographing the target smart pole after construction using the local smart pole to obtain a second photo image file according to the adjacent visible vector relationship includes:

[0023] Using the name of the target smart pole as a query parameter, query the adjacent visible vector relationship between the local smart pole and the target smart pole to obtain the name of the local smart pole and its imaging parameters;

[0024] The corresponding local smart pole and its imaging parameters are used to capture a second photo image file of the target smart pole after construction when the shooting conditions are met.

[0025] In some embodiments, the comparing the first photo image file and the second photo image file, and the area to be constructed and the area corresponding to the second photo image file, to determine the construction result of the target smart pole includes:

[0026] The Hough straight line detection method is used to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole;

[0027] If it is the same target smart pole, the Hough line detection method is used to detect the construction area in the area corresponding to the second photo image file to determine the construction status of the target smart pole.

[0028] In some embodiments, the Hough line detection method is used to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole, including:

[0029] Based on the coordinates of the upper left corner, length and width of each rectangle in the rectangular outline image, the first photo image file is cropped to obtain a first cropped photo image file;

[0030] cropping the second photo image file in the same cropping manner to obtain a second cropped photo image file;

[0031] Performing similarity detection on the first cropped photo image file and its copy to form a first matching image connected by feature point matching straight lines, and using the Hough straight line detection method on the first matching image to obtain a first parallel straight line number;

[0032] Performing similarity detection on the first cropped photo image file and the second cropped photo image file to form a second matching image connected by feature point matching straight lines, and using the Hough line detection method on the second matching image to obtain a second parallel straight line number;

[0033] When the ratio between the second number of parallel straight lines and the first number of parallel straight lines is greater than a set range, it is determined to be the same target smart pole.

[0034] In some embodiments, the Hough line detection method is used to detect the area to be constructed in the area corresponding to the second photo image file to determine the construction status of the target smart pole, including:

[0035] Acquire a detection image at the same position of the second photo image file according to the coordinates of the upper left corner, length and width of the area to be constructed;

[0036] Performing similarity detection on images matched by the detection image in the training image set to form a third matching image connected by a plurality of feature point matching straight lines;

[0037] Using the Hough line detection method for each third matching image, obtain the corresponding third parallel line number;

[0038] When the maximum number of third parallel straight lines obtained is greater than the set range, it is determined that the target smart pole has been constructed.

[0039] The second aspect of the present invention provides an unstructured data processing device for a smart pole, which collects target smart pole photo images with the same viewing angle, target and specification, thereby avoiding the problem that randomly photographed images cannot be matched and analyzed, and the unstructured image data is damaged.

[0040] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0041] A non-structured data processing device for a smart pole, comprising:

[0042] A recording module, which is used to establish an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole;

[0043] A collection module, which is used to obtain a first photo image file of the target smart pole determined according to the adjacent visible vector relationship before construction, and mark the area to be constructed in the first photo image file;

[0044] The acquisition module is further used to obtain a second photo image file of the target smart pole after construction, taken by the local smart pole according to the adjacent visible vector relationship;

[0045] The judgment module is used to compare the first photo image file and the second photo image file, as well as the area to be constructed and the area corresponding to the second photo image file, to judge the construction result of the target smart pole.

[0046] A third aspect of the present invention provides an unstructured data processing device for a smart pole, the unstructured data processing device for the smart pole comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the unstructured data processing method for the smart pole are implemented.

[0047] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the above-mentioned method for processing unstructured data of a smart pole are implemented.

[0048] Compared with the prior art, the advantages of the present invention are:

[0049] The unstructured data processing method of the smart pole in the present invention improves the timeliness and objectivity of the collection of unstructured data of photo images. Based on the data of the visual vector relationship between smart poles, the on-site touch screen computer can respond to the request of the management system in a timely manner to collect the target smart pole photo images with the same perspective, target and specification, avoiding the problem that the randomly taken images cannot be matched and analyzed, and the unstructured data of the images are damaged; the unstructured data and structured data of the smart pole photo images are organically integrated. The photo images are versioned and the image analysis of the unstructured data of multiple versions of the photos is performed to obtain the analysis results. The result data is organically integrated with the related structured data in the management system, breaking the data gap between the two; optimizing the existing operation and maintenance process. The implementation of this case simplifies the complexity of manually processing the unstructured data of massive smart pole photo images, avoids the error rate, and is conducive to conveniently grasping the on-site conditions of the smart poles and improving the efficiency of operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of a method for processing unstructured data of a smart pole in an embodiment of the present invention;

[0051] Figure 2 is a flow chart of step S1 in an embodiment of the present invention;

[0052] Figure 3 is a flow chart of step S4 in an embodiment of the present invention;

[0053] Figure 4 is a flowchart of step S41 in an embodiment of the present invention;

[0054] Figure 5 is a flowchart of step S42 in an embodiment of the present invention;

[0055] Figure 6 This is an architecture diagram of the operation and maintenance of a smart pole in an embodiment of the present invention;

[0056] Figure 7 Schematic diagram of the rectangular outline of the smart pole in the embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0058] See also Figure 1As shown, an embodiment of the present invention provides a method for processing unstructured data of a smart pole, the method comprising the following steps:

[0059] S1. Establishing an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters, and shooting conditions between the local smart pole and the target smart pole.

[0060] See also Figure 2 As shown, in a specific implementation, step S1 includes:

[0061] S11. Establish a corresponding mapping relationship based on the names of the local smart pole and the target smart pole.

[0062] S12. Set the shooting conditions for the local smart pole to shoot the target smart pole.

[0063] Wherein, step S12 specifically includes:

[0064] Generate a rectangular outline of the target smart pole using the minimum rectangle that is tangent to the projection of the outer edges of each component of the target smart pole as the boundary;

[0065] The rectangular outline of the target smart pole is used as the upper layer for rendering, and the video display component of the local smart pole when shooting is used as the lower layer for rendering;

[0066] The shooting condition is when the outer edge projections of the various components of the target smart pole shot in the video display assembly are inscribed in the rectangular outline respectively.

[0067] S13. According to the coordinates of the target smart pole, adjust the imaging parameters of the camera of the local smart pole to meet the shooting conditions, and the imaging parameters include monitoring direction, focal length value and pan / tilt angle value.

[0068] S14. Generate an adjacent visible vector relationship according to the mapping relationship, imaging parameters and shooting conditions.

[0069] It is worth noting that the adjacent visible vector relationship in the embodiment of the present invention is stable within a period and can be initialized again; the adjacent visible vector relationship includes the local smart pole and the remote smart pole (i.e., the target smart pole), and the direction is that the local smart pole points to the remote smart pole; the attribute items of the adjacent visible vector relationship include: the name of the local smart pole, the name of the local camera, the focal length value of the camera, the viewing angle value of the camera pan / tilt, the name of the remote smart pole, the file name of the rectangular outline image of the pole-type component of the remote smart pole, the name of the photo of the remote smart pole (including the version number), the creation time, and the creation person; the adjacent visible vector relationship represents the parameter requirements that are solidified when taking photos from the local smart pole to the remote smart pole at the same angle, the same target, and the same specification within a period; the specification includes: the photo size is 1080×1440 pixels, and when the length and width of the photo are reduced to 540×720 in the same proportion, the edges of the smart pole-related components of the image are projected and inscribed in the rectangular outline of the corresponding components in the rectangular outline image of the pole-type component of the smart pole of the same pole type.

[0070] In addition, the shooting conditions in this embodiment are based on a method of image rendering with double-layer overlay projection fitting, which is executed in the layout area of ​​the video display component of the video display page of the on-site touch screen computer interface; the upper layer renders a rectangular outline of the smart pole component, and the rendered outline has a transparent background and black outlines; the lower layer renders the on-site touch screen computer interface video display component to complete video playback; the coordinates of the upper left corners of the two layer rendering areas coincide, and the sizes are both 540×720; the video component displays the image of the remote smart pole, and the edges of the smart pole-related components of the image are projected and respectively inscribed in the rectangular outlines of the corresponding components in the rectangular outline of the pole-type components of the smart pole in the upper layer.

[0071] As for the rectangular outline image, its purpose is to be the minimum rectangular boundary that can be projected and tangent to the outer edges of each component of the smart pole in the smart pole photo image; a rectangular means that the two sides of the rectangle are either parallel or orthogonal to the coordinate axis of the plane in which it is located; the image size is 1080×1440 pixels, the background is white, and the rectangular outline is black; the pole types of smart poles include left and right lights, dome lights, and single-side lights; the following will be based on the left and right light models. For the left and right light pole type, the longitudinal center axis coincides with the center axis of the minimum rectangular outline of the pole body component. The rectangle runs through the entire image from top to bottom, and its size is 40×1440 pixels; with the rectangle as the axis, the left side is the minimum rectangular outline of the left light component; the right side is the minimum rectangular outline of the right light component.

[0072] The method of creating the adjacent visible vector relationship between smart poles includes:

[0073] The on-site touch screen computer background on the local smart pole body first automatically adjusts the camera monitoring direction of the local smart pole according to the latitude and longitude coordinates of the remote smart pole; secondly, the background automatically fine-tunes the focal length and pan / tilt angle of the camera until the image of the remote smart pole lamp lighting up on demand is acquired and recognized; then, the focal length and pan / tilt angle of the camera are manually and accurately adjusted to achieve the effect that the edges of the smart pole-related components in the image of the acquired video frame are projected inwardly into the rectangular outline of the corresponding components in the rectangular outline diagram of the pole-type components of the smart pole of the same pole type, and then the image is taken; finally, the relevant setting parameters of the current scene are recorded, and an adjacent visible vector relationship is saved in the adjacent visible vector relationship array.

[0074] S2. Obtain a first photo image file of the target smart pole determined according to the adjacent visible vector relationship before construction, and mark the area to be constructed in the first photo image file.

[0075] S3. According to the adjacent visible vector relationship, the local smart pole is used to photograph the target smart pole after construction to obtain a second photo image file.

[0076] Using the name of the target smart pole as a query parameter, query the adjacent visible vector relationship between the local smart pole and the target smart pole to obtain the name of the local smart pole and its imaging parameters;

[0077] The corresponding local smart pole and its imaging parameters are used to capture a second photo image file of the target smart pole after construction when the shooting conditions are met.

[0078] S4. Compare the first photo image file and the second photo image file, as well as the area to be constructed and its corresponding area in the second photo image file, to determine the construction result of the target smart pole.

[0079] See also Figure 3 As shown, step S4 specifically includes:

[0080] S41. Using the Hough line detection method, detect the first photo image file and the second photo image file to determine whether they are the same target smart pole.

[0081] See also Figure 4 As shown, in a specific implementation, step S41 includes:

[0082] S411. based on the coordinates of the upper left corner, length and width of each rectangle in the rectangular outline image, the first photo image file is cropped to obtain a first cropped photo image file;

[0083] S413. cropping the second photo image file in the same cropping manner to obtain a second cropped photo image file;

[0084] S413. Perform similarity detection on the first cropped photo image file and its copy to form a first matching image connected by feature point matching straight lines, and use the Hough straight line detection method on the first matching image to obtain a first number of parallel straight lines;

[0085] S414. Perform similarity detection on the first cropped photo image file and the second cropped photo image file to form a second matching image connected by feature point matching straight lines, and use the Hough line detection method on the second matching image to obtain a second parallel straight line number;

[0086] S415. When the ratio between the second number of parallel straight lines and the first number of parallel straight lines is greater than a set range, it is determined that they are the same target smart pole.

[0087] See also Figure 5 As shown, in a specific implementation, step S42 includes:

[0088] S42. If the target smart pole is the same, the Hough line detection method is used to detect the construction area corresponding to the second photo image file to determine the construction status of the target smart pole.

[0089] S421. Obtain a detection image at the same position in the second photo image file according to the coordinates of the upper left corner, length and width of the area to be constructed;

[0090] S422. Perform similarity detection on the images matched by the detection image in the training image set to form a third matching image connected by a plurality of feature point matching straight lines;

[0091] S423. Using the Hough line detection method for each third matching image, obtain the corresponding third parallel line number;

[0092] S424. When the maximum number of third parallel straight lines obtained is greater than the set range, it is determined that the target smart pole has been constructed.

[0093] Specifically, see Figure 6As shown, before the construction work, the management personnel mark and save the location to be constructed as a red rectangle on the image according to the latest version of the target smart pole; after the construction receipt, the management platform reviews the receipt, and uses the smart pole name in the work order as the name of the remote smart pole, and obtains the name of the local smart pole after querying the adjacent visible vector array; the management platform sends a review request for the receipt photo of the remote smart pole to the on-site touch screen computer background of the local smart pole; the background obtains the latest version of the remote smart pole photo image and the multi-angle photo image file set of the construction equipment from the management platform; the background queries the adjacent visible vector array through the remote smart pole name, obtains the viewing angle setting parameters of the local camera and sends a command to the actual camera, when the edges of the smart pole-related components of the image in the captured video frame are projected and inscribed in the same pole type. After the rectangular outline of the corresponding component in the rectangular outline diagram of the pole-type component of the smart pole is obtained, a photo is captured, and the consistency of the captured photo and the image of the latest version of the photo is analyzed and checked; the background automatically calculates the red rectangle position of the latest version of the photo image of the remote smart pole, and then extracts a local image from the same position of the captured photo of the pole; the image feature points of the local image and the images in the multi-angle photo file set of the construction equipment are matched one by one to obtain an angle photo file corresponding to the maximum matching value; when the maximum matching value is greater than 3, the remote smart pole construction work order photo is reviewed and approved, otherwise it is not passed; for photos that fail the review, the ratio of the lengths of each side of the corresponding rectangle extracted from the two images involved in the matching is compared, and when the absolute value of the difference between the two ratios is less than 10%, it is determined to be approved, otherwise it is determined to be unapproved.

[0094] In addition, some embodiments further include a step of routinely checking the remote smart pole based on the analysis of the before and after versions of the images:

[0095] First, the manager selects the remote smart pole to be inspected on the management platform interface and sends the inspection task to the on-site touch screen computer background of the local smart pole; the background obtains the adjacent visible vector relationship array from the management platform to the local computer;

[0096] Then, the background obtains the camera setting parameters, the photo image file name P0 of the remote smart pole, and the rectangular outline of the pole-shaped component of the remote smart pole through the adjacent visual vector relationship array G; the background issues a command to set the viewing angle parameters of the camera, and the camera is aimed at the remote smart pole; the on-site touch screen computer background collects real-time video streams, captures photos of the remote smart pole, and uses the outline boundary to cut out the previous version of the photo and the current version of the photo;

[0097] Then, the overall feature point matching and Hough line detection of the two versions of the photo images are carried out to determine whether the inspection object is normal; when the inspection object is determined to be abnormal, the neighboring threshold algorithm is used for local images of the new photo image one by one to complete the status judgment; finally, the inspection results are submitted to the management platform.

[0098] The following is a specific example:

[0099] A0 Initialization of the method and apparatus of the present invention

[0100] 1. The management platform has entered the resource data of the smart poles in the same management domain, which at least includes the name of the smart pole, the longitude and latitude coordinates of the smart pole, and the type of the smart pole; the management platform can manage lighting equipment and touch screen computer equipment;

[0101] 2. See Figure 7 As shown, the management platform is loaded with the outline of the smart pole type components corresponding to the type of smart pole; the outline includes two layers of rectangles; the inner layer of small rectangles includes small rectangles of the pole body components and small rectangles of the lamp components, and the outer layer is a large rectangle containing small rectangles;

[0102] 3. The management platform is loaded with front, back, left, right, top and bottom photo images of various equipment and lighting photos; the above images are JPG files with a size of 50×50 pixels;

[0103] 4. The camera under the local smart pole has been mounted, and the camera protocol has been connected to the on-site touch screen computer background of the local smart pole. The activation personnel can operate the computer interface function;

[0104] A1 At the construction site, establish the adjacent visual vector relationship between the local smart pole and the remote smart pole

[0105] 1.1. The communication connection between the on-site touch screen computer background and the management platform has been established;

[0106] 1.2. On the computer interface, the construction personnel perform the following operations: creating an adjacent visual vector relationship;

[0107] 1.3. The computer backend requests the management platform for the four remote smart pole objects closest to the local smart pole, and the results are saved in the set NSet;

[0108] 1.4. The backend requests the management platform for a hash table map_t_f that maps the smart pole type to the file name of the rectangular outline of the smart pole type component; the primary key of the hash table is the smart pole type and the value is the file name; the backend loads the set CSet of the local smart pole camera object;

[0109] 1.5. Have you completed traversing NSet? If not, go to 1.6; if yes, go to 1.22;

[0110] 1.6. Obtain a remote smart pole object and record it as smart pole P1;

[0111] 1.7. Using the pole type of the smart pole P1 as a query parameter, obtain the file name FN of the rectangular outline diagram of the pole type component of the smart pole from map_t_f; secondly, using FN as a query parameter, obtain the file PM of the corresponding rectangular outline diagram of the pole type component of the smart pole from the management platform;

[0112] 1.8. Have you completed traversing CSet? If not, go to 1.9; if yes, go to 1.5;

[0113] 1.9. Get a camera object C;

[0114] 1.10. The camera object C establishes a communication connection L with the actual camera device;

[0115] 1.11. Get the GIS coordinates (x0, y0) of the smart pole P1;

[0116] 1.12. Using the GIS coordinates (x0, y0) of the smart pole P1 as command parameters, send a command through the communication connection L to drive the camera device to turn its lens angle to the GIS coordinates to calibrate the geographic location;

[0117] Next, the video display component part of the video display page of the on-site touch screen intelligent computing interface of the local smart pole will be rendered in two layers: the upper layer renders the rectangular outline of the smart pole components, the rendered outline has a transparent background and the outline is black; the lower layer renders the video playback of the on-site touch screen computer interface video display component; the coordinates of the upper left corners of the two layer rendering areas coincide, and the sizes are both 540×720; the video component displays the image of the remote smart pole, and the edges of the smart pole-related components of the image are projected and inscribed in the rectangular outlines of the corresponding components in the rectangular outline of the pole-type components of the smart pole in the upper layer.

[0118] Subsequently, below the above-mentioned video playback component, the construction worker clicks the live broadcast button with his finger, and the interface sends a video live broadcast request to the background; the camera object C in the background obtains the real-time video stream through the communication connection L, converts the format using the FFMPEG tool, and pushes it to the video display component of the interface for playback.

[0119] 1.13. The backend initiates a light-on request to the management platform, with the following request parameters: the object name is the name of the smart pole P1, the operation is to turn on the light, and the operation duration is 1 minute;

[0120] Then, the backend requests the management platform for a file M1 of a lighting fixture photo image corresponding to the pole type of the smart pole P1;

[0121] 1.14. After receiving the lighting command sent by the management platform, the on-site touch screen computer backstage of the smart pole P1 issues a command to drive the lighting fixtures of the lighting equipment of the smart pole P1 to light up;

[0122] Then, a visual fine-tuning timer is created in the background of the touch screen computer on the local smart pole site, with a timing period of 20 seconds and a repetition frequency of 3 times;

[0123] 1.15. The background sends a command to the relevant camera device with the communication connection L, the sequence J of all focal length values ​​of the camera, and the sequence S of all pan / tilt angle values ​​of the camera as parameters to execute "A3 fine-tune the focal length value and pan / tilt angle value of the camera";

[0124] 1.16. Did the backend receive the notification from the construction personnel to stop the live video broadcast? If no, go to 1.17; if yes, go to 1.24;

[0125] 1.17. With file M1 and the captured video image frame as parameters, the background performs "automatic analysis of A4 lamp lighting image". After the above analysis is completed, are the rectangular parameters detected? If yes, go to 1.18; otherwise, go to 1.17.1.

[0126] 1.17.1. Does the visual fine-tuning timer repeat more than 3 times? No, go to 1.15; Yes, go to 1.17.2;

[0127] 1.17.2. Manually fine-tune the camera focal length and gimbal angle. After finding the brightly lit area in the video frame, go to 1.18.

[0128] 1.18. Manually adjust the camera focal length and angle in the camera parameter adjustment area of ​​the video display page of the interface to achieve the following image fitting effect:

[0129] The lamp part in the video frame image is projected into a smaller rectangular frame at the top of the rectangular outline of the smart pole body component, and the pole body part in the video frame image is projected into a smaller rectangular frame from top to bottom in the middle of the outline; the entire smart pole projection of the video frame image is inscribed in a larger rectangular frame in the outline.

[0130] Then, a dialog box will pop up on the interface to remind the construction personnel that "the video frame image has matched the pole profile, please confirm";

[0131] After the construction worker clicks the "Confirm" button on the dialog box, the background pauses the video stream push and captures the image P0, which is 1080×1440 pixels in size;

[0132] The upper left corner coordinates, the number of rows and the number of columns of the outer rectangle of the contour image are used as boundary parameters, and the image P0 is used as the source image parameter. The foreground image is extracted from the source image to obtain the image P0'. The execution steps are shown in "A6";

[0133] Perform smart rod-shaped component fitting analysis on the image P0', and refer to "A7" for the execution steps;

[0134] After the above analysis returns true, the photo image file P0 of the smart pole P1 is saved, which is specifically named "P1_P0_V1.0"; "P1" is the name of the smart pole; "P0" is the photo name; and "V1.0" is the version number;

[0135] 1.19. The background saves the focal length value of the current camera C, the PTZ angle value, the file name of the image P0 of the smart pole P1, and the P0 image file;

[0136] 1.20. The interface reminds the user to select a rectangular frame mark for the newly installed camera device on the image P0; then, the background automatically calculates the marked rectangle position information: upper left corner coordinates, length and width, and extracts the image P0' of the marked area; then, the background saves the image P0';

[0137] 1.21. The background stores data, including: the name of the local smart pole, the name of the local camera, the focal length of the camera, the pan / tilt viewing angle of the camera, the name of the remote smart pole, the file name of the corresponding contour image of the remote smart pole, the name of the photo of the remote smart pole, the marked location information, the file name of the marked image P0' file, the creation time, and the creator;

[0138] The above stored data is used as the visual adjacency vector relationship data from the current smart pole to the remote smart pole, and is saved in the array G of the relationship type;

[0139] 1.22, go to 1.8;

[0140] 1.23. The backend uploads the image P0 of the smart pole P1, the marked image P0', and the array G of adjacent visible vector relationships to the management system;

[0141] 1.24. Withdrawal;

[0142] Rectangle extraction of A2 contour map;

[0143] 1.1. Load the outline of the smart pole-shaped components and extract multiple rectangular sets; define the hash table map_r_p that maps the pole body components to the X-coordinate of the center of gravity, where the primary key is the string type of the component identifier and the value is the X-coordinate of the center of gravity;

[0144] Define a rectangular array, the array element is a rectangular information body model, the model attribute item values ​​include: upper left corner coordinate X value, upper left corner coordinate Y value, length, width, center of gravity coordinate X value, center of gravity coordinate Y value, component identification assignment;

[0145] 1.1.1. Have you traversed the above multiple rectangular sets? No, go to 1.1.2; Yes, go to 1.2;

[0146] 1.1.2. If the centroid X coordinate of the rectangle to be matched is less than 2 / 3 of the number of columns of the source image, and the centroid X coordinate of the regular rectangle is greater than 1 / 3 of the number of columns of the source image, and the number of columns of the rectangle is less than 1 / 4 of the number of columns of the source image; then the identified component identifier is recorded as the rod component, and the identifier and the centroid X coordinate value of the rectangle are added to the hash table map_r_p, and step 1.1.1 is performed; otherwise, step 1.1.3 is performed;

[0147] 1.1.3. If the X coordinate of the center of gravity of the rectangle to be matched is less than 1 / 3 of the number of columns of the source image, and the height is less than 1 / 2 of the number of rows of the source image, then the identified component ID is recorded as the left light component, and the ID and the X value of the center of gravity of the rectangle are added to the hash table map_r_p, and step 1.1.1 is performed; otherwise, step 1.1.4 is performed;

[0148] 1.1.4. If the X coordinate of the center of gravity of the rectangle to be matched is greater than 2 / 3 of the source image columns, and the height is less than 1 / 4 of the source image columns, then the identified component ID is recorded as the right light component, and the ID and the X value of the center of gravity of the rectangle are added to the hash table map_r_p, and 1.1.1 is removed; otherwise, 1.1.1 is removed;

[0149] 1.2. Get the coordinates of the upper left corner, length, width, center of gravity coordinate X value, and center of gravity coordinate Y value of the matched rectangle;

[0150] 1.3. Define a rectangle object and assign the following attributes: upper left corner coordinate X value, upper left corner coordinate Y value, length, width, center of gravity coordinate X value, center of gravity coordinate Y value, and component identifier;

[0151] 1.4. Withdrawal;

[0152] A3 fine-tunes the focal length and gimbal angle of the camera. The execution process is as follows:

[0153] 1.1. Load the camera channel connection ID; start an execution thread;

[0154] 1.2. Load the sequence J of all focal length values ​​of the camera;

[0155] 1.3. Load the sequence S of all the PTZ angle values ​​of the camera; Is the traversal of sequence J completed? No, go to 1.4; Yes, go to 1.9;

[0156] 1.4. Get the focal length value J1;

[0157] 1.5. Have you finished traversing sequence S? If yes, go to 1.3; if no, go to 1.6;

[0158] 1.6. Get the gimbal angle S1;

[0159] 1.7. Has thread T received a notification to stop fine-tuning? No, go to 1.8; Yes, go to 1.10;

[0160] 1.8. Connect ID through the camera channel and send the command to adjust the focal length value J1 to the actual camera; connect ID through the camera channel and send the command to adjust the pan / tilt angle value S1 to the actual camera;

[0161] 1.9. Is the wait time longer than 2 seconds? If not, go to 1.9; if yes, go to 1.5;

[0162] 1.10. Exit the thread;

[0163] Automatic analysis of A4 lamp lighting images

[0164] 1.1. The on-site touch screen computer background of the local smart pole uses the M1 image as the template parameter and the image F of the video frame as the target detection parameter to call the "A5 OpenCV template matching algorithm";

[0165] 1.2. When the template matching algorithm is executed and the detected result rectangle parameters are obtained, a command to stop fine-tuning the camera is issued, the current focal length value and gimbal angle value are recorded, and step 1.3 is performed; otherwise, when the template matching algorithm does not detect the result rectangle parameters, step 1.1 is performed;

[0166] 1.3. The video display component in the video frame display area of ​​the intelligent computing interface of the on-site touch screen computer of the local smart pole draws a red rectangular frame of the detected result area; then, a dialog box pops up on the interface, reminding "The on-site touch screen computer of the local smart pole has found the smart pole P1", and go to 1.4;

[0167] 1.4. Withdrawal;

[0168] A5 OpenCV's template matching algorithm is implemented as follows:

[0169] 1.1. Define the source image global variable 1 of image type and assign it to image F;

[0170] 1.2. Define the template image global variable 2 of the image type and assign it to M1;

[0171] 1.3. Define the global variable 3 of the result image of the image type;

[0172] 1.4. Define a temporary source image of an image type;

[0173] 1.5. Define temporary variables for the column values ​​of the result image, which are initialized to: the column value of the global source image variable 1, minus the column value of the global template image variable, plus 1;

[0174] 1.6. Define a temporary variable for the row value of the result image, which is initialized to: the row value of the global source image variable 1 minus the row value of the global template image variable plus 1;

[0175] 1.7. The global variable 3 of the result image calls the creation interface to complete the creation of a specified result image column value, result image row value, and image type of CV_32FC;

[0176] 1.8. Using the global variable 1 of the source image as the global parameter of the object to be searched, the global variable 2 of the template image as the template parameter, the global variable 3 of the result image as the result parameter, and the square difference matching method as the search method parameter, the following operation is performed: the probability of the mapping from the template image to the source image is searched using the template matching method;

[0177] 1.9. Linear normalization of the global variable 3 of the result image, with the parameters of the minimum value within the valve being 0 and the maximum value within the valve being 2;

[0178] 1.10. Find the minimum value of the mapping probability and the minimum value position in the array of global variable 3 of the result image, go to 1.11; otherwise, go to 1.12;

[0179] 1.11. The matching method uses the square difference matching method based on the minimum position; for the minimum value position in the result, a red rectangle is used to mark the matching position; then, the coordinates of the upper left corner, length and width of the rectangle of the matching position are returned;

[0180] 1.12. The coordinates of the upper left corner, length and width of the rectangle returning the matching position are all 0;

[0181] 1.13. End;

[0182] A6 extracts the foreground image of image P0 to obtain image P0'. The execution steps are as follows:

[0183] 1.1. Load the upper left corner coordinates, row number and column number of the outer rectangle of the rectangular outline image of the remote smart pole as the boundary, which is used as the boundary of the extracted foreground image; load the image P0;

[0184] 1.2. Define variables of image type: background variable, foreground variable, and result variable;

[0185] 1.3. Call the foreground separation function of Opencv with image P0, result variable, boundary rectangle, background variable, foreground variable, number of iterations 5, and rectangle cutting as parameters;

[0186] 1.4. Get the foreground pixel and assign it to the result variable;

[0187] 1.5. Create a white image type variable P0' of the same size as P0;

[0188] 1.6. Take the result variable as the input parameter and copy the result variable to the image type variable P0';

[0189] 1.7. Withdrawal;

[0190] A7 analyzes the extracted image P0', and the execution steps are as follows:

[0191] 1.1. Getting Started

[0192] 1.2. Load the contour map of the remote smart pole body component, and use the neighborhood threshold algorithm to extract a set VSet0 of multiple rectangles of the map. For the specific execution process, see "A8";

[0193] Then, execute "A2" to obtain the hash table map_t_x that maps the rectangle type to the barycentric coordinates;

[0194] 1.3. For the image P0', a neighborhood threshold algorithm is used to extract a set VSet1 of multiple rectangular contours. For the specific execution process, see "A8";

[0195] 1.4. Define three Boolean values: result 1, result 2, result 3; the initial values ​​are all false, indicating that none of them fit;

[0196] 1.5. Complete the traversal of VSet0? No, go to 1.6; Yes, go to 1.16;

[0197] 1.6. Get a rectangular object R0;

[0198] 1.7. Complete traversal of set VSet1? No, go to 1.8; Yes, go to 1.5;

[0199] 1.8. Get a rectangular object R1;

[0200] 1.9, |R1.center.X - R0.center.X| / R1.center.X<10%, and |R1.center.Y - R0.center.Y| / R1.center.Y<10%? Yes, remove 1.10; No, remove 1.7;

[0201] 1.10, Is R0 a left lamp component? If yes, go to 1.13; if no, go to 1.11;

[0202] 1.11. Is R0 the right lamp component? If yes, go to 1.14; if no, go to 1.12;

[0203] 1.12, R0 is the rod body? Yes, go to 1.15; No, go to 1.5;

[0204] 1.13, result 1 is determined as the left lamp component fitting; go to 1.5;

[0205] 1.14, result 2 is determined as right lamp component fitting; go to 1.5;

[0206] 1.15, Result 3 is determined as rod body component fitting; go to 1.16;

[0207] 1.16. Are the left part, right part, and rod body parts fully satisfied? Yes, go to 1.17; No, go to 1.18;

[0208] 1.17, return true; go to 1.19;

[0209] 1.18, return false; go to 1.19;

[0210] 1.19, End;

[0211] The execution process of A8's neighborhood threshold algorithm is as follows:

[0212] 1.1. Define the source image variable, the type is image type; read the source image file to be analyzed;

[0213] 1.2. Define grayscale image variables, adaptable image variables, and thresholded image variables, all of which are image types;

[0214] 1.3. Convert the source image variable from BGR type to GRAY type grayscale image type;

[0215] 1.4. Calculate the local threshold according to the brightness distribution of different areas of the image;

[0216] 1.5. Input grayscale image variable, output adaptive image variable, threshold maximum value is 255, adaptation method is neighborhood Gaussian weighted average; threshold type is set to 0 when the current point value is greater than the threshold, otherwise it is set to the maximum value; define the calculation unit as a neighborhood block of 5 pixels, and the offset value adjustment amount is 5;

[0217] 1.6. Call the adaptive threshold method of opencv to get the processed image and assign it to the adaptive image variable;

[0218] 1.7. Convert the adaptive image from GRAY to BRG target image;

[0219] 1.8. Define the contour variable contours, whose type is a collection of sets of image pixels; define the level variable, whose value is to find all rectangles; find the rectangle contours and level information from the BGR target image to obtain an array of rectangle contours.

[0220] B0 Construction Management

[0221] 1.1. The management personnel plan the location of the equipment to be constructed on the smart pole at the other end on the management platform;

[0222] 1.2. The administrator calls the latest version of the remote smart pole image P0 (the first photo image file) on the management platform interface;

[0223] 1.3. The manager uses a red rectangle to mark the location of the equipment to be constructed on the pole position of image P0, and then associates a set of JPG file names of photos of various positions of the equipment to be constructed;

[0224] 1.4. The management platform uses the remote smart pole name as a query parameter to query the visible vector array from the local smart pole to the remote smart pole, and obtains the local smart pole name; the management platform sends a construction picture analysis request to the on-site touch screen computer backend of the local smart pole, and the request parameters include: the remote smart pole name, the name of the latest version of the image P0, and an array of JPG file names of photos of various positions of the equipment to be constructed;

[0225] 1.5. Using the remote smart pole name as a parameter, the local touch screen computer background queries the adjacent visible vector relationship array from the local smart pole to the remote smart pole, and obtains the camera name, focal length value, pan / tilt angle value, and the name of the rectangular outline of the remote smart pole component;

[0226] 1.6. After the backend receives the message from the management platform that the remote smart pole construction is completed, it establishes a communication connection L between the camera and the actual device;

[0227] 1.7. Through the communication connection L, the background sends a command to set the camera focal length value and the gimbal angle value to the actual device and executes successfully;

[0228] 1.8. After the background initiates the video acquisition operation, it obtains the video stream and captures the image P1 (the second photo image file) of the remote smart pole;

[0229] The background extracts the coordinates of the upper left corner, length and width of the red rectangle at the location to be constructed in the image P0;

[0230] Then, the backend obtains the image file of the rectangular outline image of the remote smart pole type component from the management platform using the name of the rectangular outline image of the remote smart pole type component as a query parameter;

[0231] The upper left corner coordinates, length and width of the outer rectangle of the contour image are used to perform range clipping on the image P1 to obtain image P11 (a first clipped photo image file); similarly, the image P0 is range clipped to obtain image P01 (a second clipped photo image file);

[0232] Subsequently, the background performs similarity detection on the image P11 and the copy of the image P11 by using the feature corner point matching method, and forms an image P11' (first matching image) connected by feature point matching straight lines; the Hough straight line detection method is used on the image P11' to obtain the number of parallel straight lines N1; and the execution step is referred to as "B1";

[0233] 1.9. Using the image feature corner point matching method, the background performs similarity detection on the image P11 and the image P01 to form an image P01' (a second matching image) connected by feature point matching straight lines; execute step, see "B1"; use the Hough straight line detection method on the image P01' to obtain the number of parallel straight lines N2; if N2 / N1>95%, it is determined to be the same smart pole; otherwise, it is determined to be different smart poles;

[0234] 1.10. Obtain image Px (detection image) at the same position of image P1 through the upper left corner coordinates, length and width of the red rectangular box of image P0;

[0235] 1.11. Take the image Px as the image name parameter to be searched, the file name of a set S as the training image file name set parameter, and the image files corresponding to the file names in S are stored in the background folder F. Use the feature corner point matching method to determine that Px is matched with the feature corner points in the set S to form an image Sx (the third matching image) connected by the feature point matching straight lines. For its execution process, refer to "B1". Use the Hough line detection method for each Sx image to obtain the number of straight lines N3. Obtain the image Sx when N3 is the largest. If N3>2, record the photo image file name corresponding to Px, and its status is constructed, and go to 1.12. Otherwise, record the photo image P0, and the status is not constructed, and go to 1.13.

[0236] 1.12. If the record is completed, the red rectangle of the P0 photo is updated to a green rectangle, and the version number in the P0 photo image file name is increased by 1;

[0237] Subsequently, the backend uploads the latest P0 photo file, the latest P0 photo name, and the constructed and identified image Px to the management system;

[0238] 1.13. Exit.

[0239] The execution process of the feature corner point matching method of B1 is as follows:

[0240] 1.1. The detector type is SURF algorithm, and the detection type variable is defined;

[0241] 1.2. The descriptor type is SURF algorithm, and the descriptor type variable is defined;

[0242] 1.3. The matching type is FlannBased, and the matching type variable is defined;

[0243] 1.4. Define the name variable of the search image;

[0244] 1.5. Define the training image file name set variable;

[0245] 1.6. Define the default path variable for saving the result image;

[0246] 2. Define the feature detector variables;

[0247] 2.1. Define the extracted feature descriptor variables;

[0248] 2.2. Define the matcher descriptor variable;

[0249] 2.3. Create a descriptor matching detector, with the input parameters being detection type variable, descriptor type variable, matching type variable, feature detector variable, extraction descriptor variable, and matcher descriptor variable;

[0250] 3.1. Feature detector creates a feature detector variable, and the input parameter is the detection type variable;

[0251] 3.2. The descriptor extractor creates an extracted descriptor variable, and the input parameter is a descriptor type variable;

[0252] 3.3. The descriptor matcher creates a matcher descriptor variable, and the input parameter is the match type variable;

[0253] 4.1. Define a temporary variable for the search image, the type is image type;

[0254] 4.2. Define temporary variables for training images, which are arrays of image type;

[0255] 4.3. Define a temporary variable for the training image name, which is a string array type;

[0256] 4.4. Load the temporary variable Px of the search image, load the image file set S from the background folder F, and load the temporary variable of the training image and the temporary variable of the training image name, and assign values ​​successfully;

[0257] 5.1. Define a temporary variable for querying the key point set, the type of which is an array of key points;

[0258] 5.2. Define a temporary variable for the training key point set, the type of which is an array of key points;

[0259] 5.3. Feature detector variable detects key points from the query image and returns a temporary variable of the query key point set

[0260] 5.4. The feature detector variable detects key points from the training image and returns a temporary variable of the training key point set;

[0261] 6.1. Define a variable of query descriptor type, the type is image type;

[0262] 6.2. Input parameters are temporary variables of the search image, temporary variables of the output query key point set, and variables of the output query descriptor type, and feature descriptor variables are extracted for operation completion;

[0263] 6.3. Input parameters are temporary variables of training images, temporary variables of output training images, and variables of output training descriptors, and feature descriptor variables are extracted for operation completion;

[0264] 7.1. Define a temporary variable for the matcher index, whose type is the array associated with the match;

[0265] 7.2. Input parameters: variables of query descriptor type, variables of training descriptor;

[0266] 7.3. Output parameters: matcher index temporary variable, matcher descriptor variable;

[0267] 7.4. Start a timer;

[0268] 7.5. Matcher descriptor variables add variables for training descriptors;

[0269] 7.6, matcher descriptor variables start training;

[0270] 7.7. Stop a timer;

[0271] 7.8. Start a timer;

[0272] 7.9, matcher descriptor variables match query descriptor type variables, matcher index temporary variables;

[0273] 7.10. Stop a timer;

[0274] 8.1. Save the result image:

[0275] The input parameters are temporary variables for search image, temporary variables for output query key point set, temporary variables for training image, temporary variables for output training image, temporary variables for matcher index, temporary variables for training image name, and default path variables for saving result images.

[0276] 8.2. Define a temporary variable of the drawing image, the type is image type;

[0277] 8.3. Define a temporary mask variable, whose type is character array;

[0278] 8.4. Have you completed traversing the elements of the temporary variables of the training image? If not, go to 8.5; if yes, go to 8.9;

[0279] 8.5. If the element is not empty;

[0280] 8.6. Input parameters, matcher index temporary variable, timer i, temporary variable mask;

[0281] 8.7. Temporary variables for search images, temporary variables for query key point sets, temporary variables for training images, temporary variables for output training images, temporary variables for matcher indexes, and temporary variables for drawing images;

[0282] 8.8, the color of the matching straight line is blue;

[0283] 8.9. Withdrawal;

[0284] C Routine Inspection Management

[0285] 1.1. Start;

[0286] 1.2. The manager selects the name of the remote smart pole to be inspected on the management platform interface;

[0287] 1.3. The management platform uses the remote smart pole name as a query parameter and obtains the local smart pole name from the adjacent visible vector relationship array of the smart pole;

[0288] 1.4. The management platform creates an inspection task, and executes the on-site touch screen computer of the local smart pole and the name of the remote smart pole as the inspection target;

[0289] 1.5. Is the communication connection between the management platform and the on-site touch screen computer of the remote smart pole normal? If yes, go to 1.6; if no, go to 1.40;

[0290] 1.6. The management platform sends the inspection task Task to the on-site touch screen computer background of the local smart pole; the background obtains the adjacent visible vector relationship array from the management platform to the local;

[0291] 1.7. The background loads the task Task into the inspection task queue;

[0292] 1.7.1. Has the queue reached the threshold? If yes, send a message out of the queue and go to 1.8; if no, go to 1.7.1;

[0293] 1.8. After receiving the out-queue message, the background obtains the task Task from the inspection task queue;

[0294] 1.9. The background obtains the name of the inspection target remote smart pole in the task;

[0295] 1.10. The backend uses the local smart pole name as an input parameter and obtains the camera name C, camera focal length value, camera pan / tilt angle value, photo image file name P0 of the remote smart pole, and the rectangular outline image name of the pole-shaped component of the remote smart pole through the adjacent visible vector relationship array G;

[0296] 1.11. Using the name P0 as the request parameter, the backend requests the P0 image file from the management platform;

[0297] 1.12. The background establishes a communication connection L between the camera C and the actual device;

[0298] 1.13. Through the communication connection L, the background sends a command to set the focal length value and the pan / tilt angle value of the camera to the actual camera, and the camera is aimed at the remote smart pole;

[0299] 1.14. The background sends a command to start real-time video stream collection of camera C through communication connection L;

[0300] 1.15. The background sends a command to start the capture command of camera C through the communication connection L;

[0301] 1.16. The background obtains the image P1 of the smart pole; obtains the outline image file of the pole body component of the remote smart pole from the management platform through the outline image name of the pole body component of the remote smart pole;

[0302] 1.17. After the upper left corner coordinates of the contour image of the pole body of the remote smart pole are aligned with the upper left corner coordinates of P0 and P1, P0 and P1 are clipped using the outer rectangular boundary of the contour image to obtain temporary images P01 and P11;

[0303] 1.18. P01 and its copy P01' are used as query parameters and sample parameters respectively, and "B1" is called to execute the feature corner point matching method to draw the image P001 connected by the feature point matching straight lines;

[0304] 1.19. Use Hough line detection to obtain the number of straight lines N1 for image P001;

[0305] 1.20. Take P01 and its copy P11 as query parameters and sample parameters respectively, call "B1", execute the feature corner point matching method to draw the image P111 connected by the feature point matching straight line;

[0306] 1.21. Perform Hough line detection on image P111 to obtain the number of lines N2;

[0307] 1.22. Is N1 equal to N2? If yes, go to 1.23; if no, go to 1.24;

[0308] 1.23. Record the inspection task Task result as normal; go to 1.39;

[0309] 1.24. Define a rectangular position model, whose attribute items include: position number, upper left corner coordinates, length, and width. The status includes: normal, adding a mount, and changing direction; define position object 1, with the type of rectangular position model; define the rectangular position type as an array type object A0 of elements; the position number is an integer, starting from 0;

[0310] Obtain the position parameters of all green rectangular frames marked in image P01: upper left corner coordinates, length, and width; assign these parameters to position object 1; add position object 1 to array A0; until all green rectangles are loaded;

[0311] 1.25. Define a counter i, i = 0;

[0312] 1.26. Traverse each position model object of the green rectangular frame in A0 to obtain the position number i, crop image Pi' from image P11 using the upper left corner coordinates, length, and width; crop image Pi from P01 using the upper left corner coordinates, length, and width;

[0313] 1.27. For each Pi and Pi', use them as query parameters and sample parameters respectively, perform the corner feature value detection algorithm and draw the lines mapping the corners to obtain image Pi1;

[0314] 1.28. Use the Hough line detection method to detect lines in image Pi1;

[0315] 1.29. Are all the detected lines parallel to the X-axis? If yes, go to 1.30; if no, go to 1.31;

[0316] 1.30. Record the inspection result of image Pi as normal, and store the result in A0[i];

[0317] 1.31. Use the neighborhood threshold algorithm to extract a set VSet01 of multiple rectangles from Pi image;

[0318] 1.32. Use the neighborhood threshold algorithm to extract a set VSet02 of multiple rectangles from Pi' image;

[0319] 1.33. Is VSet01 < VSet02? If yes, go to 1.34; if no, go to 1.35;

[0320] 1.34, the inspection result record of image Pi is added with mounting, and the result is stored in A0[i]; go to 1.36;

[0321] 1.35. The inspection result of image Pi is recorded as changed direction, and the result is stored in A0[i]; go to 1.36;

[0322] 1.36. Have you traversed all Pi and PI'? Yes, go to 1.38; No, go to 1.37;

[0323] 1.37. Enter the optional task results and record the image A0 inspection results; go to 1.27;

[0324] 1.38. Enter the optional task results and record the image A0 inspection results;

[0325] 1.39. Submit inspection results to the management platform;

[0326] 1.40. End.

[0327] In summary, the unstructured data processing method of the smart pole in the present invention improves the timeliness and objectivity of the collection of unstructured data of photo images. Based on the data of the visual vector relationship between smart poles, the on-site touch screen computer can respond to the request of the management system in a timely manner, collect the target smart pole photo images with the same perspective, target and specification, avoid the problem that the images taken at random cannot be matched and analyzed, and the unstructured data of the image is damaged; and realize the organic fusion of the unstructured data and structured data of the smart pole photo images. The photo image is versioned and the image analysis of the unstructured data of multiple versions of the photo is performed to obtain the analysis results. The result data is organically integrated with the related structured data in the management system, breaking the data gap between the two; and optimizing the existing operation and maintenance process. The implementation of this case simplifies the complexity of manually processing the unstructured data of massive smart pole photo images, avoids the error rate, and is conducive to conveniently grasping the on-site conditions of the smart pole and improving the efficiency of operation and maintenance. At the same time, the embodiment of the present invention also provides an unstructured data processing device for a smart pole, which includes a recording module, a collection module and a judgment module.

[0328] The recording module is used to establish an adjacent visual vector relationship between the local smart pole and the target smart pole. The adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole.

[0329] The acquisition module is used to obtain a first photo image file of a target smart pole determined according to the adjacent visible vector relationship before construction, and mark the area to be constructed in the first photo image file.

[0330] The acquisition module is also used to obtain a second photo image file of the target smart pole after construction, taken by the local smart pole according to the adjacent visible vector relationship;

[0331] The judgment module is used to compare the first photo image file and the second photo image file, as well as the area to be constructed and the area corresponding to the second photo image file, to judge the construction result of the target smart pole.

[0332] In some embodiments, the recording module establishes an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole, including:

[0333] Establish a corresponding mapping relationship based on the names of the local smart pole and the target smart pole;

[0334] Set the shooting conditions of the local smart pole to shoot the target smart pole;

[0335] According to the coordinates of the target smart pole, adjust the imaging parameters of the camera of the local smart pole to meet the shooting conditions. The imaging parameters include monitoring direction, focal length and pan / tilt angle.

[0336] According to the mapping relationship, imaging parameters and shooting conditions, an adjacent visible vector relationship is generated.

[0337] In some embodiments, the recording module sets the shooting conditions for the local smart pole to shoot the target smart pole, including:

[0338] Generate a rectangular outline of the target smart pole using the minimum rectangle that is tangent to the projection of the outer edges of each component of the target smart pole as a boundary;

[0339] The rectangular outline of the target smart pole is used as the upper layer for rendering, and the video display component of the local smart pole when shooting is used as the lower layer for rendering;

[0340] The shooting condition is when the outer edge projections of the various components of the target smart pole shot in the video display assembly are inscribed in the rectangular outline respectively.

[0341] In some embodiments, the acquisition module uses the local smart pole to photograph the target smart pole after construction to obtain a second photo image file according to the adjacent visible vector relationship, including:

[0342] Using the name of the target smart pole as a query parameter, query the adjacent visible vector relationship between the local smart pole and the target smart pole to obtain the name of the local smart pole and its imaging parameters;

[0343] The corresponding local smart pole and its imaging parameters are used to capture a second photo image file of the target smart pole after construction when the shooting conditions are met.

[0344] In some embodiments, the judgment module compares the first photo image file and the second photo image file, and the area to be constructed and the area corresponding to the second photo image file, to judge the construction result of the target smart pole, including:

[0345] The Hough straight line detection method is used to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole;

[0346] If it is the same target smart pole, the Hough line detection method is used to detect the construction area in the area corresponding to the second photo image file to determine the construction status of the target smart pole.

[0347] In some embodiments, the judgment module uses the Hough line detection method to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole, including:

[0348] Based on the coordinates of the upper left corner, length and width of each rectangle in the rectangular outline image, the first photo image file is cropped to obtain a first cropped photo image file;

[0349] cropping the second photo image file in the same cropping manner to obtain a second cropped photo image file;

[0350] Performing similarity detection on the first cropped photo image file and its copy to form a first matching image connected by feature point matching straight lines, and using the Hough straight line detection method on the first matching image to obtain a first parallel straight line number;

[0351] Performing similarity detection on the first cropped photo image file and the second cropped photo image file to form a second matching image connected by feature point matching straight lines, and using the Hough line detection method on the second matching image to obtain a second parallel straight line number;

[0352] When the ratio between the second number of parallel straight lines and the first number of parallel straight lines is greater than a set range, it is determined to be the same target smart pole.

[0353] In some embodiments, the judgment module uses the Hough line detection method to detect the area to be constructed in the area corresponding to the second photo image file to judge the construction status of the target smart pole, including:

[0354] Acquire a detection image at the same position of the second photo image file according to the coordinates of the upper left corner, length and width of the area to be constructed;

[0355] Performing similarity detection on images matched by the detection image in the training image set to form a third matching image connected by a plurality of feature point matching straight lines;

[0356] Using the Hough line detection method for each third matching image, obtain the corresponding third parallel line number;

[0357] When the maximum number of third parallel straight lines obtained is greater than the set range, it is determined that the target smart pole has been constructed.

[0358] In summary, the unstructured data processing device for smart poles in the present invention improves the timeliness and objectivity of the collection of unstructured data of photo images. Based on the data of the visual vector relationship between smart poles, the on-site touch screen computer can respond to the request of the management system in a timely manner to collect target smart pole photo images with the same perspective, target and specification, avoiding the problem that randomly taken images cannot be matched and analyzed, and the image unstructured data is damaged; the organic integration of the unstructured data and structured data of smart pole photo images is achieved. Versioning management of photo images is performed, and image analysis of the unstructured data of multiple versions of photos is performed to obtain analysis results. The result data is organically integrated with the relevant structured data in the management system, breaking the data gap between the two; optimizing the existing operation and maintenance process. The implementation of this case simplifies the complexity of manually processing massive unstructured data of smart pole photo images, avoids the error rate, and is conducive to conveniently grasping the on-site conditions of smart poles and improving the efficiency of operation and maintenance.

[0359] An embodiment of the present invention also provides an unstructured data processing device for a smart pole, which includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the unstructured data processing method for the smart pole are implemented.

[0360] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0361] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0362] Establishing an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole;

[0363] Obtaining a first photo image file of a target smart pole determined according to an adjacent visible vector relationship before construction, and marking a to-be-constructed area of ​​the first photo image file;

[0364] According to the adjacent visible vector relationship, the local smart pole is used to photograph the target smart pole after construction to obtain a second photo image file;

[0365] The first photo image file and the second photo image file, as well as the area to be constructed and its corresponding area in the second photo image file are compared to determine the construction result of the target smart pole.

[0366] In one embodiment, the processor establishes an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a fixed mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole, and is used to achieve:

[0367] A corresponding mapping relationship is established based on the names of the local smart pole and the target smart pole.

[0368] Set the shooting conditions for the local smart pole to shoot the target smart pole.

[0369] According to the coordinates of the target smart pole, adjust the imaging parameters of the camera of the local smart pole to meet the shooting conditions. The imaging parameters include monitoring direction, focal length value and pan / tilt angle value.

[0370] According to the mapping relationship, imaging parameters and shooting conditions, an adjacent visible vector relationship is generated.

[0371] In one embodiment, the processor implements setting the shooting conditions of the local smart pole shooting the target smart pole, so as to achieve:

[0372] A rectangular outline of the target smart pole is generated by taking the minimum rectangle tangent to the projection of the outer edges of each component of the target smart pole as the boundary.

[0373] The rectangular outline of the target smart pole is rendered as the upper layer, and the video display component of the local smart pole when shooting is rendered as the lower layer.

[0374] The shooting condition is when the outer edge projections of the various components of the target smart pole shot in the video display assembly are inscribed in the rectangular outline respectively.

[0375] In one embodiment, the processor uses the local smart pole to photograph the target smart pole after construction to obtain a second photo image file according to the adjacent visible vector relationship, so as to achieve:

[0376] The name of the target smart pole is used as a query parameter to query the adjacent visible vector relationship between the local smart pole and the target smart pole, and the name of the local smart pole and its imaging parameters are obtained.

[0377] The corresponding local smart pole and its imaging parameters are used to capture a second photo image file of the target smart pole after construction when the shooting conditions are met.

[0378] In one embodiment, the processor compares the first photo image file and the second photo image file, and the area to be constructed and the area corresponding to the second photo image file, to determine the construction result of the target smart pole, so as to achieve:

[0379] The Hough line detection method is used to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole.

[0380] If it is the same target smart pole, the Hough line detection method is used to detect the construction area in the area corresponding to the second photo image file to determine the construction status of the target smart pole.

[0381] In one embodiment, the processor implements the Hough line detection method to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole, so as to achieve:

[0382] The first photo image file is cropped based on the upper left corner coordinates, length and width of each rectangle in the rectangular outline image to obtain a first cropped photo image file.

[0383] The second photo image file is cropped in the same cropping manner to obtain a second cropped photo image file.

[0384] A similarity detection is performed on the first cropped photo image file and its copy to form a first matching image connected by feature point matching straight lines, and a Hough line detection method is used on the first matching image to obtain a first parallel line number.

[0385] A similarity detection is performed on the first cropped photo image file and the second cropped photo image file to form a second matching image connected by feature point matching straight lines, and a Hough line detection method is used on the second matching image to obtain a second parallel line number.

[0386] When the ratio between the second number of parallel straight lines and the first number of parallel straight lines is greater than a set range, it is determined to be the same target smart pole.

[0387] In one embodiment, the processor implements the Hough line detection method to detect the area to be constructed in the area corresponding to the second photo image file, and determines the construction status of the target smart pole, so as to achieve:

[0388] Acquire a detection image at the same position of the second photo image file according to the coordinates of the upper left corner, length and width of the area to be constructed;

[0389] Performing similarity detection on images matched by the detection image in the training image set to form a third matching image connected by a plurality of feature point matching straight lines;

[0390] Using the Hough line detection method for each third matching image, obtain the corresponding third parallel line number;

[0391] When the maximum number of third parallel straight lines obtained is greater than the set range, it is determined that the target smart pole has been constructed.

[0392] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the above-mentioned method for processing unstructured data of the smart pole are implemented.

[0393] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer-readable storage medium (or a non-transitory medium) and a communication medium (or a transient medium).

[0394] As is known to those of ordinary skill in the art, the term computer-readable storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is known to those of ordinary skill in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0395] Exemplarily, the computer-readable storage medium may be an internal storage unit of the electronic device of the aforementioned embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device.

[0396] The above are only specific implementations of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the embodiments of the present invention, and these modifications or replacements should be included in the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for processing unstructured data of a smart pole, characterized in that: The method comprises the following steps: Establishing an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters, and shooting conditions between the local smart pole and the target smart pole; Acquire a first photo image file of the target smart pole determined according to the adjacent visible vector relationship before construction, and mark the area to be constructed in the first photo image file; According to the adjacent visible vector relationship, using the local smart pole to photograph the target smart pole after construction to obtain a second photo image file; The first photo image file and the second photo image file, as well as the area to be constructed and its corresponding area in the second photo image file are compared to determine the construction result of the target smart pole.

2. The method for processing unstructured data of a smart pole according to claim 1, characterized in that: The establishing of the adjacent visible vector relationship between the local smart pole and the target smart pole, wherein the adjacent visible vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole, includes: Establish a corresponding mapping relationship based on the names of the local smart pole and the target smart pole; Set the shooting conditions of the local smart pole to shoot the target smart pole; According to the coordinates of the target smart pole, adjust the imaging parameters of the camera of the local smart pole to meet the shooting conditions, wherein the imaging parameters include the monitoring direction, the focal length value and the pan / tilt angle value; The adjacent visible vector relationship is generated according to the mapping relationship, imaging parameters and shooting conditions.

3. The method for processing unstructured data of a smart pole according to claim 2, characterized in that: The setting of shooting conditions for the local smart pole to shoot the target smart pole includes: Generate a rectangular outline of the target smart pole using the minimum rectangle that is tangent to the projection of the outer edges of each component of the target smart pole as the boundary; The rectangular outline of the target smart pole is used as the upper layer for rendering, and the video display component of the smart pole at the local end when shooting is used as the lower layer for rendering; The shooting condition is when the outer edge projections of the various components of the target smart pole shot in the video display assembly are inscribed in the rectangular outline respectively.

4. The method for processing unstructured data of a smart pole according to claim 3, characterized in that: The method of using the local smart pole to photograph the target smart pole after construction to obtain a second photo image file according to the adjacent visible vector relationship includes: Using the name of the target smart pole as a query parameter, query the adjacent visible vector relationship between the local smart pole and the target smart pole to obtain the name of the local smart pole and its imaging parameters; The corresponding local smart pole and its imaging parameters are used to capture a second photo image file of the target smart pole after construction when the shooting conditions are met.

5. The method for processing unstructured data of a smart pole according to claim 4, characterized in that: The comparing the first photo image file and the second photo image file, and the area to be constructed and the area corresponding to the second photo image file, to determine the construction result of the target smart pole includes: The Hough straight line detection method is used to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole; If it is the same target smart pole, the Hough line detection method is used to detect the construction area in the area corresponding to the second photo image file to determine the construction status of the target smart pole.

6. The method for processing unstructured data of a smart pole according to claim 5, characterized in that: The Hough line detection method is used to detect the first photo image file and the second photo image file to determine whether they are the same target smart pole, including: Based on the coordinates of the upper left corner, length and width of each rectangle in the rectangular outline image, the first photo image file is cropped to obtain a first cropped photo image file; cropping the second photo image file in the same cropping manner to obtain a second cropped photo image file; Performing similarity detection on the first cropped photo image file and its copy to form a first matching image connected by feature point matching straight lines, and using the Hough straight line detection method on the first matching image to obtain a first parallel straight line number; Performing similarity detection on the first cropped photo image file and the second cropped photo image file to form a second matching image connected by feature point matching straight lines, and using the Hough line detection method on the second matching image to obtain a second parallel straight line number; When the ratio between the second number of parallel straight lines and the first number of parallel straight lines is greater than a set range, it is determined to be the same target smart pole.

7. The method for processing unstructured data of a smart pole according to claim 5, characterized in that: The Hough straight line detection method is used to detect the area to be constructed in the area corresponding to the second photo image file to determine the construction status of the target smart pole, including: Acquire a detection image at the same position of the second photo image file according to the coordinates of the upper left corner, length and width of the area to be constructed; Performing similarity detection on images matched by the detection image in the training image set to form a third matching image connected by a plurality of feature point matching straight lines; Using the Hough line detection method for each third matching image, obtain the corresponding third parallel line number; When the maximum number of third parallel straight lines obtained is greater than the set range, it is determined that the target smart pole has been constructed.

8. An unstructured data processing device for a smart pole, characterized in that: include: A recording module, which is used to establish an adjacent visual vector relationship between the local smart pole and the target smart pole, wherein the adjacent visual vector relationship includes a solidified mapping relationship, imaging parameters and shooting conditions between the local smart pole and the target smart pole; A collection module, which is used to obtain a first photo image file of the target smart pole determined according to the adjacent visible vector relationship before construction, and mark the area to be constructed in the first photo image file; The acquisition module is further used to obtain a second photo image file of the target smart pole after construction, taken by the local smart pole according to the adjacent visible vector relationship; The judgment module is used to compare the first photo image file and the second photo image file, as well as the area to be constructed and the area corresponding to the second photo image file, to judge the construction result of the target smart pole.

9. An unstructured data processing device for a smart pole, characterized in that: The unstructured data processing device of the smart pole includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the unstructured data processing method of the smart pole as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the unstructured data processing method for a smart pole as described in any one of claims 1 to 7 are implemented.

Images