Inspection target detection and positioning method, device and equipment, and storage medium

Abstract

The application provides an inspection target detection and positioning method, device, equipment and storage medium. The method comprises the following steps: detecting a first inspection target in a target inspection sub-region of an inspection device and a first geographical position of the first inspection target. The target inspection sub-region is a sub-region inspected by the inspection device during the execution of an inspection task. The first inspection target is displayed at the first geographical position in an inspection map. The technical solution does not require a user to monitor an inspection picture in real time.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, and particularly relates to a method and device for detecting and positioning a target for inspection, equipment and a storage medium. BACKGROUND

[0002] With the development of science and technology, more and more industries have begun to choose unmanned aerial vehicles (UAVs) for inspection instead of traditional manual inspection. The UAVs can be equipped with various tools, such as cameras, infrared sensors, etc.

[0003] After setting a flight task for a UAV, the UAV transmits images and videos collected by the UAV to a background system in real time during execution of the flight task, and a background staff member monitors the inspection pictures in real time to determine the target in the pictures, which is not convenient. SUMMARY

[0004] The present application provides a method and device for detecting and positioning a target for inspection, equipment and a storage medium to solve the technical problem that a background staff member needs to monitor inspection pictures in real time to position a target during UAV inspection.

[0005] In a first aspect, a method for detecting and positioning a target for inspection is provided, comprising:

[0006] detecting a first target for inspection in a target inspection sub-region of an inspection device and a first geographical position of the first target for inspection, the target inspection sub-region being a sub-region inspected by the inspection device during execution of an inspection task;

[0007] displaying the first target for inspection at the first geographical position in an inspection map.

[0008] In this technical solution, the target for inspection in the inspection sub-region of the inspection device and the geographical position of the target for inspection are detected, and then the target for inspection in the inspection sub-region is displayed at the corresponding geographical position in the inspection map. A user can quickly determine the target for inspection and the geographical position of the target for inspection inspected by the inspection device during execution of the inspection task by checking the inspection map, without the need for the user to monitor the inspection pictures in real time, which is more simple and convenient.

[0009] With reference to the first aspect, in a possible implementation manner, the displaying the first inspection target at the first geographic position on the inspection map comprises: adding the first inspection target and the first geographic position to an inspection target set, the inspection target set comprising all inspection targets and geographic positions of the all inspection targets that have been detected in a process in which the inspection device performs an inspection task; and displaying all inspection targets in the inspection target set at corresponding geographic positions of the all inspection targets in the inspection map. By adding the inspection target and the inspection target to the inspection target set comprising all inspection targets and geographic positions of the all inspection targets that have been detected by the inspection device, and displaying all inspection targets in the inspection target set at corresponding geographic positions in the inspection map, a user can determine all inspection targets and positions of the all inspection targets in the entire inspection process according to the inspection map, thereby completing global target statistics.

[0010] With reference to the first aspect, in a possible implementation manner, before the adding the first inspection target and the first geographic position to the inspection target set, the method further comprises: obtaining a target inspection sub-region; and deleting, in the inspection target set, a second inspection target and a geographic position of the second inspection target, the second inspection target being an inspection target in the inspection target set and located in the target inspection sub-region. Before adding the detected inspection target and the position of the inspection target to the inspection target set, the inspection target and the position located in the inspection sub-region are deleted from the inspection target set, which can realize deduplication of the same inspection target in the inspection sub-region and ensure accurate statistics of the target.

[0011] With reference to the first aspect, in a possible implementation manner, before the deleting, in the inspection target set, the second inspection target and the geographic position of the second inspection target, the method further comprises: determining whether the second geographic position located in the target inspection sub-region exists in the inspection target set; if the second geographic position exists in the inspection target set, determining that the second inspection target exists in the inspection target set; and if the second geographic position does not exist in the inspection target set, determining that the second inspection target does not exist in the inspection target set. By detecting whether the geographic position located in the inspection sub-region exists in the inspection target set to detect whether the inspection target located in the inspection sub-region exists in the inspection target set, the implementation manner is simple.

[0012] With reference to the first aspect, in a possible implementation manner, the obtaining the target inspection sub-region comprises: obtaining an inspection image of the inspection device; and determining a target inspection sub-region of the inspection image as the target inspection sub-region.

[0013] With reference to the first aspect, in a possible implementation manner, before the patrol image corresponding patrol map region is determined as the target patrol sub-region, the method further includes: obtaining a vertex image coordinate of the patrol image, the vertex image coordinate being a coordinate in a pixel coordinate system; performing coordinate conversion on the vertex image coordinate to obtain a patrol position coordinate corresponding to the patrol image, the patrol position coordinate being a coordinate in a world coordinate system; and in the patrol map, a sub-region enclosed by the patrol position coordinate is determined as the patrol map region corresponding to the patrol image. The patrol sub-region of the patrol device can be accurately determined by means of image acquisition and coordinate conversion.

[0014] With reference to the first aspect, in a possible implementation manner, the detection of the first patrol target in the target patrol sub-region of the patrol device and the first geographic position of the first patrol target includes: performing patrol target identification on the patrol image of the patrol device to obtain the first patrol target and a first image coordinate of the first patrol target in the patrol image; and performing coordinate conversion on the first image coordinate to obtain a first geographic position coordinate of the first patrol target, the first geographic position coordinate being a coordinate in a world coordinate system. The geographic position of the patrol target can be accurately determined by means of target identification and coordinate conversion.

[0015] With reference to the first aspect, in a possible implementation manner, the performing of the patrol target identification on the patrol image of the patrol device to obtain the first patrol target and the first image coordinate of the first patrol target in the patrol image includes: performing the patrol target identification on the patrol image of the patrol device by using a target identification model to obtain the first patrol target and the first image coordinate of the first patrol target in the patrol image.

[0016] With reference to the first aspect, in a possible implementation manner, the performing of the coordinate conversion on the first image coordinate to obtain the first geographic position coordinate of the first patrol target includes: obtaining a first device pose of the patrol device and a first device position coordinate, the first device pose being a device pose of the patrol device when the patrol image is acquired, the first device position coordinate being a geographic position coordinate when the patrol image is acquired, and the first device position coordinate being a coordinate in a world coordinate system; determining a target conversion matrix according to the first device pose and the first device position coordinate, the target conversion matrix being used to represent a conversion relationship between a pixel coordinate system and the world coordinate system when the patrol image is acquired by the patrol device; and performing the coordinate conversion on the first image coordinate according to the target conversion matrix to obtain the first geographic position coordinate.

[0017] In a second aspect, a target detection and positioning device is provided, comprising:

[0018] a target detection module configured to detect a first target and a first geographical position of the first target in a target sub-region of the inspection device, the target sub-region being a sub-region inspected by the inspection device during the execution of the inspection task;

[0019] a target display module configured to display the first target at the first geographical position in the inspection map.

[0020] In a third aspect, a computer device is provided, comprising a memory and one or more processors, the memory being connected to the one or more processors, and the one or more processors being configured to execute one or more computer programs stored in the memory, and the one or more processors, when executing the one or more computer programs, causing the computer device to implement the target detection and positioning method of the first aspect.

[0021] In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program comprising program instructions, and the program instructions, when executed by a processor, causing the processor to execute the target detection and positioning method of the first aspect.

[0022] The present application can achieve the following technical effects: the user can quickly determine the target and the geographical position of the target in the inspection map, without the need for the user to monitor the inspection picture in real time, which is more simple and convenient. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 a schematic diagram of an inspection system provided by an embodiment of the present application;

[0024] Figure 2 a flowchart of a target detection and positioning method provided by an embodiment of the present application;

[0025] Figure 3 a schematic diagram of displaying a target in an inspection map provided by an embodiment of the present application;

[0026] Figure 4 a flowchart of another target detection and positioning method provided by an embodiment of the present application;

[0027] Figure 5 a schematic diagram of an inspection image and an inspection map region corresponding to the inspection image provided by an embodiment of the present application;

[0028] Figure 6This is a schematic diagram showing the inspection targets and their locations on an inspection map, provided as an embodiment of this application.

[0029] Figure 7 This is a schematic diagram of the structure of an inspection target detection and positioning device provided in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0032] The technical solution of this application is applicable to inspection scenarios, which can refer to scenarios in which drones are used to perform inspection tasks, or scenarios in which other inspection equipment is used to perform inspection tasks, such as scenarios in which inspection robots are used to perform ground inspection tasks.

[0033] The technical solution of this application can be applied to inspection equipment that performs inspection tasks, and also to the inspection control platform of an inspection system that includes inspection equipment. (See first...) Figure 1 , Figure 1 A schematic diagram of an inspection system provided in an embodiment of this application is shown below. Figure 1 As shown, the inspection system 10 may include an inspection device 101 and an inspection control platform 102. The inspection device 101 is used to replace a person to move in the area to be inspected (i.e., the inspection area), collect relevant data (image data and various sensor data) in the inspection area in real time, and transmit the collected relevant data to the inspection control platform 102 to complete the inspection task. The inspection control platform 102 is a user-facing device. The inspection control platform 102 is used to issue inspection instructions to the inspection device 101 to instruct the inspection device 101 to perform inspection tasks according to the inspection instructions. The area covered by the inspection device performing the inspection task is the inspection area. The inspection control platform 102 can respond to user commands and send inspection instructions to the inspection device 101, enabling the inspection device 101 to perform inspection tasks under user control.

[0034] Specifically, the inspection equipment 101 can be a drone or an inspection robot, etc.; the inspection control platform 102 may include a cloud platform, a back-end server, and human-interactive devices (such as a display screen), etc.

[0035] based on Figure 1 The inspection system shown can realize the technical solution of this application. The technical solution of this application is described in detail below.

[0036] See Figure 2 ,Figure 2 A flowchart of a method for detecting and positioning a target for inspection provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the method comprises the following steps: Figure 2

[0037] S201, detecting a first target for inspection in a target sub-region for inspection of the inspection device and a first geographic position of the first target for inspection.

[0038] Here, the target sub-region for inspection is a sub-region detected by the inspection device during the execution of the inspection task by the inspection device. The execution of the inspection task by the inspection device can be, but is not limited to, a road inspection process, a power inspection process, a safety inspection process, etc. The target sub-region for inspection can be understood as a region within the maximum range that can be detected by the sensor on the inspection device during the execution of the task by the inspection device. The target sub-region for inspection belongs to a part of the inspection region of the inspection device. The inspection region of the inspection device refers to the region that needs to be covered by the inspection device during the execution of the inspection task. The inspection region of the inspection device is related to the inspection task performed by the inspection device, and the inspection region of the inspection device is different when the inspection task performed by the inspection device is different. For example, when the inspection task performed by the inspection device is to inspect the entire campus region, the inspection region of the inspection device is the entire campus region, and the target sub-region for inspection can be a classroom region, a playground region, an office region, etc. in the campus. Both the target sub-region for inspection and the inspection region can be represented on an inspection map, which is a map used by the inspection device during the execution of the inspection task.

[0039] The target for inspection refers to a target that needs to be acquired / detected by the inspection device during the execution of the inspection task. The target for inspection is different when the inspection task performed by the inspection device is different. For example, when the inspection task performed by the inspection device is a road inspection task, the target for inspection can be pedestrians, vehicles, traffic light signs, etc. on the road. For another example, when the inspection task performed by the inspection device is a power inspection process, the target for inspection can be a power pole. The present application does not make any limitation.

[0040] The first target for inspection in the target sub-region for inspection refers to a target for inspection located in the target sub-region for inspection when the inspection device performs the task. The first geographic position of the first target for inspection refers to the geographic position of the first target for inspection on the inspection map, and the first geographic position is represented by coordinates in a world coordinate system.

[0041] In some possible cases, the first target for inspection and the first geographic position of the first target for inspection can be detected based on image recognition and target positioning.

[0042] In which, the first target for inspection and the first geographic position of the first target for inspection can be detected by the following steps A1-A2.

[0043] ​A1, performing inspection target recognition on the inspection image of the inspection device to obtain a first inspection target and a first image coordinate of the first inspection target in the inspection image of the inspection device.

[0044] Here, the inspection image of the inspection device refers to an image acquired by the inspection device in the process of performing an inspection task. The inspection image of the inspection device can be obtained by a camera in the inspection device.

[0045] In a possible implementation, the inspection target recognition model can be used to perform inspection target recognition on the inspection image of the inspection device to obtain a first inspection target and a first image coordinate of the first inspection target in the inspection image.

[0046] The target recognition model can be a model trained based on deep learning technology for identifying inspection targets, and the target recognition model includes but is not limited to a region convolutional neural network (RCNN) model, a YOLO (you only look once) model, an SSD (single shot multibox detector) model, and the like, without limitation.

[0047] After the inspection image of the inspection device is input to the target recognition model, the target recognition model can output information {u t , v t , w, h, c} of the inspection target in the inspection image, the inspection target in the inspection image being the first inspection target, (u t , v t ) being the pixel coordinates of the center point of the first inspection target in the inspection image of the inspection device, w and h being the width and height of the first inspection target in the inspection image of the inspection device, respectively, and c being the category of the first inspection target. For example, the inspection task performed by the inspection device is a road inspection task, and the category of the first inspection target can be one of targets on the road such as pedestrians, vehicles, and traffic light signs.

[0048] It should be understood that (u t , v t ) can also be converted into coordinates (x t , y t ) in the image coordinate system, which can be converted from the coordinates in the pixel coordinate system. The coordinates in the image coordinate system and the coordinates in the pixel coordinate system satisfy the following conversion relationship: x = (u - u o ) * W, y = (v - v o ) * H, (x, y) representing the coordinates in the image coordinate system, (u, v) representing the coordinates in the pixel coordinate system, W and H being the width and height of the inspection image, respectively, ando , v o ) is the first image coordinate of the first inspection target in the inspection image of the inspection device. The first image coordinate of the first inspection target in the inspection image of the inspection device can be (u t , v t ), or (x t , y t ).

[0049] A2, coordinate conversion is performed on the first image coordinate of the first inspection target in the inspection image of the inspection device to obtain the first geographic position coordinate of the first inspection target.

[0050] The first geographic position coordinate is a coordinate in a world coordinate system. The coordinate in the world coordinate system can be a global positioning system (GPS) coordinate. The coordinate in the world coordinate system can be represented as (X, Y, Z). The first geographic position coordinate is (X t , Y t , Z t ).

[0051] Here, the first image coordinate of the first inspection target in the inspection image of the inspection device can be coordinate-converted by steps A21-A23:

[0052] A21, obtain the first device pose and the first device position coordinate of the inspection device.

[0053] The first device pose of the inspection device is the device pose when the inspection device obtains the inspection image. The first device pose can be represented by Euler angles. The first device pose can be represented as (γ, θ, φ). γ is the heading angle, which is the angle of rotation relative to the z-axis of the world coordinate system. θ is the roll angle, which is the angle of rotation relative to the x-axis of the world coordinate system. φ is the pitch angle, which is the angle of rotation relative to the y-axis of the world coordinate system.

[0054] The first device position coordinate of the inspection device is the position coordinate when the inspection device obtains the inspection image. The first device position coordinate is a coordinate in a world coordinate system. The first device position coordinate can be represented as (X0, Y0, Z0). X0, Y0, Z0 can represent the translation parameters of the inspection device relative to the origin of the world coordinate system when the inspection image is obtained. In the WGS-84 coordinate system as the world coordinate system of the GPS coordinate system, the origin of the world coordinate system can be the center of the Earth.

[0055] A22, determine the target conversion matrix according to the first device pose and the first device position coordinate of the inspection device.

[0056] The target conversion matrix is used to represent a conversion relationship between a pixel coordinate system and a world coordinate system when the inspection device acquires the inspection image. The target conversion matrix can include a first projection conversion matrix and a second projection conversion matrix. The first projection conversion matrix is used to represent a conversion relationship between a camera coordinate system and a device coordinate system. The second projection conversion matrix is used to represent a conversion relationship between the device coordinate system and the world coordinate system. The first projection conversion matrix can be represented as R1. Since the camera is fixed on the inspection device, the relative position relationship between the camera and the inspection device is fixed. The first conversion matrix can be obtained by pre-calibration. The second conversion matrix includes a rotation matrix R2 and a translation vector T. The rotation matrix R2 can be obtained according to a first device posture when the inspection device acquires the inspection image. The translation vector T is obtained based on a first device position coordinate.

[0057] A23, according to the target conversion matrix, performing coordinate conversion on the first image coordinate of the first inspection target in the inspection image of the inspection device to obtain the first geographic position coordinate.

[0058] Here, the first image coordinate of the first inspection target in the inspection image of the inspection device can be converted according to the following conversion formulas (1)-(3) to obtain the first geographic position coordinate:

[0059] P2=ZK-1P1 (1)

[0060] P3=R1 P2(2)

[0061] P4=R2 P3+T(3)

[0062] Wherein, K is a camera intrinsic matrix of a camera in the inspection device, P1 is a coordinate in a pixel coordinate system, P1 here can be (u t , v t , 1) mentioned above, P2 is a coordinate in a camera coordinate system, P3 is a coordinate in a device coordinate system of the inspection device, and P4 is a coordinate in a world coordinate system.

[0063] By means of target recognition and coordinate conversion, the geographic position of the inspection target can be accurately determined.

[0064] S202, displaying the first inspection target at the first geographic position in the inspection map.

[0065] Exemplarily, displaying the first inspection target at the first geographic position in the inspection map can be as shown in FIG. 2B. Figure 3 Figure 3 The d1 in FIG. 2B is the detected first inspection target.

[0066] ​Optionally, the first geographical coordinates of the first inspection target can also be displayed on the inspection map, for example, the first geographical coordinates are (X... t Y t Z t Then, at that first geographical location, the coordinates (X) of the first geographical location can be displayed. t Y t Z t This makes it easier for users to determine the specific geographical location of the inspection target.

[0067] exist Figure 2 In the corresponding technical solution, the inspection targets and their geographical locations within the inspection sub-area of ​​the inspection equipment are detected. Then, the inspection targets within the inspection sub-area are displayed at their corresponding geographical locations on the inspection map. Users can quickly determine the inspection targets and their geographical locations during the inspection process by viewing the inspection map, without requiring users to monitor the inspection screen in real time, making it simpler and more convenient.

[0068] See Figure 4 , Figure 4 This is a flowchart illustrating another inspection target detection and positioning method provided in an embodiment of this application, as shown below. Figure 4 As shown, the method includes the following steps:

[0069] S301, the first inspection target and the first geographical location of the first inspection target within the target inspection sub-area of ​​the inspection equipment.

[0070] For details on the specific implementation of step S301, please refer to the description of step S201 above, which will not be repeated here.

[0071] S302, Obtain the target inspection sub-area of ​​the inspection equipment.

[0072] Here is an introduction to the target inspection sub-area, which can be found in the description of step S201 above, and will not be repeated here.

[0073] In one feasible implementation, the target inspection sub-region of the inspection equipment can be obtained based on image recognition. Specifically, the inspection image of the inspection equipment can be acquired; the inspection map area corresponding to the inspection image of the inspection equipment can be determined as the target inspection sub-region.

[0074] The inspection map region corresponding to the inspection image is the region formed by mapping the image vertices of the inspection image to the inspection map. For example, refer to... Figure 5 Assuming the inspection image of the inspection equipment is as follows: Figure 5 As shown in P1, the image vertices of the inspection image are mapped to the regions formed in the inspection map, as shown in Figure P1. Figure 5P2 as shown in FIG. 2B, and the image content in P2 is an inspection map.

[0075] Wherein, the inspection map region corresponding to the inspection image can be determined by the following steps B1-B3.

[0076] B1, obtaining the vertex image coordinates of the inspection image of the inspection device.

[0077] Wherein, the vertex image coordinates can be coordinates in a pixel coordinate system, or coordinates in an image coordinate system. In the case of vertex image coordinates in the pixel coordinate system, the vertex image coordinates of the inspection image include (0, 0, 1), (0, H, 1), (W, 0, 1), (W, H, 1), which respectively represent the coordinates of the top left corner, the bottom left corner, the top right corner and the bottom right corner of the inspection image in the pixel coordinate system.

[0078] B2, performing coordinate conversion on the vertex image coordinates of the inspection image to obtain the inspection position coordinates corresponding to the inspection image.

[0079] Wherein, the inspection position coordinates are coordinates in a world coordinate system. The inspection position coordinates corresponding to the inspection image include (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), (X4, Y4, Z4), which respectively represent the coordinates of the top left corner, the bottom left corner, the top right corner and the bottom right corner of the inspection image in the world coordinate system.

[0080] The vertex image coordinates of the inspection image can be converted by steps B21-B23:

[0081] B21, obtaining the first device pose and the first device position coordinates of the inspection device.

[0082] B22, determining a target conversion matrix according to the first device pose and the first device position coordinates of the inspection device.

[0083] Here, the specific implementation of steps B21-B22 can refer to the description of the aforementioned steps A21-A22, which will not be described here.

[0084] B23, performing coordinate conversion on the vertex image coordinates of the inspection image according to the target conversion matrix to obtain the inspection position coordinates corresponding to the inspection image.

[0085] Here, the vertex image coordinates can be converted according to the conversion formulas (1)-(3) in the aforementioned step A23 to obtain the inspection position coordinates corresponding to the inspection image of the inspection device. P1 here can be (0, 0, 1), (0, H, 1), (W, 0, 1) or (W, H, 1).

[0086] A3. In the inspection map, the sub-area enclosed by the inspection position coordinates corresponding to the inspection image is determined as the inspection map area corresponding to the inspection image of the inspection equipment.

[0087] Specifically, four map points can be determined on the inspection map, corresponding to the coordinates of four inspection locations. These four map points can then be connected on the inspection map to obtain the inspection map area corresponding to the inspection image of the inspection equipment.

[0088] By acquiring images and performing coordinate transformations, the inspection sub-regions of the inspection equipment can be determined accurately.

[0089] Alternatively, the target inspection sub-area of ​​the inspection equipment can also be obtained based on methods such as lidar scanning or infrared scanning; this application does not impose any restrictions.

[0090] S303, In the inspection target set, delete the geographical location of the second inspection target and the second inspection target.

[0091] Here, the inspection target set includes all inspection targets detected by the inspection equipment during the inspection process and their geographical locations. The geographical locations of all inspection targets in the target set are represented using coordinates in the world coordinate system. In addition to the inspection targets and their geographical locations, the inspection target set can also include the categories of inspection targets. The inspection target set can include multiple subsets of inspection targets, and different subsets can include different categories of inspection targets.

[0092] The second inspection target is the inspection target located within the target inspection sub-region in the inspection target set. Deleting the second inspection target and its geographical location from the inspection target set serves to ensure the uniqueness of the inspection targets in the inspection target set.

[0093] Optionally, before deleting the geographical location of the second inspection target from the inspection target set, it can be determined whether the second inspection target exists in the inspection target set. If the second inspection target exists in the inspection target set, it means that the inspection target in the target inspection sub-region may have been inspected before, and the first inspection target may already exist in the inspection target set. Directly adding it may cause duplicate inspection targets in the inspection target set, so step S303 needs to be executed; if the second inspection target does not exist in the inspection target set, it means that the inspection target in the target inspection sub-region has not been inspected before, so step S303 can be skipped and step S304 can be executed directly.

[0094] In one feasible implementation, the existence of a second inspection target in the inspection target set can be determined through the following steps C1-C3:

[0095] C1, determine whether the second geographic location in the target inspection sub-region exists in the inspection target set.

[0096] Specifically, each geographic location in the inspection target set can be compared with the geographic location contained in the target inspection sub-region to determine whether each geographic location in the inspection target set is in the target inspection sub-region. If any geographic location in the inspection target set is in the target inspection sub-region, it means that the second geographic location in the target inspection sub-region exists in the inspection target set. If each geographic location in the inspection target set is not in the target inspection sub-region, it means that the second geographic location in the target inspection sub-region does not exist in the inspection target set, and step 304 can be directly executed.

[0097] C2, if the second geographic location exists in the inspection target set, determine that the second inspection target exists in the inspection target set.

[0098] In the case where the second inspection target exists in the inspection target set, it can also be determined whether the geographic location of the second inspection target is the same as the first geographic location. If the geographic location of the second inspection target is the same as the first geographic location, it means that the inspection target in the target inspection sub-region and the geographic location of the inspection target have not changed, and steps S303 and S304 can be skipped to directly execute step S305. If the geographic location of the second inspection target is different from the first geographic location, it means that the inspection target in the target inspection sub-region and / or the geographic location of the inspection target has changed, and S303 is executed.

[0099] C3, if the second geographic location does not exist in the inspection target set, determine that the second inspection target does not exist in the inspection target set.

[0100] By detecting whether the target location in the target inspection sub-region exists in the inspection target set, it is detected whether the inspection target exists in the inspection target set, and the implementation is simple.

[0101] Alternatively, other implementations can also be used to determine whether the second inspection target exists in the inspection target set.

[0102] S304, add the first inspection target and the first geographic location of the first inspection target to the inspection target set.

[0103] S305, display all inspection targets in the inspection target set at the geographic locations corresponding to all inspection targets in the inspection map.

[0104] Here, displaying all inspection targets in the inspection target set at the geographic locations corresponding to all inspection targets in the inspection map can be as shown in Figure 6 Figure 6 ​The region S in the region S is a target inspection sub-region, and the inspection target and the geographic position in the region S are a first inspection target and a first geographic position; and the inspection target and the geographic position outside the region S are an inspection target and a position that have been detected in the process of the inspection device performing an inspection task.

[0105] In the above Figure 4 In the corresponding technical solution, after obtaining the target inspection sub-region of the inspection device and detecting the geographic position of the inspection target in the target inspection sub-region, the inspection target located in the target inspection sub-region and the geographic position of the inspection target located in the target inspection sub-region are deleted from the inspection target set, and then the detected inspection target and the geographic position of the inspection target are added to the inspection target set, which can realize the deduplication of the same inspection target in the inspection target set and ensure the accurate statistics of the inspection target. Finally, all the inspection targets in the inspection target set are displayed at the corresponding geographic position in the inspection map, and the user can determine all the inspection targets and the positions of all the inspection targets in the entire inspection process according to the inspection map, thereby completing the global target statistics.

[0106] Further, the global target statistics include all the inspection targets in the inspection region in the inspection task, such as the types of the inspection targets in the region, such as the fixed obstacles in the inspection region, such as buildings, trees, power poles, towers, and the like, and the mobile targets such as vehicles, pedestrians, and the like. The user can select the type of the object to be displayed on the inspection map by filtering the types of the inspection targets, and can filter the specific inspection target to count the motion state and the motion trajectory of the mobile target.

[0107] The above describes the method of the application, and the device of the application is described below.

[0108] Referring to Figure 7 , Figure 7 is a structural schematic diagram of an inspection target detection and positioning device provided by an embodiment of the application. As Figure 7 shown, the inspection target detection and positioning device 40 includes:

[0109] The target detection module 401 is configured to detect a first inspection target in a target inspection sub-region of an inspection device and a first geographic position of the first inspection target, and the target inspection sub-region is a sub-region inspected by the inspection device in the process of performing an inspection task.

[0110] The target display module 402 is configured to display the first inspection target at the first geographic position in an inspection map.

[0111] In a possible design, the target display module 402 is specifically configured to: add the first inspection target and the first geographic position to an inspection target set, the inspection target set including all inspection targets and geographic positions of the all inspection targets that have been detected in a process in which the inspection device performs an inspection task; and display all inspection targets in the inspection target set at corresponding geographic positions of the all inspection targets in the inspection map.

[0112] In a possible design, the target display module 402 is further configured to: obtain the target inspection sub-region; and delete, in the inspection target set, a second inspection target and a geographic position of the second inspection target, the second inspection target being an inspection target in the inspection target set and located in the target inspection sub-region.

[0113] In a possible design, the target display module 402 is further configured to: determine whether a second geographic position in the target inspection sub-region exists in the inspection target set; if the second geographic position exists in the inspection target set, determine that the second inspection target exists in the inspection target set; and if the second geographic position does not exist in the inspection target set, determine that the second inspection target does not exist in the inspection target set.

[0114] In a possible design, the target display module 402 is specifically configured to: obtain an inspection image of the inspection device; and determine a target inspection sub-region corresponding to a target inspection image region of the inspection image as the target inspection sub-region.

[0115] In a possible design, the target display module 402 is specifically configured to: obtain a vertex image coordinate of the inspection image, the vertex image coordinate being a coordinate in a pixel coordinate system; perform coordinate conversion on the vertex image coordinate to obtain an inspection position coordinate corresponding to the inspection image, the inspection position coordinate being a coordinate in a world coordinate system; and determine, in the inspection map, a sub-region enclosed by the inspection position coordinate as the target inspection image region corresponding to the inspection image.

[0116] In a possible design, the target detection module 401 is specifically configured to: perform inspection target identification on an inspection image of the inspection device to obtain a first inspection target and a first image coordinate of the first inspection target in the inspection image; and perform coordinate conversion on the first image coordinate to obtain a first geographic position coordinate of the first inspection target, the first geographic position coordinate being a coordinate in a world coordinate system.

[0117] In a possible design, the target detection module 401 is specifically configured to: perform target identification on the inspection image of the inspection device by using a target identification model, to obtain the first inspection target and a first image coordinate of the first inspection target in the inspection image.

[0118] In a possible design, the target detection module 401 is specifically configured to: obtain a first device posture and a first device position coordinate of the inspection device, the first device posture being a device posture of the inspection device when the inspection image is acquired, the first device position coordinate being a geographic position coordinate of the inspection device when the inspection image is acquired, and the first device position coordinate being a coordinate in a world coordinate system; determine a target conversion matrix according to the first device posture and the first device position coordinate, the target conversion matrix being used to represent a conversion relationship between a pixel coordinate system and the world coordinate system when the inspection image is acquired by the inspection device; and perform coordinate conversion on the first image coordinate according to the target conversion matrix, to obtain the first geographic position coordinate.

[0119] It should be noted that, Figure 7 The content not mentioned in the corresponding embodiments can be referred to the description of the foregoing method embodiments, which will not be described here.

[0120] The apparatus described above can display the inspection target in the inspection sub-region of the inspection device and the geographic position of the inspection target, and then display the inspection target in the inspection sub-region at the corresponding geographic position of the inspection map, so that the user can quickly determine the inspection target and the geographic position of the inspection target in the process of performing the inspection task by the inspection device by checking the inspection map, without the need for the user to monitor the inspection picture in real time, and the method is more simple and convenient.

[0121] See Figure 8 , Figure 8 FIG. 1 is a structural schematic diagram of a computer device provided in an embodiment of the present application. The computer device 50 includes a processor 501 and a memory 502. The memory 502 is connected to the processor 501, for example, by being connected to the processor 501 through a bus.

[0122] The processor 501 is configured to support the computer device 50 to perform the corresponding functions in the methods in the above-mentioned method embodiments. The processor 501 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0123] The memory 502 is configured to store program codes and the like. The memory 502 can include a volatile memory (VM) such as a random access memory (RAM), and / or a non-volatile memory (NVM) such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), and / or a combination thereof.

[0124] The processor 501 can invoke the program codes to perform the following operations:

[0125] detect a first inspection target in a target inspection sub-region of the inspection device and a first geographic position of the first inspection target, the target inspection sub-region being a sub-region inspected by the inspection device during the execution of the inspection task;

[0126] display the first inspection target at the first geographic position in the inspection map.

[0127] The embodiments of the present application also provide a computer readable storage medium storing a computer program, the computer program including program instructions, the program instructions causing a computer to perform the method according to the above-mentioned embodiments when executed by the computer.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiment methods can be included. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like.

[0129] The above disclosure is only the preferred embodiment of the present application, and of course cannot limit the scope of the right of the present application, so the equivalent changes made according to the claims of the present application still belong to the scope covered by the present application.

Claims

1. A method for detecting and locating inspection targets, characterized in that, include: The first inspection target and the first geographical location of the first inspection target within the target inspection sub-area of ​​the inspection equipment are detected. The target inspection sub-area is the sub-area inspected by the inspection equipment during the execution of the inspection task. Display the first inspection target at the first geographical location on the inspection map; Displaying the first inspection target at the first geographical location on the inspection map includes: Obtain the target inspection sub-region; Determine whether a second geographical location exists within the target inspection sub-region of the inspection target set; If the second geographical location exists in the set of inspection targets, then it is determined that the second inspection target exists in the set of inspection targets. If the second geographical location does not exist in the set of inspection targets, then it is determined that the second inspection target does not exist in the set of inspection targets. If the second inspection target exists in the inspection target set, delete the second inspection target and its geographical location. The second inspection target is the inspection target located in the target inspection sub-region in the inspection target set. The first inspection target and the first geographical location are added to the inspection target set, which includes all inspection targets detected by the inspection equipment during the inspection task and the geographical locations of all inspection targets. Display all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map.

2. The method according to claim 1, characterized in that, The acquisition of the target inspection sub-region includes: Acquire inspection images from the inspection equipment; The inspection map area corresponding to the inspection image is determined as the target inspection sub-area.

3. The method according to claim 2, characterized in that, Before determining the inspection map area corresponding to the inspection image as the target inspection sub-region, the following steps are included: Obtain the vertex image coordinates of the inspection image, wherein the vertex image coordinates are coordinates in the pixel coordinate system; The vertex image coordinates are transformed to obtain the inspection position coordinates corresponding to the inspection image, where the inspection position coordinates are coordinates in the world coordinate system. In the inspection map, the sub-region enclosed by the inspection location coordinates is defined as the inspection map region corresponding to the inspection image.

4. The method according to claim 1, characterized in that, The first inspection target within the target inspection sub-area of ​​the inspection and monitoring equipment and the first geographical location of the first inspection target include: The inspection target is identified in the inspection image of the inspection equipment to obtain the first inspection target and the first image coordinates of the first inspection target in the inspection image; The first image coordinates are transformed to obtain the first geographical location coordinates of the first inspection target, which are coordinates in the world coordinate system.

5. The method according to claim 4, characterized in that, The step of identifying the inspection target in the inspection image of the inspection equipment to obtain the first inspection target and the first image coordinates of the first inspection target in the inspection image includes: By using a target recognition model, the inspection target of the inspection equipment is identified in the inspection image, and the first inspection target and the first image coordinates of the first inspection target in the inspection image are obtained.

6. The method according to claim 4, characterized in that, The step of performing coordinate transformation on the first image coordinates to obtain the first geographical location coordinates of the first inspection target includes: The first device posture and the first device position coordinates of the inspection equipment are obtained. The first device posture is the device posture when the inspection equipment acquires the inspection image. The first device position coordinates are the geographical coordinates of the inspection equipment when it acquires the inspection image. The first device position coordinates are coordinates in the world coordinate system. Based on the first device posture and the first device position coordinates, a target transformation matrix is ​​determined. The target transformation matrix is ​​used to represent the transformation relationship between the pixel coordinate system and the world coordinate system when the inspection device acquires the inspection image. Based on the target transformation matrix, the first image coordinates are transformed to obtain the first geographical location coordinates.

7. A target detection and positioning device for inspection, characterized in that, include: The target detection module is used to detect the first inspection target and the first geographical location of the first inspection target within the target inspection sub-area of ​​the inspection equipment. The target inspection sub-area is the sub-area inspected by the inspection equipment during the execution of the inspection task. The target display module is used to display the first inspection target at the first geographical location in the inspection map; The target display module is specifically used for: Obtain the target inspection sub-region; Determine whether a second geographical location exists within the target inspection sub-region of the inspection target set; If the second geographical location exists in the set of inspection targets, then it is determined that the second inspection target exists in the set of inspection targets. If the second geographical location does not exist in the set of inspection targets, then it is determined that the second inspection target does not exist in the set of inspection targets. If the second inspection target exists in the inspection target set, delete the second inspection target and its geographical location. The second inspection target is the inspection target located in the target inspection sub-region in the inspection target set. The first inspection target and the first geographical location are added to the inspection target set, which includes all inspection targets detected by the inspection equipment during the inspection task and the geographical locations of all inspection targets. Display all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map.

8. A computer device, characterized in that, The device includes a memory, a processor, and connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the computer device to perform the method as described in any one of claims 1-6 when executing the one or more computer programs.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-6.

Images