Inspection target detection and positioning method, device and equipment, and storage medium

Abstract

The application provides a target inspection detection positioning method, device and equipment and a storage medium. The method comprises the following steps: in a target inspection process, a target inspection sub-region of a target inspection device is acquired, and a first inspection target in the target sub-inspection region and a first geographical position of the first inspection target are detected. The target inspection process is a process in which a plurality of inspection devices perform an inspection task in an inspection region. A second inspection target in an inspection target set and a geographical position of the second inspection target are deleted. The second inspection target is an inspection target in the inspection target set located in the target sub-inspection region. The first inspection target and the first geographical position are added to the inspection target set. All inspection targets in the inspection target set are displayed at the geographical positions corresponding to all inspection targets in the inspection map. The technical solution can avoid repeated statistics of inspection targets in the inspection target set due to overlapping of regions covered by a plurality of inspection devices.

Description

Technical Field

[0001] This application relates to the field of target detection, and in particular to methods, devices, equipment and storage media for detecting and locating inspection targets. Background Technology

[0002] With the development of science and technology, more and more industries are choosing to use drones for inspections instead of traditional manual inspections. Drones are equipped with various tools, such as cameras and infrared sensors. After a flight mission is set for the drone, it transmits the images and videos it collects in real time to a backend system during the mission. Backend staff then monitor the inspection footage in real time, identify targets in the images, and thus achieve remote inspection.

[0003] Cluster inspection schemes involve using multiple drones to fly over an inspection area and transmit images and videos to a backend system to complete the inspection task. Because the areas covered by multiple drones overlap, targets may be counted repeatedly at different locations and / or at different times, leading to statistical bias. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, and storage medium for detecting and locating inspection targets, in order to solve the technical problem of statistical bias caused by repeated counting of targets.

[0005] Firstly, a method for detecting and locating inspection targets is provided, including:

[0006] During the target inspection process, the target inspection sub-area of ​​the target inspection device is acquired, and the first inspection target and the first geographical location of the first inspection target within the target sub-inspection area are detected. The target inspection process is the process by which the multiple inspection devices perform inspection tasks in the inspection area. The target inspection device is any one of the multiple inspection devices. The target inspection sub-area is the sub-area inspected by the target inspection device and is part of the inspection area.

[0007] Delete the second inspection target and its geographical location from the inspection target set. The inspection target set includes all inspection targets detected by the multiple inspection devices during the target inspection process and the geographical locations of all inspection targets. The second inspection target is the inspection target located in the target sub-inspection area of ​​the inspection target set.

[0008] Add the first inspection target and the first geographical location to the inspection target set;

[0009] Display all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map.

[0010] In this technical solution, during the process of multiple inspection devices performing tasks in the inspection area, after acquiring the target inspection sub-area of ​​the inspection device and detecting the inspection targets and their geographical locations within the target inspection sub-area, the inspection targets and their geographical locations within the target inspection sub-area are first deleted from the inspection target set. Then, the detected inspection targets and their geographical locations are added back to the inspection target set. This ensures that the geographical locations of the inspection targets and their geographical locations within the target inspection sub-area in the inspection target set are unique, avoiding duplicate counting of inspection targets in the inspection target set due to overlapping areas covered by multiple inspection devices. This helps improve the accuracy of inspection target statistics.

[0011] In conjunction with the first aspect, in one possible implementation, the target inspection device is the i-th inspection device among the plurality of inspection devices, 1≤i≤N, where N is the number of the plurality of inspection devices; after adding the first inspection target and the first geographical location to the inspection target set, the method further includes: taking the (i+1)-th inspection device among the plurality of inspection devices as the target inspection device and performing the step of obtaining the target inspection sub-region of the target inspection device. By performing inspection area detection and target identification and localization on each of the plurality of inspection devices, global statistics on the inspection area can be achieved.

[0012] In conjunction with the first aspect, in one possible implementation, the method further includes: displaying the inspection sub-areas inspected by each of the multiple inspection devices in the inspection map using different colors. Displaying the inspection sub-areas inspected by each of the multiple inspection devices using different colors helps users distinguish the routes and areas inspected by the inspection devices.

[0013] In conjunction with the first aspect, in one possible implementation, before deleting the second inspection target and its geographical location from the inspection target set, the method further includes: determining whether a second geographical location exists within the target inspection sub-region in the inspection target set; if the second geographical location exists in the inspection target set, then it is determined that the second inspection target exists in the inspection target set; if the second geographical location does not exist in the inspection target set, then it is determined that the second inspection target does not exist in the inspection target set. Detecting the existence of an inspection target within an inspection sub-region by detecting whether a geographical location exists within the inspection target set is also simple.

[0014] In conjunction with the first aspect, in one possible implementation, obtaining the target inspection sub-region of the target inspection device includes: obtaining the target inspection image of the target inspection device; and determining the inspection map region corresponding to the target inspection image as the target inspection sub-region.

[0015] In conjunction with the first aspect, in one possible implementation, before determining the inspection map region corresponding to the target inspection image as the target inspection sub-region, the method includes: obtaining the vertex image coordinates of the target inspection image, wherein the vertex image coordinates are coordinates in the pixel coordinate system; performing coordinate transformation on the vertex image coordinates to obtain the inspection position coordinates corresponding to the target inspection image, wherein the inspection position coordinates are coordinates in the world coordinate system; and determining the sub-region enclosed by the inspection position coordinates in the inspection map as the inspection map region corresponding to the target inspection image. By obtaining images and performing coordinate transformation to determine the inspection sub-region of the inspection equipment, the inspection sub-region of the inspection equipment can be accurately determined.

[0016] In conjunction with the first aspect, in one possible implementation, detecting the first inspection target and the first geographical location of the first inspection target within the target sub-inspection area includes: performing inspection target recognition on the target inspection image to obtain the first inspection target and its first image coordinates in the target inspection image; and performing coordinate transformation on the first image coordinates to obtain the first geographical location coordinates of the first inspection target, wherein the first geographical location coordinates are coordinates in the world coordinate system. Determining the geographical location of the inspection target through target recognition and coordinate transformation allows for accurate determination of the inspection target's geographical location.

[0017] In conjunction with the first aspect, in one possible implementation, the step of performing inspection target identification on the target inspection image to obtain the first inspection target and the first image coordinates of the first inspection target in the target inspection image includes: performing inspection target identification on the target inspection image through a target identification model to obtain the first inspection target and the first image coordinates of the first inspection target in the target inspection image.

[0018] In conjunction with the first aspect, in one possible implementation, the step of performing coordinate transformation on the first image coordinates to obtain the first geographic location coordinates of the first inspection target, wherein the first geographic location coordinates are coordinates in the world coordinate system, includes: acquiring the first device posture and the first device position coordinates of the target inspection device, wherein the first device posture is the device posture of the target inspection device when acquiring the target inspection image, and the first device position coordinates are the geographic location coordinates of the target inspection device when acquiring the target inspection image, wherein the first device position coordinates are coordinates in the world coordinate system; determining a target transformation matrix based on the first device posture and the first device position coordinates, wherein the target transformation matrix is ​​used to represent the transformation relationship between the pixel coordinate system and the world coordinate system when the target inspection device acquires the target inspection image; and performing coordinate transformation on the first image coordinates based on the target transformation matrix to obtain the first geographic location coordinates.

[0019] Secondly, a target detection and positioning device for inspection is provided, comprising:

[0020] The target detection module is used to acquire the target inspection sub-area of ​​the target inspection device during the target inspection process, and detect the first inspection target and the first geographical location of the first inspection target within the target sub-inspection area. The target inspection process is the process in which the multiple inspection devices perform inspection tasks in the inspection area. The target inspection device is any one of the multiple inspection devices. The target inspection sub-area is the sub-area inspected by the target inspection device and is part of the inspection area.

[0021] The target deletion module is used to delete the second inspection target and the geographical location of the second inspection target from the inspection target set. The inspection target set includes all inspection targets detected by the plurality of inspection devices during the target inspection process and the geographical locations of all the inspection targets. The second inspection target is the inspection target located in the target sub-inspection area of ​​the inspection target set.

[0022] The target saving module is used to add the first inspection target and the first geographical location to the inspection target set;

[0023] The target display module is used to display all inspection targets in the inspection target set at the corresponding geographical locations of all inspection targets in the inspection map.

[0024] Thirdly, a computer device is provided, including a memory and one or more processors, the memory being connected to the one or more processors, the one or more processors being configured to execute one or more computer programs stored in the memory, and the one or more processors, when executing the one or more computer programs, enabling the computer device to implement the inspection target detection and positioning method of the first aspect described above.

[0025] Fourthly, a computer-readable storage medium is provided, which stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the inspection target detection and positioning method of the first aspect.

[0026] This application can achieve the following technical effects: it can make the geographical location of the inspection targets located in the target inspection sub-region of the inspection target set unique, avoid the double counting of inspection targets in the inspection target set due to the overlapping areas covered by multiple inspection devices, and thus help improve the accuracy of inspection target statistics. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a cluster inspection system provided in an embodiment of this application;

[0028] Figure 2 A schematic diagram illustrating the repeated detection of an inspection target as provided in an embodiment of this application;

[0029] Figure 3 A flowchart illustrating a patrol target detection and positioning method provided in an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of the target inspection image and the corresponding inspection map area provided in the embodiments of this application.

[0031] Figure 5 This is a schematic diagram showing the inspection targets and their locations on an inspection map, provided as an embodiment of this application.

[0032] Figure 6 A flowchart illustrating another inspection target detection and positioning method provided in this application embodiment;

[0033] Figure 7 This is a schematic diagram of the structure of an inspection target detection and positioning device provided in an embodiment of this application;

[0034] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0036] The technical solution of this application is applicable to cluster inspection scenarios. Cluster inspection scenarios can refer to scenarios in which multiple drones fly in the same area to perform inspection tasks, or scenarios in which other inspection equipment is used to perform inspection tasks, such as scenarios in which multiple inspection robots walk on the ground to perform ground inspection tasks.

[0037] To facilitate understanding, the cluster inspection system involved in this application will first be introduced. See [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram of a cluster inspection system provided in an embodiment of this application, as shown below. Figure 1 As shown, the cluster inspection system 10 may include multiple inspection devices 101 and an inspection control platform 102. The multiple inspection devices 101 are used to replace humans in moving within the area to be inspected (i.e., the inspection area), collecting relevant data (image data and various sensor data) in the inspection area in real time, and transmitting the collected data to the inspection control platform 102 to complete the inspection task. The inspection control platform 102 is a user-facing device. It is used to issue inspection instructions to the multiple inspection devices 101, instructing them to perform inspection tasks according to the instructions. The area covered by the multiple inspection devices 101 performing the inspection task is the inspection area. The inspection control platform 102 can respond to user commands, sending inspection instructions to the multiple inspection devices 101, enabling the multiple inspection devices 101 to perform inspection tasks under user control. The inspection devices 101 may be drones or inspection robots, etc.; the inspection control platform 102 may include a cloud platform, a backend server, and human-interactive devices (such as displays).

[0038] To ensure that multiple inspection devices 101 can completely cover the inspection area, such as Figure 2 As shown, the areas covered by multiple inspection devices 101 during the execution of inspection tasks may partially overlap. In this case, during the execution of inspection tasks, different inspection devices 101 may inspect the same target at different times or at the same time, resulting in the target being counted repeatedly and causing statistical bias. The technical solution of this application aims to solve this technical problem.

[0039] The technical solution of this application can be applied to: Figure 1 The inspection control platform in the cluster inspection system shown below. The technical solution of this application is described in detail below.

[0040] See Figure 3 , Figure 3This is a flowchart illustrating a method for detecting and locating inspection targets provided in an embodiment of this application, as shown below. Figure 3 As shown, the method includes the following steps:

[0041] S201, during the target inspection process, acquire the target inspection sub-area of ​​the target inspection equipment.

[0042] The target inspection process involves multiple inspection devices performing inspection tasks within an inspection area. These multiple devices can form a cluster inspection system, which can be configured as follows: Figure 1 As shown. The target inspection equipment can be any one of multiple inspection equipment. The target inspection process can specifically be a road inspection process, a power line inspection process, a security inspection process, etc., and is not limited to the examples shown here.

[0043] The target inspection sub-area is the sub-area inspected by the target inspection equipment during the target inspection process. The target inspection sub-area can be understood as the maximum area detectable by the sensors on the target inspection equipment when it performs its task. The target inspection sub-area is part of the inspection area of ​​multiple inspection equipment. The inspection areas of multiple inspection equipment are related to the inspection tasks performed by the multiple inspection equipment. Different inspection tasks result in different inspection areas for the multiple inspection equipment. For example, if the inspection task of multiple inspection equipment is to inspect the entire campus area, then the inspection area of ​​the multiple inspection equipment is the entire campus area, and the target inspection sub-area could be classroom areas, playground areas, office areas, etc., within the campus. Both the target inspection sub-area and the inspection areas of multiple inspection equipment can be represented on the inspection map, which is the map used during the inspection process by the multiple inspection equipment. The location of the inspection equipment can be displayed on the inspection map.

[0044] In some possible scenarios, the target inspection sub-region of the target inspection device can be obtained based on image recognition. This involves acquiring the target inspection image, which is the inspection image of the target inspection device; and defining the inspection map area corresponding to the target inspection image as the target inspection sub-region. The target inspection image can be understood as the inspection image acquired by the target inspection device when performing an inspection task.

[0045] Target inspection images can be captured by cameras in target inspection equipment.

[0046] The inspection map region corresponding to the target inspection image is the region formed by mapping the image vertices of the target inspection image onto the inspection map. For example, refer to... Figure 4 Assuming the target inspection image is as follows: Figure 4 As shown in P1, the image vertices of the target inspection image are mapped to the regions formed in the inspection map, as shown in Figure P1. Figure 4As shown in P2, the image content in P2 is the inspection map.

[0047] In one feasible implementation, the inspection map area corresponding to the target inspection image can be determined through the following steps A1-A3.

[0048] A1. Obtain the vertex image coordinates of the target inspection image.

[0049] The vertex image coordinates can be either pixel coordinates or image coordinates. Image coordinates can be obtained by transforming pixel coordinates, and the transformation relationship between image coordinates and pixel coordinates is as follows: x = (uu...) o )*W, y=(vv o )*H, (x, y) represent the coordinates in the image coordinate system, (u, v) represent the coordinates in the pixel coordinate system, W and H are the width and height of the target inspection image, respectively, (u*H ...*H*H*H*H*H*H* o (vo) represents the pixel coordinates of the center of the target inspection image. When the vertex image coordinates are in pixel coordinates, the vertex image coordinates of the target inspection image include (0, 0), (0, H), (W, 0), and (W, H), representing the coordinates of the top-left, bottom-left, top-right, and bottom-right corners of the target inspection image in pixel coordinates, respectively.

[0050] A2. Perform coordinate transformation on the vertex image coordinates of the target inspection image to obtain the inspection position coordinates corresponding to the target inspection image.

[0051] The inspection location coordinates are in the world coordinate system, which can be represented as (X, Y, Z) using the Global Positioning System (GPS). The inspection location coordinates corresponding to the target inspection image include (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3), and (X4, Y4, Z4), representing the coordinates of the top-left, bottom-left, top-right, and bottom-right corners of the target inspection image in the world coordinate system, respectively.

[0052] The vertex image coordinates of the target inspection image can be transformed through steps A21-A23:

[0053] A21. Obtain the first equipment attitude and first equipment position coordinates of the target inspection equipment.

[0054] Among them, the first device attitude of the target inspection equipment is the device attitude when the target inspection equipment acquires the target inspection image. The first device attitude can be represented by Euler angles. The first device attitude can be represented as (γ, θ, φ), where γ is the heading angle, which is the angle of rotation relative to the z-axis of the world coordinate system, θ is the roll angle, which is the angle of rotation relative to the x-axis of the world coordinate system, and φ is the pitch angle, which is the angle of rotation relative to the y-axis of the world coordinate system.

[0055] The first device position coordinates of the target inspection equipment are the position coordinates of the target inspection equipment when acquiring the target inspection image. The first device position coordinates are coordinates in the world coordinate system and can be represented as (X0, Y0, Z0). X0, Y0, and Z0 can represent the translation parameters of the inspection equipment relative to the origin of the world coordinate system when acquiring the target inspection image. In the world coordinate system using the WGS-84 coordinate system as the GPS coordinate system, the origin of the world coordinate system can be the Earth's center of mass.

[0056] During the target inspection process, multiple inspection devices report inspection images, as well as their attitude and position at the time of image capture, to the inspection control platform. Based on the inspection images, attitudes, and positions reported by the multiple inspection devices, the inspection control platform can obtain the target inspection image, the first device attitude, and the first device position coordinates of the target inspection device.

[0057] A22. Determine the target transformation matrix based on the first equipment attitude and the first equipment position coordinates of the target inspection equipment.

[0058] The target transformation matrix represents the transformation relationship between the pixel coordinate system and the world coordinate system when the target inspection device acquires the target inspection image. The target transformation matrix may include a first projection transformation matrix and a second projection transformation matrix. The first projection transformation matrix represents the transformation relationship between the camera coordinate system and the device coordinate system, and the second projection transformation matrix represents the transformation relationship between the device coordinate system and the world coordinate system. The first projection transformation matrix can be represented as R1. Since the camera is fixed on the inspection device, the relative positional relationship between the camera and the inspection device is fixed, and the first transformation matrix can be obtained through pre-calibration. The second transformation matrix includes a rotation matrix R2 and a translation vector T. The rotation matrix R2 can be obtained based on the first device posture when the inspection device acquires the target inspection image, and the translation vector T is obtained based on the first device position coordinates.

[0059] A23. Based on the target transformation matrix, perform coordinate transformation on the vertex image coordinates of the target inspection image to obtain the inspection position coordinates corresponding to the target inspection image.

[0060] Here, the vertex coordinates of the target inspection image can be transformed according to the following transformation formulas (1)-(3) to obtain the inspection position coordinates corresponding to the target inspection image:

[0061] P2 = ZK -1 P1 (1)

[0062] P3 = R1 P2(2)

[0063] P4 = R2P3 + T(3)

[0064] Where K is the camera intrinsic parameter matrix of the camera in the inspection equipment, P1 is the coordinate in the pixel coordinate system, where P1 can be (0, 0, 1), (0, H, 1), (W, 0, 1), (W, H, 1), P2 is the coordinate in the camera coordinate system, P3 is the coordinate in the equipment coordinate system of the inspection equipment, and P4 is the coordinate in the world coordinate system.

[0065] A3. In the inspection map, the sub-region enclosed by the inspection position coordinates corresponding to the target inspection image is determined as the inspection map area corresponding to the target inspection image.

[0066] Specifically, four map points can be determined on the inspection map, corresponding to the coordinates of four inspection locations. These four map points can then be connected on the inspection map to obtain the inspection map area corresponding to the target inspection image.

[0067] By acquiring images and performing coordinate transformations, the inspection sub-regions of the inspection equipment can be determined accurately.

[0068] Alternatively, the target inspection sub-area of ​​the inspection equipment can also be obtained based on methods such as lidar scanning or infrared scanning; this application does not impose any restrictions.

[0069] S202, the first inspection target and the first geographical location of the first inspection target within the inspection sub-area of ​​the detection target.

[0070] The inspection target refers to the target that multiple inspection devices need to acquire / detect when performing inspection tasks. The inspection targets will differ depending on the inspection tasks performed by the multiple inspection devices. For example, if the inspection task performed by multiple inspection devices is a road inspection, the inspection targets could be pedestrians, vehicles, traffic light signs, etc. Similarly, if the inspection task performed by multiple inspection devices is a power line inspection, the inspection targets could be utility poles. This application does not impose any limitations.

[0071] The first inspection target refers to the inspection target located within the target inspection sub-area when the target inspection equipment is performing its task. The first geographical location of the first inspection target refers to its geographical location on the inspection map, which is represented by coordinates in the world coordinate system.

[0072] In one feasible implementation, the first inspection target within the target inspection sub-region and the first geographical location of the first inspection target within the target inspection sub-region can be detected based on image recognition and target localization.

[0073] Specifically, the first inspection target and the first geographical location of the first inspection target can be detected through the following steps B1-B2.

[0074] B1. Perform inspection target identification on the target inspection image to obtain the first inspection target and the first image coordinates of the first inspection target in the target inspection image.

[0075] In one possible implementation, a target recognition model can be used to identify the inspection target in the target inspection image, thereby obtaining the first inspection target and its first image coordinates in the target inspection image.

[0076] The target recognition model can be a model trained based on deep learning technology for recognizing inspection targets. The target recognition model includes, but is not limited to, region convolutional neural networks (RCNN) models, YOLO (you only look once), SSD (single shot multibox detector) models, etc., and this application does not impose any restrictions.

[0077] After the target inspection image is input into the target recognition model, the target recognition model can output the information of the inspection target in the target inspection image {u t v t The first inspection target is the inspection target in the target inspection image, where w, h, c}. t v t Let be the pixel coordinates of the center point of the first inspection target in the target inspection image, w and h be the width and height of the first inspection target in the target inspection image, and c be the category of the first inspection target. For example, if multiple inspection devices perform a road inspection task, the category of the first inspection target can be one of the targets on the road, such as pedestrians, vehicles, or traffic light signs.

[0078] It should be understood that (u t v t It can also be converted to coordinates in the image coordinate system (x) t y t The first image coordinates of the first inspection target in the current target inspection image can be (u t v t ), or (xt y t ).

[0079] B2. Perform coordinate transformation on the first image coordinates of the first inspection target in the target inspection image to obtain the first geographical location coordinates of the first inspection target.

[0080] The first geographical location coordinates are the coordinates in the world coordinate system. The first geographical location coordinates are (X... t Y t Z t ).

[0081] Here, the coordinate transformation of the first image coordinates of the first inspection target in the current target inspection image can be performed through steps B21-B23:

[0082] B21. Obtain the first equipment attitude and the first equipment position coordinates of the inspection equipment.

[0083] B22. Determine the target transformation matrix based on the first equipment attitude and the first equipment position coordinates of the inspection equipment.

[0084] For details on the specific implementation of steps B21-B22, please refer to the descriptions of steps A21-A22 above; they will not be repeated here.

[0085] B23. Based on the target transformation matrix, perform coordinate transformation on the first image coordinates of the first inspection target in the target inspection image to obtain the first geographical location coordinates.

[0086] Specifically, the first image coordinates of the first inspection target in the target inspection image can be transformed according to the transformation formulas (1)-(3) in step A23 to obtain the first geographical location coordinates. Here, P1 is the first image coordinate of the first inspection target in the target inspection image, and the first image coordinate is (u t v t ,1).

[0087] S203, Delete the geographical location of the second inspection target and the second inspection target from the inspection target set.

[0088] Here, the inspection target set includes all inspection targets detected by multiple inspection devices during the target inspection process and the geographical locations of all inspection targets. The geographical locations of all inspection targets in the inspection target set are represented by coordinates in the world coordinate system. In addition to the inspection targets and their geographical locations, the inspection target set may also include the categories of inspection targets. The inspection target set may include multiple subsets of inspection targets, and different subsets may include different categories of inspection targets.

[0089] The second inspection target is the inspection target located within the target inspection sub-region of the inspection target set. Deleting the second inspection target from the inspection target set and the geographical location of the second inspection target serves to ensure the uniqueness of the inspection targets in the inspection target set.

[0090] Optionally, before deleting the second inspection target and its geographical location from the inspection target set, it is possible to first determine whether the second inspection target exists in the inspection target set. If the second inspection target exists in the inspection target set, it means that other inspection devices have already inspected the inspection target within the target inspection sub-area. The first inspection target may already exist in the inspection target set, and adding it directly would result in duplicate inspection targets in the inspection target set, so step S204 needs to be executed. If the second inspection target does not exist in the inspection target set, it means that other inspection devices have not inspected the inspection target within the target inspection sub-area, so step S203 can be skipped and step S204 can be executed directly.

[0091] In one feasible implementation, the existence of a second inspection target in the inspection target set can be determined through the following steps C1-C3:

[0092] C1. Determine whether there is a second geographical location within the target inspection sub-region in the inspection target set.

[0093] Specifically, each geographical location in the inspection target set can be compared with the geographical locations included in the target inspection sub-area to determine whether each geographical location in the inspection target set is within the target inspection sub-area. If any geographical location in the inspection target set is within the target inspection sub-area, it indicates that there is a second geographical location within the target inspection sub-area in the inspection target set. If none of the geographical locations in the inspection target set are within the target inspection sub-area, it indicates that there is no second geographical location within the target inspection sub-area in the inspection target set, and step S204 can be executed directly.

[0094] C2. If a second geographical location exists in the set of inspection targets, then it is determined that a second inspection target exists in the set of inspection targets.

[0095] If a second inspection target exists in the inspection target set, it can be determined whether the geographical location of the second inspection target is the same as that of the first geographical location. If the geographical location of the second inspection target is the same as that of the first geographical location, it means that the inspection targets and their geographical locations within the target inspection sub-area have not changed, and steps S203 and S204 can be skipped, and step S205 can be executed directly. If the geographical location of the second inspection target is different from that of the first geographical location, it means that the inspection targets and / or their geographical locations within the target inspection sub-area have changed, and step S203 should be executed.

[0096] C3. If there is no second geographical location in the set of inspection targets, then it is determined that there is no second inspection target in the set of inspection targets.

[0097] The detection method is simple: it detects whether there is a target within the target inspection sub-region in the target inspection target set.

[0098] Alternatively, other implementation methods can be used to determine whether a second inspection target exists in the inspection target set.

[0099] S204, add the first inspection target and the first geographical location of the first inspection target to the inspection target set.

[0100] S205 displays all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map.

[0101] Here, displaying all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map can be done as follows: Figure 5 As shown, Figure 5 There are four inspection devices performing inspection tasks in the same inspection area.

[0102] In the above Figure 3 In the corresponding technical solution, during the process of multiple inspection devices performing tasks in the inspection area, after obtaining the target inspection sub-area of ​​the inspection device and detecting the inspection targets and their geographical locations within the target inspection sub-area, the inspection targets and their geographical locations within the target inspection sub-area are first deleted from the inspection target set. Then, the detected inspection targets and their geographical locations are added back to the inspection target set. This ensures that the geographical locations of the inspection targets and their geographical locations within the target inspection sub-area in the inspection target set are unique, avoiding duplicate counting of inspection targets in the inspection target set due to overlapping areas covered by multiple inspection devices. This helps improve the accuracy of inspection target statistics.

[0103] See Figure 6 , Figure 6 This is a flowchart illustrating another inspection target detection and positioning method provided in an embodiment of this application, as shown below. Figure 6 As shown, the method includes the following steps:

[0104] S301, during the target inspection process, the i-th inspection device among multiple inspection devices is taken as the target inspection device.

[0105] S302, Obtain the target inspection sub-area of ​​the target inspection equipment.

[0106] S303, the first inspection target and the first geographical location of the first inspection target within the inspection sub-area of ​​the detection target.

[0107] S304, Delete the geographical location of the second inspection target in the inspection target set.

[0108] S305, add the first inspection target and the first geographical location of the first inspection target to the inspection target set.

[0109] S306 displays all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map.

[0110] The specific implementation of steps S302 to S306 can be found in the description of steps S201 to S205 above, and will not be repeated here.

[0111] S307, determine if i is equal to N.

[0112] Where N represents the number of multiple inspection devices.

[0113] If i equals N, it means that all the inspection devices in the multiple inspection devices have been traversed, and the next round of traversal can be carried out. Step S308 is executed. If i is less than N, it means that all the devices in the multiple inspection devices have not been traversed. Increment i by 1 and return to execute step S301.

[0114] S308, set i to 1, and return to step S301.

[0115] When i is set to 1, if the timer period is reached, the execution can return to step S301. The timer period can be set to 1 second for example.

[0116] In the above Figure 6 In the corresponding technical solution, by periodically performing area detection and target detection on each of the multiple inspection devices, global statistics of the inspection area can be achieved.

[0117] The following describes the process of four inspection devices working together in the inspection area, with N equal to four. Figure 6 The following is an introduction to the scheme. Assuming that there are four inspection devices, namely inspection device x1, inspection device x2, inspection device x3 and inspection device x4, the inspection process may include the following steps:

[0118] (1) Obtain the inspection sub-region q1 of the inspection equipment x1, and detect the inspection target d11 and the geographical location l11 of the inspection target d11 within the inspection sub-region q1. The inspection sub-region q1 of the inspection equipment x1 can be obtained in the manner described in the aforementioned step S201, and the inspection target d11 and the geographical location l11 within the inspection sub-region q1 can be detected in the manner described in the aforementioned step S202.

[0119] (2) Delete the inspection target d12 located in the inspection sub-region q1 and the geographical location l12 of the inspection target d12 in the inspection target set Q. The existence of the inspection target d12 in the inspection target set can be determined by the method described in the previous steps C1 to C2.

[0120] (3) Add the inspection target d11 and the geographical location l11 of the inspection target d11 to the inspection target set Q.

[0121] (4) Obtain the inspection sub-region q2 of the inspection equipment x2, and detect the inspection target d21 and the geographical location l21 of the inspection target d21 within the inspection sub-region q2. The inspection sub-region q2 of the inspection equipment x2 can be obtained in the manner described in the aforementioned step S201, and the inspection target d21 and the geographical location l21 of the inspection target d21 within the inspection sub-region q2 can be detected in the manner described in the aforementioned step S202.

[0122] (5) Delete the inspection target d22 located in the inspection sub-region q2 and the geographical location l22 of the inspection target d22 in the inspection target set Q. The existence of the inspection target d22 in the inspection target set can be determined by the method described in the previous steps C1 to C2.

[0123] (6) Add the inspection target d21 and the geographical location l21 of the inspection target d21 to the inspection target set Q.

[0124] (7) Obtain the inspection sub-region q3 of the inspection equipment x3, and detect the inspection target d31 and the geographical location l31 of the inspection target d31 within the inspection sub-region q3. The inspection sub-region q3 of the inspection equipment x3 can be obtained in the manner described in the aforementioned step S201, and the inspection target d31 and the geographical location l31 of the inspection target d31 within the inspection sub-region q3 can be detected in the manner described in the aforementioned step S202.

[0125] (8) Delete the inspection target d32 located in the inspection sub-region q3 and the geographical location l32 of the inspection target d32 in the inspection target set Q. The existence of the inspection target d32 in the inspection target set can be determined by the method described in the previous steps C1 to C2.

[0126] (9) Add the inspection target d31 and the geographical location l31 of the inspection target d31 to the inspection target set Q.

[0127] (10) Obtain the inspection sub-region q4 of the inspection equipment x4, and detect the inspection target d41 and the geographical location l41 of the inspection target d41 within the inspection sub-region q4. The inspection sub-region q4 of the inspection equipment x4 can be obtained in the manner described in the aforementioned step S201, and the inspection target d41 and the geographical location l41 of the inspection target d41 within the inspection sub-region q4 can be detected in the manner described in the aforementioned step S202.

[0128] (11) Delete the inspection target d42 located in the inspection sub-region q4 and the geographical location l42 of the inspection target d42 in the inspection target set Q. The presence of inspection target d42 in the inspection target set can be determined by the method described in steps C1 to C2 above.

[0129] (12) Add the inspection target d41 and the geographical location l41 of the inspection target d41 to the inspection target set Q.

[0130] Through the above steps (1) to (12), the traversal of inspection equipment 1 to inspection equipment 4 can be completed; then, step (1) can be returned to perform a new round of traversal to complete the global statistics of the inspection targets in the inspection area.

[0131] Optionally, in some possible cases, the number of inspection target sets can be multiple, and the number of inspection target sets can be equal to the number of inspection devices. Each inspection target set is used to store the inspection targets and their geographical locations detected by one inspection device during the target inspection process. After obtaining the target inspection sub-region of the target inspection device and the inspection targets and their geographical locations within that sub-region, steps C1-C2 can be referred to to determine whether the geographical location of the inspection targets in each inspection target set is located within that target inspection sub-region. If the geographical location of the inspection targets in the first inspection target set is located within that target inspection sub-region, the inspection targets and their geographical locations within that sub-region can be deleted from the first inspection target set. Then, the newly obtained inspection targets and their geographical locations within that sub-region can be added to the inspection target set corresponding to the target inspection device. This ensures that the inspection targets in the inspection target sets do not overlap, achieving deduplication. By using different sets of inspection targets to store the inspection targets detected by different inspection devices and their geographical locations, it is convenient to subsequently determine the inspection sub-areas and inspection targets inspected by each inspection device.

[0132] Optionally, in some possible cases, when displaying inspection targets in the inspection target set on the inspection map, the inspection sub-areas inspected by multiple inspection devices can also be displayed in different colors on the inspection map. Different colors can be used to display the inspection sub-areas inspected by different inspection devices.

[0133] For example, for Figure 5 The four inspection devices can each display the inspection sub-areas inspected by the four inspection devices using four different colors.

[0134] By displaying the inspection sub-areas inspected by multiple inspection devices using different colors, users can distinguish the routes and areas inspected by the inspection devices.

[0135] The method of this application has been described above; the apparatus of this application will be described below.

[0136] See Figure 7 , Figure 7 This is a schematic diagram of the structure of a patrol target detection and positioning device provided in an embodiment of this application. Figure 7 As shown, the inspection target detection and positioning device 40 includes:

[0137] The target detection module 401 is used to acquire the target inspection sub-area of ​​the target inspection device during the target inspection process, and detect the first inspection target and the first geographical location of the first inspection target within the target sub-inspection area. The target inspection process is the process in which the multiple inspection devices perform inspection tasks in the inspection area. The target inspection device is any one of the multiple inspection devices. The target inspection sub-area is the sub-area inspected by the target inspection device and is part of the inspection area.

[0138] The target deletion module 402 is used to delete the second inspection target and the geographical location of the second inspection target from the inspection target set. The inspection target set includes all inspection targets detected by the plurality of inspection devices during the target inspection process and the geographical locations of all inspection targets. The second inspection target is the inspection target located in the target sub-inspection area of ​​the inspection target set.

[0139] The target storage module 403 is used to add the first inspection target and the first geographical location to the inspection target set;

[0140] The target display module 404 is used to display all the inspection targets in the inspection target set at the geographical locations corresponding to all the inspection targets in the inspection map.

[0141] In one possible design, the target detection module 401 is further configured to: take the (i+1)th inspection device among the plurality of inspection devices as the target inspection device, and execute the step of obtaining the target inspection sub-region of the target inspection device.

[0142] In one possible design, the target display module 404 is also used to display the inspection sub-areas inspected by each of the multiple inspection devices in the inspection map by color.

[0143] In one possible design, the target deletion module 402 is further configured to: determine whether there is a second geographical location located within the target inspection sub-area in the inspection target set; if there is a second geographical location in the inspection target set, then determine that there is a second inspection target in the inspection target set; if there is no second geographical location in the inspection target set, then determine that there is no second inspection target in the inspection target set.

[0144] In one possible design, the target detection module 401 is specifically used to: acquire a target inspection image, wherein the target inspection image is the inspection image of the target inspection device; and determine the inspection map area corresponding to the target inspection image as the target inspection sub-region.

[0145] In one possible design, the target detection module 401 is specifically used to: obtain the vertex image coordinates of the target inspection image, wherein the vertex image coordinates are coordinates in the pixel coordinate system; perform coordinate transformation on the vertex image coordinates to obtain the inspection position coordinates corresponding to the target inspection image, wherein the inspection position coordinates are coordinates in the world coordinate system; and determine the sub-region enclosed by the inspection position coordinates in the inspection map as the inspection map region corresponding to the target inspection image.

[0146] In one possible design, the target detection module 401 is specifically used to: identify the inspection target in the target inspection image to obtain the first inspection target and the first image coordinates of the first inspection target in the target inspection image, wherein the target inspection image is the inspection image of the target inspection equipment; and perform coordinate transformation on the first image coordinates to obtain the first geographical location coordinates of the first inspection target, wherein the first geographical location coordinates are coordinates in the world coordinate system.

[0147] In one possible design, the target detection module 401 is specifically used to: identify the inspection target in the target inspection image through a target recognition model, and obtain the first inspection target and the first image coordinates of the first inspection target in the target inspection image.

[0148] In one possible design, the target detection module 401 is specifically used to: acquire the first device posture and the first device position coordinates of the target inspection device, wherein the first device posture is the device posture when the target inspection device acquires the target inspection image, and the first device position coordinates are the geographical location coordinates when the target inspection device acquires the target inspection image, and the first device position coordinates are coordinates in the world coordinate system; determine a target transformation matrix based on the first device posture and the first device position coordinates, wherein the target transformation matrix is ​​used to represent the transformation relationship between the pixel coordinate system and the world coordinate system when the target inspection device acquires the target inspection image; and perform coordinate transformation on the first image coordinates based on the target transformation matrix to obtain the first geographical location coordinates.

[0149] It should be noted that, Figure 7 For any content not mentioned in the corresponding embodiments, please refer to the description of the foregoing method embodiments, which will not be repeated here.

[0150] The aforementioned device, during the process of multiple inspection devices performing tasks in the inspection area, after acquiring the target inspection sub-area of ​​the inspection device and detecting the inspection targets and their geographical locations within the target inspection sub-area, first deletes the inspection targets and their geographical locations within the target inspection sub-area from the inspection target set, and then adds the detected inspection targets and their geographical locations back to the inspection target set. This ensures that the geographical locations of the inspection targets and their geographical locations within the target inspection sub-area in the inspection target set are unique, avoiding duplicate counting of inspection targets in the inspection target set due to overlapping areas covered by multiple inspection devices, thereby helping to improve the accuracy of inspection target statistics.

[0151] See Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device 50 provided in an embodiment of this application. The computer device 50 includes a processor 501 and a memory 502. The memory 502 is connected to the processor 501, for example, via a bus.

[0152] Processor 501 is configured to support the computer device 50 in performing the corresponding functions in the methods described in the above method embodiments. Processor 501 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0153] Memory 502 is used to store program code, etc. Memory 502 may include volatile memory (VM), such as random access memory (RAM); memory 502 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 502 may also include combinations of the above types of memory.

[0154] Processor 501 can call the program code to perform the following operations:

[0155] During the target inspection process, the target inspection sub-area of ​​the target inspection device is acquired, and the first inspection target and the first geographical location of the first inspection target within the target sub-inspection area are detected. The target inspection process is the process by which the multiple inspection devices perform inspection tasks in the inspection area. The target inspection device is any one of the multiple inspection devices. The target inspection sub-area is the sub-area inspected by the target inspection device and is part of the inspection area.

[0156] Delete the second inspection target and its geographical location from the inspection target set. The inspection target set includes all inspection targets detected by the multiple inspection devices during the target inspection process and the geographical locations of all inspection targets. The second inspection target is the inspection target located in the target sub-inspection area of ​​the inspection target set.

[0157] Add the first inspection target and the first geographical location to the inspection target set;

[0158] Display all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map.

[0159] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in the foregoing embodiments.

[0160] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0161] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for detecting and locating inspection targets, characterized in that, include: During the target inspection process, the target inspection sub-area of ​​the target inspection equipment is acquired, and the first inspection target and the first geographical location of the first inspection target within the target inspection sub-area are detected. The target inspection process is a process in which multiple inspection equipment perform inspection tasks in the inspection area. The target inspection equipment is any one of the multiple inspection equipment. The target inspection sub-area is the sub-area inspected by the target inspection equipment and is part of the inspection area. Delete the second inspection target and its geographical location from the inspection target set. The inspection target set includes all inspection targets detected by the multiple inspection devices during the target inspection process and the geographical locations of all inspection targets. The second inspection target is the inspection target located in the target inspection sub-region of the inspection target set. Add the first inspection target and the first geographical location to the inspection target set; Display all inspection targets in the inspection target set at their corresponding geographical locations on the inspection map; The step of acquiring the target inspection sub-area of ​​the target inspection equipment includes: Acquire a target inspection image, wherein the target inspection image is the inspection image of the target inspection equipment; The inspection map area corresponding to the target inspection image is determined as the target inspection sub-region.

2. The method according to claim 1, characterized in that, The target inspection device is the i-th inspection device among the plurality of inspection devices, 1≤i≤N, where N is the number of the plurality of inspection devices; After adding the first inspection target and the first geographical location to the inspection target set, the method further includes: The (i+1)th inspection device among the plurality of inspection devices is taken as the target inspection device, and the step of obtaining the target inspection sub-region of the target inspection device is executed.

3. The method according to claim 1 or 2, characterized in that, The method further includes: In the inspection map, the inspection sub-areas inspected by each of the multiple inspection devices are displayed by color.

4. The method according to claim 1, characterized in that, Before determining the inspection map area corresponding to the target inspection image as the target inspection sub-region, the following steps are included: Obtain the vertex image coordinates of the target inspection image, wherein the vertex image coordinates are coordinates in the pixel coordinate system; The vertex image coordinates are transformed to obtain the inspection position coordinates corresponding to the target inspection image, where the inspection position coordinates are coordinates in the world coordinate system. In the inspection map, the sub-region enclosed by the inspection location coordinates is defined as the inspection map region corresponding to the target inspection image.

5. The method according to claim 1 or 2, characterized in that, The detection of the first inspection target and the first geographical location of the first inspection target within the target inspection sub-region includes: The target inspection image is used to identify the inspection target, and the first inspection target and the first image coordinates of the first inspection target in the target inspection image are obtained. The target inspection image is the inspection image of the target inspection equipment. The first image coordinates are transformed to obtain the first geographical location coordinates of the first inspection target, which are coordinates in the world coordinate system.

6. The method according to claim 5, characterized in that, The step of identifying the inspection target in the target inspection image to obtain the first inspection target and the first image coordinates of the first inspection target in the target inspection image includes: By using a target recognition model, the target inspection image is identified to obtain the first inspection target and its first image coordinates in the target inspection image.

7. The method according to claim 5, characterized in that, The step of performing coordinate transformation on the first image coordinates to obtain the first geographical location coordinates of the first inspection target includes: The first device posture and the first device position coordinates of the target inspection device are obtained. The first device posture is the device posture of the target inspection device when it acquires the target inspection image. The first device position coordinates are the geographical coordinates of the target inspection device when it acquires the target inspection image. The first device position coordinates are coordinates in the world coordinate system. Based on the first device posture and the first device position coordinates, a target transformation matrix is ​​determined. The target transformation matrix is ​​used to represent the transformation relationship between the pixel coordinate system and the world coordinate system when the target inspection device acquires the target inspection image. Based on the target transformation matrix, the first image coordinates are transformed to obtain the first geographical location coordinates.

8. A target detection and positioning device for inspection, characterized in that, include: The target detection module is used to acquire the target inspection sub-area of ​​the target inspection device during the target inspection process, and detect the first inspection target and the first geographical location of the first inspection target within the target inspection sub-area. The target inspection process is a process in which multiple inspection devices perform inspection tasks in the inspection area. The target inspection device is any one of the multiple inspection devices. The target inspection sub-area is the sub-area inspected by the target inspection device and is part of the inspection area. The target deletion module is used to delete the second inspection target and the geographical location of the second inspection target from the inspection target set. The inspection target set includes all inspection targets detected by the plurality of inspection devices during the target inspection process and the geographical locations of all inspection targets. The second inspection target is the inspection target located in the target inspection sub-region of the inspection target set. The target saving module is used to add the first inspection target and the first geographical location to the inspection target set; The target display module is used to display all inspection targets in the inspection target set at the corresponding geographical locations on the inspection map.

9. A computer device, characterized in that, The device includes a memory, a processor, and connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, wherein, when executing the one or more computer programs, the processor causes the computer device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-7.

Images