Method and device for determining a point to be inspected, and method for optimizing attribute data of a point to be inspected

Abstract

The application discloses a kind of determination methods of being visited point, device and the attribute data optimization method of being visited object, by the geometric morphology fitting of being visited object, obtain the cropped viewport for representing being visited object, then under the constraint of visual angle, the virtual field of view space formed when light beam is emitted by projection center to cropped viewport is determined, again, the region that the virtual field of view space constitutes obstruction is identified and eliminated, obtain safe visible region;Then under the premise of ensuring that cropped viewport does not lose feature, the photographable area of tour equipment in virtual field of view space range is calculated;In the case where there is intersection region in the safe visible region and the photographable area, the final tour point of the visited object is determined according to intersection region. Thus, by solving the final tour point in the intersection region calculated, more reasonable and accurate tour position can be obtained to improve the success rate, accuracy and safety of intelligent power inspection task.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of substation inspection, and in particular to a method and device for determining a point to be inspected, and a method for optimizing attribute data of an object to be inspected. BACKGROUND

[0002] Currently, one of the key technologies in intelligent power inspection methods is to automatically plan an inspection route for a UAV or a ground robot (hereinafter collectively referred to as an inspection device) to collect data from a real scene to be inspected (hereinafter referred to as a scene to be inspected). The inspection route is represented by a series of points (hereinafter referred to as inspection points), wherein the collected data can be at least one of multi-view images, videos, and laser point clouds, and can be used for digital analysis from the aspect of safety.

[0003] However, the scene to be inspected is often very complex due to the large number of power components, compact deployment, and environmental interference. The inspection device needs to be kept at a safe distance considering the power threat, and a certain target object (hereinafter referred to as an object to be inspected) that needs to be inspected may be blocked by other objects around it. However, in the process of planning the above inspection route, the existing method does not fully consider the blocking and safety, which may cause unreasonable pose of the inspection point relative to the object to be inspected.

[0004] How to seek a more reasonable and accurate optimal inspection position relative to the object to be inspected to improve the success rate, accuracy, and safety of intelligent power inspection tasks is a problem that needs to be solved. SUMMARY

[0005] The present application provides a method and device for determining a point to be inspected, and a method for optimizing attribute data of an object to be inspected, which aims to obtain a more reasonable and accurate optimal inspection position to improve the success rate, accuracy, and safety of intelligent power inspection tasks.

[0006] An embodiment of the present application provides a method for determining a point to be inspected, comprising:

[0007] obtaining a geometric shape of an object to be inspected, a type of the object to be inspected, a type of an inspection device, a focal length of a camera in the inspection device, an imaging angle of the camera in the inspection device, and a safety distance between the inspection device and each obstacle;

[0008] performing plane fitting on the geometric shape of the object to be inspected, and taking a plane region obtained by the fitting as a clipping viewport for representing the object to be inspected;

[0009] determining a visible angle constraint under the condition of meeting the safety distance according to the type of the object to be inspected, the type of the inspection device, and the safety distance;

[0010] determining a projection center and a virtual field of view space formed when a light beam is emitted from the projection center to the cropped viewport under the visual angle constraint;

[0011] identifying and eliminating regions that constitute an occlusion to the virtual field of view space to obtain a safe visual region;

[0012] determining, according to a focal length of a camera in the patrol device, an imaging angle of the camera in the patrol device, and the cropped viewport, a photographable region of the patrol device within the virtual field of view space under the premise that the cropped viewport does not lose features;

[0013] in a case where the safe visual region and the photographable region exist in an intersection region, determining a final patrol point of the patrolled object according to the intersection region.

[0014] Further, the determination of the projection center comprises:

[0015] determining a geometric center of the cropped viewport;

[0016] drawing a perpendicular line of the cropped viewport through the geometric center;

[0017] constructing a right-handed coordinate system with an intersection point of two sides of the cropped viewport as a coordinate origin O, with an arbitrary side of the intersection point as a Z axis, with a straight line perpendicular to the cropped viewport and passing through the coordinate origin as an X axis, and with a straight line passing through the coordinate origin and perpendicular to the Z axis and the X axis as a Y axis;

[0018] rotating a straight line located on the X axis in the X-O-Y plane in the direction of the Y axis according to an angle corresponding to the visual angle constraint, to obtain a rotated straight line;

[0019] taking an intersection point of the rotated straight line and the perpendicular line as the projection center.

[0020] Further, the identification and elimination of regions that constitute an occlusion to the virtual field of view space to obtain a safe visual region comprises:

[0021] obtaining a set of patches reflecting a surface of the patrolled object;

[0022] calculating a geometric center point of each patch in the set of patches, and constructing a corresponding unit circle with each geometric center point as a center;

[0023] sampling local regions in which each unit circle is located in the virtual field of view space to obtain a plurality of sampling points;

[0024] dividing the virtual field of view space into a plurality of levels of grids, and setting an attribute of the plurality of levels of grids as a visible grid;

[0025] projecting a light ray from each geometric center point as a starting point and in a direction of each sampling point;

[0026] sequentially performing occlusion detection on each of the projected light rays, and after completing the occlusion detection of all the projected light rays, regarding a region corresponding to a multi-level grid whose attribute is adjusted to be a non-visible grid as a region constituting an occlusion to the virtual field of view space;

[0027] determining a safe warning region corresponding to each obstacle according to the safe distance;

[0028] eliminating the safe warning region corresponding to each obstacle and the region constituting an occlusion to the virtual field of view space from the virtual field of view space to obtain a safe visible region;

[0029] The occlusion detection comprises:

[0030] identifying whether the current projected light ray is occluded by an obstacle,

[0031] if yes, adjusting an attribute of a multi-level grid located behind the obstacle to be a non-visible grid;

[0032] if no, not adjusting the attribute of the multi-level grid.

[0033] Further, the determining of the photographable region of the patrol device in the virtual field of view space under the premise that the cropped viewport does not lose features comprises:

[0034] determining a farthest photographing distance according to the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and a preset first scaling ratio; the first scaling ratio is a minimum ratio between an actual size of the cropped viewport and an imaging size under the premise that the imaging can be completely displayed;

[0035] determining a nearest photographing distance according to the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and a preset second scaling ratio; the second scaling ratio is a maximum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be completely displayed;

[0036] determining the photographable region of the patrol device in the virtual field of view space according to the nearest photographing distance and the farthest photographing distance.

[0037] Further, the determining of the final patrol point of the patrolled object according to the intersection region in the case that the safe visible region and the photographable region have an intersection region comprises:

[0038] calculating a normal vector of each patch in the patch set;

[0039] randomly generating a plurality of initial patrol points;

[0040] for each initial patrol point, calculating a relationship score of the current initial patrol point relative to each patch in the patch set; wherein the relationship score is determined by the balanced angle and distance between the patrol point and the patch;

[0041] selecting the initial patrol point with the highest score higher than a preset threshold as the final patrol point.

[0042] Further, the method for determining the patrolled point further comprises:

[0043] if there is no relationship score higher than the preset threshold, then using the remaining different types of patrol devices.

[0044] Further, the method for determining the patrolled point further comprises:

[0045] if there is no intersection area between the safe visible area and the photographable area, then using the remaining different types of patrol devices.

[0046] An embodiment of the present application also provides a device for determining a patrolled point, comprising: a data acquisition module, a plane fitting module, a visible angle constraint determination module, a virtual field of view space determination module, a safe visible area determination module, a photographable area determination module, and a final patrol point determination module.

[0047] The data acquisition module is configured to acquire the geometric shape of the patrolled object, the type of the patrolled object, the type of the patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle.

[0048] The plane fitting module is configured to perform plane fitting on the geometric shape of the patrolled object, and take the fitted plane area as a clipping viewport for representing the patrolled object.

[0049] The visible angle constraint determination module is configured to determine the visible angle constraint under the condition of meeting the safe distance according to the type of the patrolled object, the type of the patrol device, and the safe distance.

[0050] The virtual field of view space determination module is configured to determine the projection center and the virtual field of view space formed when the projection center emits a light beam toward the clipping viewport under the visible angle constraint.

[0051] The safe visible area determination module is configured to identify and eliminate the area that constitutes an obstruction to the virtual field of view space, and obtain a safe visible area.

[0052] The photographable area determination module is configured to determine, according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device and the clipping viewport, a photographable area of the patrol device in a virtual field of view space range under the premise that the clipping viewport does not lose features.

[0053] The final patrol point determination module is configured to determine, in the case that the safe visible area and the photographable area have an intersection area, a final patrol point of the patrolled object according to the intersection area.

[0054] An embodiment of the present application further provides an attribute data optimization method of a patrolled object, comprising:

[0055] calculating a spatial attribute between the patrolled object and the final patrol point, wherein the final patrol point is determined by the determination method of the patrolled point;

[0056] generating attribute data of the patrolled object after optimization according to the spatial attribute, the geometric shape of the patrolled object and the type of the patrolled object.

[0057] Further, the spatial attribute is determined by the following method:

[0058] calculating a geometric center of an outer bounding box generated by the patch set to obtain a position attribute;

[0059] calculating a distance between the final patrol point and the geometric center of the outer bounding box to obtain a distance attribute;

[0060] calculating a direction of the geometric center of the outer bounding box pointing to the final patrol point to obtain a direction attribute;

[0061] determining the spatial attribute according to the position attribute, the distance attribute and the direction attribute.

[0062] The present application has the following beneficial effects by implementation:

[0063] The present application provides a determination method of a patrolled point, which determines a visible angle constraint under the condition of meeting a safety distance by obtaining the type of a patrolled object, the type of a patrol device and the safety distance between the patrol device and each obstacle, determines a projection center under the visible angle constraint, then emits a light beam to a clipping viewport representing the patrolled object through the projection center to form a virtual field of view space, and removes an area constituting an obstruction from the virtual field of view space, so that a finally formed safe visible area fully takes into account the obstruction and safety.

[0064] Then, according to the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropped viewport, a photographable region of the patrol device in the virtual field of view space is determined under the premise that the cropped viewport does not lose features, thereby avoiding the situation that the features of the patrolled object cannot be distinguished due to too small imaging, or the patrolled object cannot completely appear in the imaging due to too large imaging.

[0065] In summary, the safe visible region and the photographable region are used as the key basis for determining the final patrol point, and the final patrol point is refined in the intersection region, so that when subsequent inspection is performed based on the final patrol point, the imaging quality can be ensured under the condition that the safety of the shooting process is met, and the final patrol point can meet the rationality and accuracy of the real patrol business. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0067] Figure 1 is a flowchart of a method for determining a patrolled point provided by an embodiment of the present application;

[0068] Figure 2 is a schematic diagram of center projection for a cropped viewport provided by an embodiment of the present application;

[0069] Figure 3 is a schematic diagram of a virtual field of view space observed in the negative direction of the Z axis provided by an embodiment of the present application;

[0070] Figure 4 is a schematic diagram of a visible region provided by an embodiment of the present application;

[0071] Figure 5 is a schematic diagram of a safe visible region provided by an embodiment of the present application;

[0072] Figure 6 is a schematic diagram of an imaging angle of view corresponding to the farthest shooting distance and the nearest shooting distance of the virtual field of view space provided by an embodiment of the present application;

[0073] Figure 7 is a schematic diagram of a photographable region provided by an embodiment of the present application;

[0074] Figure 8 is a schematic diagram of an intersection region provided by an embodiment of the present application;

[0075] Figure 9is a schematic diagram of the influence of a patrol device on an intersection region according to an embodiment of the present application;

[0076] Figure 10 is a structural schematic diagram of a determination device of a patrolled point according to an embodiment of the present application;

[0077] Figure 11 is a flow schematic diagram of an attribute data optimization method of a patrolled object according to an embodiment of the present application;

[0078] Figure 12 is a schematic diagram of attribute data of a patrolled object according to an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above description of drawings are intended to cover non-exclusive inclusion.

[0081] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly and specifically limited.

[0082] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily independent or alternative embodiments to each other. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0083] In the description of the embodiments of the present application, the term "and / or" is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0084] In the description of the embodiments of the present application, the term "a plurality of" refers to two or more (including two), and similarly, "a plurality of groups" refers to two or more groups (including two groups), and "a plurality of pieces" refers to two or more pieces (including two pieces).

[0085] In the description of the embodiments of the present application, unless otherwise explicitly specified and limited, the technical terms "mounting", "connecting", "connecting", "fixing" and the like should be understood in a broad sense, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanical connection, or it can be electrical connection; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0086] Referring to Figure 1 is a flowchart of a determination method of a patrolled point provided by an embodiment of the present application, comprising:

[0087] S1, obtaining the geometric shape of the patrolled object, the type of the patrolled object, the type of the patrolling device, the focal length of the camera in the patrolling device, the imaging angle of view of the camera in the patrolling device, and the safety distance between the patrolling device and each obstacle;

[0088] Illustratively, the patrolled object data including the geometric shape of the patrolled object and the type of the patrolled object need to be obtained, wherein the patrolled object data is obtained from a three-dimensional scene model;

[0089] Specifically, the three-dimensional scene model can be any one of a mesh model, a DEM model and a point cloud model; for example, in the case of using a three-dimensional mesh model to represent the surface geometric details of the patrolled scene, some triangular patches in the three-dimensional mesh model are extracted as the patrolled object data based on the correspondence between the multi-view image and the three-dimensional mesh model; wherein the three-dimensional mesh model is obtained by three-dimensional reconstruction according to the multi-view image; the patch set obtained after extraction represents a local surface of the patrolled object, which can at least reflect the geometric shape of the patrolled object in three-dimensional space; the geometric center of the outer bounding box of the patch set is taken as the position of the patrolled object;

[0090] S1, acquiring the object data of the object to be patrolled, the patrolling device data of the patrolling device, and the electronic fence data of the electronic fence;

[0091] Specifically, different patrolling devices are equipped with cameras with different parameters, such as single-lens reflex cameras used for manual patrolling, multiple-zoom cameras used for robot patrolling, and onboard cameras used for unmanned aerial vehicle patrolling.

[0092] S1, acquiring the object data of the object to be patrolled, the patrolling device data of the patrolling device, and the electronic fence data of the electronic fence;

[0093] Specifically, the electronic fence data includes safety distance data associated with the three-dimensional scene model, such as the safety distance between the unmanned aerial vehicle and the trees.

[0094] S2, performing plane fitting on the geometric shape of the object to be patrolled, and taking the fitted plane region as a clipping viewport for representing the object to be patrolled;

[0095] Specifically, for a certain object to be patrolled, in order to simplify the geometric representation, the RANSAC algorithm can be used to perform plane fitting on the above-mentioned object data, and the fitted plane region is taken as a clipping viewport for representing the object to be patrolled.

[0096] S3, determining the visual angle constraint under the condition of meeting the safety distance according to the type of the object to be patrolled, the type of the patrolling device, and the safety distance;

[0097] Specifically, according to the type of the object to be patrolled, the type of the patrolling device, and the safety distance obtained in step S1, the visual angle constraint under the condition of meeting the safety distance is determined by looking up a table; wherein the visual angle constraint corresponds to a visual angle a.

[0098] Specifically, the object type, the patrolling device type, the safety distance, and the preset visual angle a are associated and stored in a preset data table in advance; or, the object type, the camera type, and the visual angle a are associated and stored in a first data table in advance, and the object type, the camera type, and the safety distance are associated and stored in a second data table in advance; when the visual angle constraint is needed, the corresponding visual angle constraint and the visual angle corresponding to the visual angle constraint can be obtained by looking up the data table.

[0099] S4, determining the projection center and the virtual field space formed when the projection center emits a light beam to the clipping viewport under the visual angle constraint;

[0100] The virtual field of view space R refers to a space in which a light ray continues to propagate after passing through a certain object being patrolled. The virtual field of view space R is affected by the visual angle constraint. Therefore, before the virtual field of view space R is obtained, the projection center needs to be determined.

[0101] In a preferred embodiment, the determination of the projection center comprises:

[0102] determining a geometric center of the clipping viewport;

[0103] drawing a perpendicular line of the clipping viewport through the geometric center;

[0104] constructing a right-handed coordinate system with an intersection point of two sides of the clipping viewport as a coordinate origin O, with an arbitrary side of the intersection point as a Z axis, with a straight line perpendicular to the clipping viewport and passing through the coordinate origin as an X axis, and with a straight line passing through the coordinate origin and perpendicular to the Z axis and the X axis as a Y axis;

[0105] rotating a straight line located on the X axis in the X-O-Y plane in the direction of the Y axis according to an angle corresponding to the visual angle constraint, to obtain a rotated straight line, with the coordinate origin as a fulcrum;

[0106] taking an intersection point of the rotated straight line and the perpendicular line as the projection center;

[0107] Specifically, referring to Figure 2 for example, taking a triangular clipping viewport, a right-handed coordinate system is constructed for the clipping viewport, in which an intersection point of two sides of the clipping viewport is taken as a coordinate origin O, an arbitrary side of the intersection point is taken as a Z axis, a straight line perpendicular to the clipping viewport and passing through the coordinate origin is taken as an X axis, and a straight line passing through the coordinate origin and perpendicular to the Z axis and the X axis is taken as a Y axis;

[0108] Specifically, referring to Figure 3 a straight line located on the X axis is rotated in the X-O-Y plane in the direction of the Y axis according to a visual angle α corresponding to the visual angle constraint, to obtain a rotated straight line, with the coordinate origin as a fulcrum, and then an intersection point of the rotated straight line and the perpendicular line is taken as the projection center;

[0109] Thus, the obtained projection center is located directly in front of the clipping viewport, that is, an included angle between a vector obtained by connecting the projection center and the geometric center of the clipping viewport and a positive direction of the X axis is 0 degrees; wherein the positive direction of the X axis represents a direction in which a certain light ray emitted from the projection center continues to propagate straight after vertically passing through the clipping viewport;

[0110] Specifically, after the projection center is obtained, a light beam emitted from the clipping viewport in accordance with the principle of central projection will form a virtual field of view space R behind the clipping viewport.

[0111] S5, identifying and removing a region constituting an occlusion to the virtual view space, to obtain a safe viewable region;

[0112] Illustratively, since the patrolled scene data in the three-dimensional scene model can have other object data constituting a line-of-sight occlusion to the patrolled object or / and a safety threat, it is necessary to perform bounding box detection on the other object data than the patrolled object data, and to regard the detected bounding box as an obstacle;

[0113] After obtaining the obstacle information in the patrolled scene, it is necessary to remove a region constituting an occlusion to the virtual view space R;

[0114] In a preferred embodiment, the identifying and removing a region constituting an occlusion to the virtual view space, to obtain a safe viewable region, comprises:

[0115] obtaining a set of patches for reflecting a surface of the patrolled object;

[0116] calculating a geometric center point of each patch in the set of patches, and constructing a corresponding unit circle with each geometric center point as a center;

[0117] sampling a local region in which each unit circle is located in the virtual view space, to obtain a plurality of sampling points;

[0118] dividing the virtual view space into a plurality of hierarchical grids, and setting a property of the plurality of hierarchical grids as a viewable grid;

[0119] projecting a light ray with each geometric center point as a starting point and each sampling point as a direction;

[0120] sequentially performing occlusion detection on each projected light ray, and after completing the occlusion detection of all the projected light rays, regarding a region corresponding to the plurality of hierarchical grids whose properties are adjusted as non-viewable grids as a region constituting an occlusion to the virtual view space;

[0121] determining a safe alert region corresponding to each obstacle according to the safe distance;

[0122] removing the safe alert region corresponding to each obstacle and the region constituting an occlusion to the virtual view space from the virtual view space, to obtain a safe viewable region;

[0123] The occlusion detection comprises:

[0124] identifying whether a current projected light ray is occluded by an obstacle,

[0125] if yes, adjusting a property of a plurality of hierarchical grids located behind the obstacle and having a viewable grid property as a non-viewable grid property;

[0126] If not, the property of the multi-level grid is not adjusted;

[0127] Specifically, the geometric center point of each patch in the patch set is calculated, and a corresponding unit circle is constructed with each geometric center point as the center. The local area of each unit circle in the virtual field space R is sampled by any one of a variety of ways such as uniform sampling, stratified sampling, and random sampling, to obtain a plurality of corresponding sampling points;

[0128] Then, a light ray is projected from each geometric center point to each sampling point, and the occlusion detection is performed on each projection light ray in turn. For any projection light ray, after the projection light ray first encounters an obstacle, the property of the multi-level grid behind the obstacle is automatically adjusted from visible grid to invisible grid.

[0129] It should be noted that once the property of the multi-level grid becomes invisible grid, it cannot enter the occlusion detection of the next projection light ray.

[0130] Referring to Figure 4 After the occlusion detection of all projection light rays is completed, the region corresponding to the multi-level grid whose property is adjusted to invisible grid is regarded as a region occluding the virtual field space, and the region occluding the virtual field space is removed from the virtual field space to obtain a visible region A', wherein A' represents a space that will not be occluded when the camera faces the patrolled object to take a picture.

[0131] Referring to Figure 5 Then, according to the safety distance between the patrolling device and each obstacle, the safety warning area corresponding to each obstacle is determined, that is, the safety warning area is determined by extending the periphery of the corresponding obstacle by a certain range according to the safety distance.

[0132] Finally, the safety warning area corresponding to each obstacle is removed from the view area A' to obtain the final safe visible area A.

[0133] S6, according to the focal length of the camera in the patrolling device, the imaging angle of the camera in the patrolling device, and the clipping viewport, determine the photographable area of the patrolling device in the virtual field space range under the premise of ensuring that the clipping viewport does not lose features.

[0134] In a preferred embodiment, the determination of the photographable area of the patrolling device in the virtual field space range under the premise of ensuring that the clipping viewport does not lose features according to the focal length of the camera in the patrolling device, the imaging angle of the camera in the patrolling device, and the clipping viewport comprises:

[0135] determine a farthest shooting distance according to the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and a preset first zoom ratio, wherein the first zoom ratio is a minimum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be completely displayed;

[0136] determine a closest shooting distance according to the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and a preset second zoom ratio, wherein the second zoom ratio is a maximum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be completely displayed;

[0137] determine a shootable area of the patrol device in a virtual field of view space according to the closest shooting distance and the farthest shooting distance;

[0138] Illustratively, the size of the cropped viewport is denoted as H, the distance between the cropped viewport and the patrol device is denoted as d, the focal length of the camera in the patrol device is denoted as f, the imaging angle of view of the camera in the patrol device is denoted as θ, and the size of the cropped viewport imaged by the camera is denoted as h, so that the above parameters satisfy the following formula:

[0139]

[0140] Specifically, referring to Figure 6 Since the focal length f of the camera in the patrol device and the imaging angle of view θ of the camera in the patrol device are known quantities, when the zoom ratio is the first zoom ratio, the farthest shooting distance between the cropped viewport and the patrol device can be determined as d max , wherein the first zoom ratio is a ratio corresponding to the minimum h / H;

[0141] Similarly, referring to Figure 6 Since the focal length f of the camera in the patrol device and the imaging angle of view θ of the camera in the patrol device are known quantities, when the zoom ratio is the second zoom ratio, the closest shooting distance between the cropped viewport and the patrol device can be determined as d min , wherein the second zoom ratio is a ratio corresponding to the maximum h / H;

[0142] Take the cropped viewport as a target object, obtain the longest diagonal line from the image, make a circle with the center of the diagonal line as the center and the diagonal line as the diameter, calculate d min and d max , and the curves composed of d min and d max are rotated along the axes perpendicular to the center, that is, the first cutting surface corresponding to d min and the second cutting surface corresponding to d max are obtained, and then referring to Figure 7The first cutting plane, the second cutting plane, and two straight-line boundaries distributed on both sides of the virtual field of view space R form a photographable area B.

[0143] S7. When there is an intersection between the safe visual area and the photographable area, determining a final inspection point of the inspected object according to the intersection;

[0144] See also Figure 8 Specifically, it is determined whether the safe visual area A and the photographable area B have an intersection. If This means that there is an intersection area A∩B, where the calculation method of the intersection area is not unique, for example, it can be calculated by boundary constraints, voxel segmentation, etc.

[0145] In a preferred embodiment, when there is an intersection between the safe visual area and the photographable area, determining the final inspection point of the inspected object according to the intersection includes:

[0146] Calculating a normal vector for each face in the face set;

[0147] Randomly generate several initial patrol points;

[0148] For each initial patrol point, calculate the relationship score of the current initial patrol point relative to each face in the face set; wherein the relationship score is determined by the balance angle and distance between the patrol point and the face;

[0149] The initial inspection point with the highest score above the preset threshold is used as the final inspection point;

[0150] Specifically, calculate the normal vector of each face in the face set T Where T={T1,T2,…,T i ,T n}, n represents the total number of faces; then find the final patrol point P from the intersection area A∩B * , so that the final patrol point P * The relationship score F(P) with the positions and normal vectors of all triangles is the highest;

[0151] Specifically, the final patrol point is calculated using the following formula:

[0152]

[0153] Among them, F(P) represents the average score of the initial patrol point P in each patch, which is calculated by the following formula:

[0154]

[0155] Among them, fi (P) represents a score of the initial patrol point P relative to the triangular facet T i , which is calculated by the following formula:

[0156]

[0157] wherein θ i represents the angle between the line connecting the initial patrol point P and the triangular facet T i and the normal vector ; d i represents the distance between the initial patrol point P and the triangular facet T i ; and α represents a first weight coefficient for determining the influence of the angle on the score, and β represents a second weight coefficient for determining the influence of the distance on the score.

[0158] In a preferred embodiment, the method for determining the patrol point further comprises:

[0159] if there is no relationship score higher than the preset threshold, then the remaining different types of patrol devices are used;

[0160] Specifically, if the relationship scores F(P) of all the initial patrol points are all less than the preset threshold S, then the remaining different types of patrol devices are used.

[0161] If, after replacing all the patrol devices, it is found that the intersection of the safe visible region A and the photographable region B is empty or the scores of the initial patrol points are all less than the preset threshold S, then it is considered that no patrol point meeting the requirements can be obtained in the safe visible region A and the photographable region B currently obtained, and thus the object to be patrolled needs to be adjusted, i.e., the plane of the object to be patrolled needs to be refitted.

[0162] In a preferred embodiment, the method for determining the patrol point further comprises:

[0163] if there is no intersection region between the safe visible region and the photographable region, then the remaining different types of patrol devices are used;

[0164] Specifically, if it is indicated that the object to be patrolled cannot be photographed under safe conditions, and the remaining different types of patrol devices are used.

[0165] Specifically, referring to Figure 9 , it is assumed that an object to be patrolled is a meter installed in the middle of the device, and in order to obtain the meter reading, a long-focus camera can be used to remotely photograph the front of the meter under safe conditions (i.e. Figure 9 ​The camera on the unmanned aerial vehicle will face security threats if it needs to shoot close to the front of the meter inside the electronic fence (i.e. Figure 9 The camera on the unmanned aerial vehicle will face security threats if it needs to shoot close to the front of the meter inside the electronic fence (i.e.

[0166] It can be understood that the parameters of the cameras in different patrol devices can be respectively associated with different visual angles a, for example: the meter has a glass protective cover and is reflective, the camera faces the glass protective cover of the meter and shoots, and the camera plus a polarizing filter can eliminate the reflection, in which case the camera parameter can be defined as x1, and a corresponding to x1 can be set to 60°; otherwise, the camera parameter can be defined as x2, x2 is different from x1, and a corresponding to x2 can be set to 30°.

[0167] In the case where the object being patrolled and the scene in which it is located remain unchanged, the safe visible area A is only related to the value of the visual angle a, and after changing the camera parameters, if the visual angle a2 required after the change (for the sake of distinction, it can be defined as a2) is equal to the visual angle a1 required at a certain time after the change (it can be defined as a1), A remains unchanged; if 0 < a2-a1 < Δa, only the overlapping part of the space before and after the change needs to be calculated to detect the new intersection area A∩B.

[0168] Embodiment Two

[0169] Reference Figure 10 is a determination device for a point being patrolled provided by an embodiment of the present application, comprising: a data acquisition module, a plane fitting module, a visual angle constraint determination module, a virtual field of view space determination module, a safe visible area determination module, a photographable area determination module, and a final patrol point determination module;

[0170] The data acquisition module is configured to acquire the geometric shape of the object being patrolled, the type of the object being patrolled, the type of the patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle.

[0171] The plane fitting module is configured to perform plane fitting on the geometric shape of the object being patrolled, and take the plane region obtained by fitting as a cropped viewport for representing the object being patrolled.

[0172] The visual angle constraint determination module is configured to determine the visual angle constraint under the condition of meeting the safe distance according to the type of the object being patrolled, the type of the patrol device, and the safe distance.

[0173] The virtual field of view space determination module is configured to determine a projection center and a virtual field of view space formed when the projection center emits a light beam to the cropped viewport under the visual angle constraint;

[0174] The safe visual area determination module is configured to identify and eliminate areas that constitute an obstruction to the virtual field of view space, to obtain a safe visual area;

[0175] The photographable area determination module is configured to determine, according to a focal length of a camera in the patrol device, an imaging visual angle of the camera in the patrol device, and the cropped viewport, a photographable area of the patrol device in the virtual field of view space range under the premise that the cropped viewport does not lose features.

[0176] The final patrol point determination module is configured to determine, in a case where the safe visual area and the photographable area exist in an intersection area, a final patrol point of the patrolled object according to the intersection area.

[0177] Embodiment Three

[0178] Referring to Figure 11 is a property data optimization method of a patrolled object provided by an embodiment of the present application, comprising:

[0179] D1, calculating a spatial property between the patrolled object and a final patrol point, wherein the final patrol point is determined by the determination method of the patrolled point;

[0180] In a preferred embodiment, the spatial property is determined by the following method:

[0181] calculating a geometric center of an outer bounding box generated by the patch set to obtain a position property;

[0182] calculating a distance between the final patrol point and the geometric center of the outer bounding box to obtain a distance property;

[0183] calculating a direction of the geometric center of the outer bounding box pointing to the final patrol point to obtain a direction property;

[0184] determining the spatial property according to the position property, the distance property, and the direction property;

[0185] Specifically, the final patrol point is a final patrol point determined by any one of the determination method embodiments of the patrol point of the present application;

[0186] Specifically, after the final inspection point is obtained by the embodiment one, firstly, the geometric center of the outer bounding box generated by the patch set is calculated to obtain the position attribute; then, the distance between the final inspection point and the geometric center of the outer bounding box is calculated to obtain the distance attribute; finally, the direction of the geometric center of the outer bounding box pointing to the final inspection point is calculated to obtain the direction attribute.

[0187] Specifically, the position attribute, the distance attribute and the direction attribute jointly constitute the optimized spatial attribute.

[0188] D2, generating the optimized attribute data of the inspected object according to the spatial attribute, the geometric shape of the inspected object and the type of the inspected object;

[0189] Specifically, referring to Figure 12 , generating the optimized attribute data of the inspected object according to the spatial attribute, the geometric shape of the inspected object and the type of the inspected object, wherein the geometric shape of the inspected object and the type of the inspected object jointly constitute the inspected object data;

[0190] Therefore, through the optimized attribute data of the inspected object, it is helpful to simplify the subsequent application process such as the inspection route planning, improve the overall application effect, and better manage and maintain the inspected object, reduce the inspection cost, and improve the efficiency and reliability.

[0191] The above is the preferred embodiment of the present application, it should be noted that for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements also considered to be within the scope of the present application.

Claims

1. A method for determining patrol points, characterized in that: include: Obtain the geometry of the inspected object, the type of the inspected object, the type of the inspection device, the focal length of the camera in the inspection device, the imaging angle of the camera in the inspection device, and the safe distance between the inspection device and each obstacle; Performing plane fitting on the geometric shape of the inspected object, and using the plane area obtained by fitting as a clipping viewport for representing the inspected object; Determining, according to the type of the patrolled object, the type of the patrol device, and the safety distance, a viewing angle constraint while satisfying the safety distance; Under the viewing angle constraint, determining a projection center and a virtual field of view space formed when a light beam is emitted from the projection center to the clipping viewport; Identify and eliminate areas that block the virtual visual field to obtain a safe visual area; Determining, based on the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the cropping viewport, a photographable area of ​​the patrol device within the virtual field of view while ensuring that features of the cropping viewport are not lost; In the case where there is an intersection area between the safe visual area and the photographable area, a final patrol point of the patrolled object is determined according to the intersection area.

2. The method for determining patrol points according to claim 1, wherein: The determination of the projection center includes: Determine the geometric center of the clipping viewport; Draw a perpendicular line to the clipping viewport through the geometric center; Construct a right-handed coordinate system with the intersection of the two sides of the clipping viewport as the coordinate origin O, any side of the intersection as the Z axis, a straight line perpendicular to the clipping viewport and passing through the coordinate origin as the X axis, and a straight line passing through the coordinate origin and perpendicular to both the Z axis and the X axis as the Y axis; With the coordinate origin as a fulcrum, according to the angle corresponding to the viewing angle constraint, the straight line located on the X-axis is rotated in the Y-axis direction on the XOY plane to obtain a rotated straight line; The intersection of the rotated straight line and the vertical line is used as the projection center.

3. The method for determining patrol points according to claim 1, wherein: The identifying and eliminating the area that blocks the virtual visual field to obtain a safe visual area includes: Obtaining a facet set for reflecting the surface of the inspected object; Calculating the geometric center point of each patch in the patch set, and constructing a corresponding unit circle with each geometric center point as the center of the circle; Sampling a local area of ​​each unit circle in the virtual field of view to obtain a plurality of sampling points; Dividing the virtual visual field into a multi-level grid, and setting the attributes of the multi-level grid as a visible grid; Taking each geometric center point as the starting point and each sampling point as the direction, project light; performing occlusion detection on each of the projected rays in turn, and after completing the occlusion detection on all the projected rays, taking the area corresponding to the multi-level grid whose attributes are adjusted to invisible grids as the area that occludes the virtual field of view space; Determine the safety warning area corresponding to each obstacle based on the safety distance; Eliminate the safety warning areas corresponding to the obstacles and the areas blocking the virtual visual field from the virtual visual field to obtain a safe visual area; The occlusion detection includes: Identify whether the current projection light is blocked by an obstacle, If so, adjust the attributes of the multi-level grid behind the obstacle, which is a visible grid, to an invisible grid; If not, the properties of the multi-level grid are not adjusted.

4. The method for determining patrol points according to claim 1, wherein: The determining, based on the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the cropping viewport, of a photographable area of ​​the patrol device within the virtual field of view while ensuring that the cropping viewport does not lose features, includes: Determining the maximum shooting distance based on the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset first scaling ratio; wherein the first scaling ratio is the minimum ratio between the actual size of the cropped viewport and the image size under the premise that the image can be fully displayed; Determining the minimum shooting distance based on the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset second scaling ratio; wherein the second scaling ratio is the maximum ratio between the actual size of the cropped viewport and the image size under the premise that the image can be fully displayed; The photographic area of ​​the patrol device within the virtual field of view space is determined according to the closest photographing distance and the farthest photographing distance.

5. The method for determining patrol points according to claim 3, wherein: When there is an intersection between the safe visual area and the photographable area, determining a final inspection point of the inspected object according to the intersection includes: Calculating a normal vector for each face in the face set; Randomly generate several initial patrol points; For each initial patrol point, calculate the relationship score of the current initial patrol point relative to each face in the face set; wherein the relationship score is determined by the balance angle and distance between the patrol point and the face; The initial patrol point with the highest score above the preset threshold is taken as the final patrol point.

6. The method for determining patrol points according to claim 5, wherein: Also includes: If there is no relationship score higher than the preset threshold, other different types of patrol devices are used instead.

7. The method for determining patrol points according to claim 1, wherein: Also includes: In the case that there is no intersection between the safe visual area and the photographable area, other different types of patrol equipment are used instead.

8. A device for determining patrol points, characterized in that: include: Data acquisition module, plane fitting module, visual angle constraint determination module, virtual field of view space determination module, safe visual area determination module, shootable area determination module and final patrol point determination module; The data acquisition module is used to obtain the geometric shape of the patrolled object, the type of the patrolled object, the type of the patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle; The plane fitting module is used to perform plane fitting on the geometric shape of the inspected object and use the plane area obtained by fitting as a clipping viewport for representing the inspected object; The viewing angle constraint determination module is configured to determine the viewing angle constraint when the safety distance is met according to the type of the patrolled object, the type of the patrol device, and the safety distance; The virtual field of view space determination module is used to determine the projection center and the virtual field of view space formed when the light beam is emitted from the projection center to the clipping viewport under the viewing angle constraint; The safe visual area determination module is used to identify and eliminate areas that block the virtual visual field space to obtain a safe visual area; The photographable area determination module is configured to determine, based on the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the cropping viewport, the photographable area of ​​the patrol device within the virtual field of view, while ensuring that the cropping viewport does not lose features; The final patrol point determination module is configured to determine the final patrol point of the patrolled object according to the intersection area when there is an intersection area between the safe visual area and the photographable area.

9. A method for optimizing attribute data of an inspected object, characterized in that: include: Calculating the spatial attributes between the inspected object and the final inspection point; wherein the final inspection point is determined by the method for determining the inspected point according to any one of claims 1 to 7; Optimized attribute data of the inspected object is generated according to the spatial attributes, the geometric shape of the inspected object, and the type of the inspected object.

10. The method for optimizing attribute data of an inspected object according to claim 9, wherein: The spatial properties are determined by: Calculate the geometric center of the outer bounding box generated by the face set and obtain the position attribute; Calculating the distance between the final patrol point and the geometric center of the outer bounding box to obtain a distance attribute; Calculating the direction from the geometric center of the outer bounding box to the final patrol point to obtain a direction attribute; The spatial attribute is determined according to the position attribute, the distance attribute, and the direction attribute.

Images