Method and device for determining patrolled point and method for optimizing attribute data of patrolled object

Abstract

The invention discloses a method and device for determining a patrolled point and a method for optimizing attribute data of a patrolled object, and the method comprises the steps: obtaining a cutting viewport for representing the patrolled object through the fitting of the geometric shape of the patrolled object; then determining a virtual view space formed when a light beam is emitted from the projection center to the cutting viewport under the constraint of a visual angle, and then identifying and rejecting an area which shields the virtual view space to obtain a safe visual area; then, on the premise of ensuring that the cutting viewport does not lose features, calculating to obtain a shooting area of the patrol equipment in a virtual view space range; and determining a final patrol point of the patrolled object according to an intersection area under the condition that the intersection area exists between the safe visible area and the shooting area. Therefore, the final inspection point is solved in the intersection area obtained through calculation, a more reasonable and accurate inspection position can be obtained, and the success rate, accuracy and safety of an intelligent electric power inspection task are improved.

Description

Technical Field

[0001] The present invention relates to the field of substation inspection, and in particular to a method and device for determining inspected points and a method for optimizing attribute data of inspected objects. Background Art

[0002] At present, one of the key technologies in the intelligent power inspection method is the automatic planning of inspection routes so that drones or ground robots (hereinafter referred to as inspection equipment) can collect data on the real scene being inspected (hereinafter referred to as the inspected scene). The inspection route is represented by a series of points (hereinafter referred to as inspection points), among which the collected data can be at least one of multiple data such as multi-perspective images, videos and laser point clouds, which can be used for digital analysis from the perspective of safety.

[0003] However, the patrol scene is often very complex due to the large number of power components, compact deployment and environmental interference. The patrol equipment needs to be kept at a safe distance to take into account power threats, and a target object that needs to be patrolled (hereinafter referred to as the patrol object) may be blocked by other objects around it. However, in the process of planning the above patrol routes, the existing methods do not fully take into account the obstruction and safety, which easily leads to unreasonable posture of the patrol point relative to the patrol object.

[0004] How to find a more reasonable and accurate optimal inspection position relative to the inspected object to improve the success rate, accuracy and safety of intelligent power inspection tasks is an urgent problem to be solved. Summary of the invention

[0005] The present invention provides a method and device for determining patrol points and a method for optimizing attribute data of patrol objects, aiming to obtain more reasonable and accurate optimal patrol positions, so as to improve the success rate, accuracy and safety of intelligent power patrol tasks.

[0006] An embodiment of the present invention provides a method for determining patrolled points, including:

[0007] Obtain the geometry of the patrolled object, the type of the patrolled object, the type of patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle;

[0008] Performing plane fitting on the geometric shape of the inspected object, and using the plane area obtained by fitting as a clipping viewport for characterizing the inspected object;

[0009] Determining, according to the type of the patrolled object, the type of the patrol device and the safety distance, a viewing angle constraint when the safety distance is met;

[0010] Under the visual angle constraint, determining a projection center and a virtual field of view space formed when a light beam is emitted from the projection center to the clipping viewport;

[0011] Identify and eliminate areas that block the virtual visual field to obtain a safe visible area;

[0012] Determine, based on the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropping viewport, a photographable area of ​​the patrol device within the virtual field of view, while ensuring that the cropping viewport does not lose features;

[0013] In the case where there is an intersection area between the safe visible area and the photographable area, a final patrol point of the patrolled object is determined according to the intersection area.

[0014] Furthermore, the determination of the projection center includes:

[0015] Determine the geometric center of the clipping viewport;

[0016] Draw a perpendicular line of the clipping viewport through the geometric center;

[0017] A right-handed coordinate system is constructed by taking the intersection of the two sides of the clipping viewport as the coordinate origin O, taking any side of the intersection as the Z axis, taking a straight line perpendicular to the clipping viewport and passing through the coordinate origin as the X axis, and taking a straight line passing through the coordinate origin and perpendicular to both the Z axis and the X axis as the Y axis;

[0018] Taking the coordinate origin as a fulcrum, according to the angle corresponding to the viewing angle constraint, the straight line located on the X-axis is rotated in the Y-axis direction on the XOY plane to obtain a rotated straight line;

[0019] The intersection point of the rotated straight line and the vertical line is taken as the projection center.

[0020] Furthermore, the identifying and eliminating the area that blocks the virtual visual field space to obtain a safe visual area includes:

[0021] Obtain a set of patches used to reflect the surface of the inspected object;

[0022] Calculating the geometric center point of each facet in the facet set, and constructing a corresponding unit circle with each geometric center point as the center of the circle;

[0023] Sampling a local area of ​​each unit circle in the virtual field of view space to obtain a plurality of sampling points;

[0024] Dividing the virtual visual field space into a multi-level grid, and setting the attributes of the multi-level grid as a visible grid;

[0025] Taking each geometric center point as the starting point and each sampling point as the direction, project the light;

[0026] Performing occlusion detection on each of the projection rays in turn, and after completing the occlusion detection on all the projection rays, taking the area corresponding to the multi-level grid whose attributes are adjusted to be an invisible grid as the area constituting occlusion of the virtual field of view space;

[0027] According to the safety distance, determine the safety warning area corresponding to each obstacle;

[0028] Eliminate the safety warning areas corresponding to the obstacles and the areas that block the virtual visual field from the virtual visual field to obtain a safe visual area;

[0029] Wherein, the occlusion detection includes:

[0030] Identify whether the current projection light is blocked by an obstacle.

[0031] If yes, adjust the attributes of the multi-level grid whose attributes are visible grids behind the obstacle to invisible grids;

[0032] If not, the properties of the multi-level grid are not adjusted.

[0033] Furthermore, the determining, based on the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the cropping viewport, of the photographable area of ​​the patrol device within the virtual field of view, while ensuring that the cropping viewport does not lose features, includes:

[0034] Determine the farthest shooting distance according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset first zoom ratio; wherein the first zoom ratio is the minimum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be fully displayed;

[0035] Determine the minimum shooting distance according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset second zoom ratio; wherein the second zoom ratio is the maximum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be fully displayed;

[0036] According to the shortest shooting distance and the farthest shooting distance, a shootable area of ​​the patrol device within the virtual field of view space is determined.

[0037] Furthermore, when there is an intersection area between the safe visible area and the photographable area, determining the final patrol point of the patrolled object according to the intersection area includes:

[0038] Calculating a normal vector for each face in the face set;

[0039] Randomly generate several initial patrol points;

[0040] For each initial patrol point, a relationship score of the current initial patrol point relative to each facet in the facet set is calculated; wherein the relationship score is determined by a balance angle and distance between the patrol point and the facet;

[0041] The initial patrol point with the highest score above the preset threshold is taken as the final patrol point.

[0042] Furthermore, the method for determining the patrolled points further includes:

[0043] If there is no relationship score higher than the preset threshold, other different types of patrol devices are used instead.

[0044] Furthermore, the method for determining the patrolled points further includes:

[0045] When there is no intersection between the safe visual area and the photographable area, other different types of patrol equipment are used instead.

[0046] An embodiment of the present invention further provides a device for determining patrol points, including: a data acquisition module, a plane fitting module, a visual angle constraint determination module, a virtual field of view space determination module, a safe visual area determination module, a photographable area determination module, and a final patrol point determination module;

[0047] The data acquisition module is used to acquire the geometric shape of the patrolled object, the type of the patrolled object, the type of patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle;

[0048] The plane fitting module is used to perform plane fitting on the geometric shape of the inspected object, and use the plane area obtained by fitting as a clipping viewport for representing the inspected object;

[0049] The visual angle constraint determination module is used to determine the visual angle constraint when the safety distance is met according to the type of the patrolled object, the type of the patrol device and the safety distance;

[0050] The virtual field of view space determination module is used to determine the projection center and the virtual field of view space formed when a light beam is emitted from the projection center to the clipping viewport under the visual angle constraint;

[0051] The safe visual area determination module is used to identify and remove the area that blocks the virtual visual field space to obtain a safe visual area;

[0052] The photographable area determination module is used to determine the photographable area of ​​the patrol device within the virtual field of view space under the premise of ensuring that the cropped viewport does not lose features, based on the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropped viewport;

[0053] The final patrol point determination module is used to determine the final patrol point of the patrolled object according to the intersection area when there is an intersection area between the safe visual area and the photographable area.

[0054] An embodiment of the present invention further provides a method for optimizing attribute data of an inspected object, comprising:

[0055] Calculating the spatial attribute between the inspected object and the final inspection point; wherein the final inspection point is determined by the inspection point determination method;

[0056] Optimized attribute data of the inspected object is generated according to the spatial attributes, the geometric shape of the inspected object and the type of the inspected object.

[0057] Furthermore, the spatial attribute is determined by the following method:

[0058] Calculate the geometric center of the outer bounding box generated by the face set to obtain a position attribute;

[0059] Calculating the distance between the final patrol point and the geometric center of the outer bounding box to obtain a distance attribute;

[0060] Calculate the direction from the geometric center of the outer bounding box to the final patrol point to obtain a direction attribute;

[0061] The spatial attribute is determined according to the position attribute, the distance attribute, and the direction attribute.

[0062] The following beneficial effects are achieved by implementing the present invention:

[0063] The present invention provides a method for determining patrol points. The method determines the visual angle constraint under the condition of satisfying the safety distance by acquiring the type of patrol object, the type of patrol equipment, and the safety distance between the patrol equipment and each obstacle, and determines the projection center under the visual angle constraint. Then, through the projection center, a light beam is emitted to a clipping viewport used to characterize the patrol object to form a pseudo visual field space, and the area that blocks the virtual visual field space is removed from the virtual visual field space, so that the finally formed safe visual area fully takes into account the occlusion and safety.

[0064] Then, according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device and the cropping viewport, the photographable area of ​​the patrol device within the virtual field of view is determined under the premise that the cropping viewport does not lose features, thereby avoiding the situation where the features of the patrol object cannot be distinguished due to the image being too small, and the patrol object cannot be fully displayed in the image due to the image being too large;

[0065] In summary, the safe visible area and the photographable area are used as the key basis for determining the final inspection point, and the final inspection point is extracted in the intersection area, so that when subsequent inspections are carried out based on the final inspection point, the quality of imaging can be guaranteed while ensuring the safety of the shooting process, so that the final inspection point can meet the rationality and accuracy of actual inspection business. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the implementation manner will be briefly introduced below. Obviously, the drawings described below are only some implementation manners of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 It is a flowchart of a method for determining patrolled points provided in a certain embodiment of the present application;

[0068] Figure 2 This is a schematic diagram of projection toward the center of a cropping viewport provided by a certain embodiment of the present application;

[0069] Figure 3 This is a schematic diagram of observing a virtual field of view space in the negative direction of the Z axis provided by a certain embodiment of the present application;

[0070] Figure 4 is a schematic diagram of a visible area provided in a certain embodiment of the present application;

[0071] Figure 5 is a schematic diagram of a safe visual area provided in a certain embodiment of the present application;

[0072] Figure 6 is a schematic diagram of an imaging viewing angle, a maximum shooting distance, and a minimum shooting distance corresponding to a virtual field of view space provided by a certain embodiment of the present application;

[0073] Figure 7 is a schematic diagram of a photographable area provided in a certain embodiment of the present application;

[0074] Figure 8 is a schematic diagram of an intersection area provided in a certain embodiment of the present application;

[0075] Fig. 9is a schematic diagram of the effect of a patrol device provided in a certain embodiment of the present application on an intersection area;

[0076] Fig.10 It is a structural schematic diagram of a device for determining patrolled points provided in a certain embodiment of the present application;

[0077] Fig.11 It is a flowchart of a method for optimizing attribute data of an inspected object provided in a certain embodiment of the present application;

[0078] Fig.12 It is a schematic diagram of the attribute data of the inspected object provided by a certain embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0081] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0082] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0083] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0084] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0085] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0086] See also Figure 1 , is a flow chart of a method for determining patrolled points provided by an embodiment of the present invention, comprising:

[0087] S1. Obtaining the geometric shape of the patrolled object, the type of the patrolled object, the type of patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle;

[0088] Illustratively, it is necessary to obtain the inspected object data including the geometric shape of the inspected object and the type of the inspected object, wherein the inspected object data is obtained from a three-dimensional scene model;

[0089] Specifically, the 3D scene model can be any one of a mesh model, a DEM model and a point cloud model; for example, when the 3D mesh model is used to represent the surface geometric details of the inspected scene, some triangular facets in the 3D mesh model are extracted as the data of the inspected object based on the correspondence between the multi-view image and the 3D mesh model; wherein the 3D mesh model is obtained by 3D reconstruction based on the multi-view image; the facet set obtained by extraction represents a local surface of the inspected object, which can at least reflect the geometric shape of the inspected object in the 3D space; the geometric center of the outer bounding box of the facet set is taken as the position of the inspected object;

[0090] Indicatively, it is necessary to obtain patrol device data including the type of patrol device, the focal length of the camera in the patrol device, and the imaging angle of the camera in the patrol device;

[0091] Specifically, different patrol equipment is equipped with cameras with different parameters, such as: SLR cameras used for manual patrols, multi-zoom cameras used for robot patrols, and airborne cameras used for drone patrols;

[0092] Indicatively, it is necessary to obtain the electronic fence data including the safe distance between the patrol device and each obstacle;

[0093] Specifically, the electronic fence data includes safety distance data that matches the three-dimensional scene model, for example, the safety distance between the drone and the trees during patrol.

[0094] S2, performing plane fitting on the geometric shape of the inspected object, and using the plane area obtained by fitting as a clipping viewport for representing the inspected object;

[0095] Illustratively, for a certain inspected object, in order to simplify the geometric representation, the RANSAC algorithm may be used to perform plane fitting on the inspected object data, and the plane area obtained by fitting is used as a clipping viewport for characterizing the inspected object.

[0096] S3, determining a viewing angle constraint under the condition of satisfying the safety distance according to the type of the patrolled object, the type of the patrol device and the safety distance;

[0097] Indicatively, according to the type of the patrol object, the type of the patrol device and the safety distance obtained in step S1, a table is looked up to determine the viewing angle constraint when the safety distance is met; wherein the viewing angle constraint corresponds to a viewing angle α;

[0098] Specifically, the type of the inspected object, the type of the inspection device, the safety distance and the preset viewing angle α are pre-associated and stored in a preset data table; or, the type of the inspected object is pre-associated with the camera type and the viewing angle α and stored in a first data table, and the type of the inspected object is pre-associated with the camera type and the safety distance and stored in a second data table; when the viewing angle constraint is needed, the corresponding viewing angle constraint and the viewing angle corresponding to the viewing angle constraint can be obtained by searching the data table.

[0099] S4. Under the visual angle constraint, determine a projection center and a virtual field of view space formed when a light beam is emitted from the projection center to the clipping viewport;

[0100] Schematically, the virtual field of view space R refers to the space where light continues to propagate after passing through a patrolled object. The virtual field of view space R is affected by the visual angle constraint. Therefore, before obtaining the virtual field of view space R, the projection center needs to be determined first;

[0101] In a preferred embodiment, the determination of the projection center includes:

[0102] Determine the geometric center of the clipping viewport;

[0103] Draw a perpendicular line of the clipping viewport through the geometric center;

[0104] A right-handed coordinate system is constructed by taking the intersection of the two sides of the clipping viewport as the coordinate origin O, taking any side of the intersection as the Z axis, taking a straight line perpendicular to the clipping viewport and passing through the coordinate origin as the X axis, and taking a straight line passing through the coordinate origin and perpendicular to both the Z axis and the X axis as the Y axis;

[0105] Taking the coordinate origin as a fulcrum, according to the angle corresponding to the viewing angle constraint, the straight line located on the X-axis is rotated in the Y-axis direction on the XOY plane to obtain a rotated straight line;

[0106] The intersection of the rotated straight line and the vertical line is taken as the projection center;

[0107] Specifically, see Figure 2 , taking the cropping viewport in a triangle shape as an example, a right-handed coordinate system is constructed for the cropping viewport, wherein the intersection of the two sides of the cropping viewport is taken as the coordinate origin O, any side of the intersection is taken as the Z axis, the straight line perpendicular to the cropping viewport and passing through the coordinate origin is taken as the X axis, and the straight line passing through the coordinate origin and perpendicular to both the Z axis and the X axis is taken as the Y axis;

[0108] Specifically, see Figure 3 , taking the coordinate origin as a fulcrum, according to the viewing angle α corresponding to the viewing angle constraint, rotating the straight line located on the X-axis in the Y-axis direction on the XOY plane to obtain a rotated straight line, and then taking the intersection of the rotated straight line and the vertical line as the projection center;

[0109] Thus, the obtained projection center is located right in front of the clipping viewport, that is, the angle between the vector obtained by connecting the projection center and the geometric center of the clipping viewport and the positive direction of the X-axis is 0 degrees; wherein the positive direction of the X-axis represents the direction in which a certain light ray emitted from the projection center continues to propagate in a straight line after vertically passing through the clipping viewport;

[0110] Specifically, after the projection center is obtained, the light beam emitted to the clipping viewport according to the central projection principle will form a virtual field of view space R behind the clipping viewport.

[0111] S5, identifying and eliminating the area that blocks the virtual visual field space to obtain a safe visible area;

[0112] Indicatively, since the patrolled scene data in the three-dimensional scene model may contain other object data that may block the line of sight of the patrolled object or / and threaten the safety, it is necessary to perform bounding box detection on the object data other than the patrolled object data, and regard the detected bounding box as an obstacle;

[0113] After obtaining the obstacle information in the patrolled scene, it is necessary to remove the area that blocks the virtual field of view space R;

[0114] In a preferred embodiment, the identifying and eliminating the area that blocks the virtual visual field space to obtain a safe visual area includes:

[0115] Obtain a set of patches used to reflect the surface of the inspected object;

[0116] Calculating the geometric center point of each facet in the facet set, and constructing a corresponding unit circle with each geometric center point as the center of the circle;

[0117] Sampling a local area of ​​each unit circle in the virtual field of view space to obtain a plurality of sampling points;

[0118] Dividing the virtual visual field space into a multi-level grid, and setting the attributes of the multi-level grid as a visible grid;

[0119] Taking each geometric center point as the starting point and each sampling point as the direction, project the light;

[0120] Performing occlusion detection on each of the projection rays in turn, and after completing the occlusion detection on all the projection rays, taking the area corresponding to the multi-level grid whose attributes are adjusted to be an invisible grid as the area constituting occlusion of the virtual field of view space;

[0121] According to the safety distance, determine the safety warning area corresponding to each obstacle;

[0122] Eliminate the safety warning areas corresponding to the obstacles and the areas that block the virtual visual field from the virtual visual field to obtain a safe visual area;

[0123] Wherein, the occlusion detection includes:

[0124] Identify whether the current projection light is blocked by an obstacle.

[0125] If yes, adjust the attributes of the multi-level grid whose attributes are visible grids behind the obstacle to invisible grids;

[0126] If not, the properties of the multi-level grid are not adjusted;

[0127] Specifically, the geometric center point of each patch in the patch set is calculated, and a corresponding unit circle is constructed with each geometric center point as the center of the circle, and a local area of ​​each unit circle in the virtual field of view space R is sampled by any of a variety of methods such as uniform sampling, layered sampling, random sampling, etc., to obtain a number of corresponding sampling points;

[0128] Next, taking each geometric center point as the starting point, projecting light to each sampling point, and sequentially performing occlusion detection on each projected light. For any projected light, after the projected light encounters an obstacle for the first time, the attributes of the multi-level grids that are automatically located behind the obstacle and are attributes of visible grids are adjusted to invisible grids;

[0129] It should be noted that once the attribute of the multi-level grid becomes an invisible grid, it cannot enter the occlusion detection of the next projection ray;

[0130] See also Figure 4 After completing the occlusion detection of all projected rays, the area corresponding to the multi-level grid whose attributes are adjusted to the invisible grid is used as the area that constitutes the occlusion of the virtual field of view space, and the area that constitutes the occlusion of the virtual field of view space is removed from the virtual field of view space to obtain the visible area A', where A' represents the space that will not be blocked when the camera faces the patrolled object;

[0131] See also Figure 5 Next, it is necessary to determine the safety warning area corresponding to each obstacle according to the safety distance between the patrol equipment and each obstacle, that is, expand a certain range around the corresponding obstacle according to each safety distance as the safety warning area;

[0132] Finally, the safety warning areas corresponding to the obstacles are removed from the visual area A' to obtain the final safe visual area A.

[0133] S6. Determine, based on the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the cropping viewport, a photographable area of ​​the patrol device within the virtual field of view, while ensuring that the cropping viewport does not lose features;

[0134] In a preferred embodiment, the determining, based on the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropping viewport, the photographable area of ​​the patrol device within the virtual field of view, while ensuring that the cropping viewport does not lose features, includes:

[0135] Determine the farthest shooting distance according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset first zoom ratio; wherein the first zoom ratio is the minimum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be fully displayed;

[0136] Determine the minimum shooting distance according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset second zoom ratio; wherein the second zoom ratio is the maximum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be fully displayed;

[0137] Determine the photographable area of ​​the patrol device within the virtual field of view space according to the shortest photographing distance and the farthest photographing distance;

[0138] Instructively, the size of the cropped viewport is represented as H, the distance between the cropped viewport and the patrol device is represented as d, the focal length of the camera in the patrol device is represented as f, the imaging angle of the camera in the patrol device is represented as θ, and the size of the cropped viewport imaged by the camera is represented as h. Therefore, the above parameters satisfy the following formula:

[0139]

[0140] Specifically, see Figure 6 Since the focal length f of the camera in the patrol device and the imaging angle θ of the camera in the patrol device are known quantities, when the scaling ratio is the first scaling ratio, it can be determined that the farthest shooting distance between the cropping viewport and the patrol device is d max , where the first scaling ratio is expressed as the ratio corresponding to the minimum h / H;

[0141] Likewise, see Figure 6 Since the focal length f of the camera in the patrol device and the imaging angle θ of the camera in the patrol device are known quantities, when the scaling ratio is the second scaling ratio, it can be determined that the shortest shooting distance between the cropping viewport and the patrol device is d min , where the second scaling ratio is expressed as the ratio corresponding to the maximum h / H;

[0142] Take the crop viewport as the target, get the longest diagonal from the image, draw a circle with the center of the diagonal as the center and the diagonal as the diameter, and calculate the diameter d in each direction min and d max , d min and d max The curves formed are rotated along the axis perpendicular to the center of the circle, and the obtained min The corresponding first cutting surface and d max The corresponding second cutting surface, see Figure 7, a photographable area B is enclosed based on the first cutting plane, the second cutting plane, and two straight line boundaries distributed on both sides of the virtual field of view space R.

[0143] S7, when there is an intersection area between the safe visible area and the photographable area, determining a final patrol point of the patrolled object according to the intersection area;

[0144] See also Figure 8 Specifically, it is determined whether the safe visual area A and the photographable area B have an intersection. If This means that there is an intersection area A∩B, where the calculation method of the intersection area is not unique, for example, it can be calculated by boundary constraints, voxel segmentation, etc.

[0145] In a preferred embodiment, when there is an intersection area between the safe visible area and the photographable area, determining the final patrol point of the patrolled object according to the intersection area includes:

[0146] Calculating a normal vector for each face in the face set;

[0147] Randomly generate several initial patrol points;

[0148] For each initial patrol point, a relationship score of the current initial patrol point relative to each facet in the facet set is calculated; wherein the relationship score is determined by a balance angle and distance between the patrol point and the facet;

[0149] The initial inspection point with the highest score above the preset threshold is taken as the final inspection point;

[0150] Specifically, calculate the normal vector of each face in the face set T Where T={T1,T2,…,T i ,T n}, n represents the total number of patches; then find the final patrol point P from the intersection area A∩B * , so that the final patrol point P * The relationship score F(P) with the positions and normal vectors of all triangles is the highest;

[0151] Specifically, the final patrol point is calculated by the following formula:

[0152]

[0153] Among them, F(P) represents the average score of the initial patrol point P in each patch, which is calculated by the following formula:

[0154]

[0155] Among them, fi (P) represents the initial patrol point P relative to the triangle patch T i The score is calculated by the following formula:

[0156]

[0157] Among them, θ i Represents the initial patrol point P and the triangle patch T i Connection With normal vector The angle between i Represents the initial patrol point P and the triangle patch T i distance; α represents a first weight coefficient, β represents a second weight coefficient, the first weight coefficient is used to determine the influence of the balance angle on the score, and the second weight coefficient is used to determine the influence of the distance on the score.

[0158] In a preferred embodiment, the method for determining the patrolled points further includes:

[0159] If there is no relationship score higher than the preset threshold, other different types of patrol devices are used instead;

[0160] Specifically, assuming that the relationship scores F(P) of each initial patrol point currently calculated are all less than the preset threshold S, it is necessary to replace and use other different types of patrol equipment;

[0161] If after replacing all patrol devices, it is found that the intersection of the safe visible area A and the photographable area B is an empty set or the scores of the initial patrol points are all lower than the preset threshold S, it is considered that no patrol points that meet the requirements can be obtained in the currently obtained safe visible area A and the photographable area B, so it is necessary to adjust the patrol object, that is, to refit the plane of the patrol object.

[0162] In a preferred embodiment, the method for determining the patrolled points further includes:

[0163] In the case where there is no intersection between the safe visual area and the photographable area, other different types of patrol equipment are used instead;

[0164] Specifically, if It means that it is not possible to photograph the patrolled object under safe conditions, and it is necessary to use other different types of patrol equipment;

[0165] Specifically, see Fig. 9 , assuming that a patrolled object is a meter installed in the middle of a device, in order to obtain the meter reading, a telephoto camera can be used to shoot the front of the meter from a distance under safe conditions (i.e. Fig. 9The solid black dot in the middle of the meter), and using the camera on the drone to shoot the front of the meter at a close distance will face security threats because it is necessary to enter the electronic fence to shoot (i.e. Fig. 9 Therefore, if there is no intersection area, the patrol equipment needs to be replaced in time;

[0166] It is understandable that the parameters of the cameras in different patrol devices can be associated with different viewing angles α, for example, the meter reflects light because it has a glass protective cover, and the camera faces the glass protective cover of the meter to shoot, and the camera is equipped with a polarizing filter to eliminate the reflection. In this case, the camera parameter can be defined as x1, and the α corresponding to x1 can be set to 60°; otherwise, the camera parameter can be defined as x2, x2 is different from x1, and the α corresponding to x2 can be set to 30°;

[0167] When the inspected object and the inspected scene remain unchanged, the safe visual area A is only related to the value of the visual angle α. After changing the camera parameters, if the required visual angle α2 (for easy distinction, it can be defined as α2) after the change is equal to the required visual angle α1 (can be defined as α1) at a certain time after the change, A remains unchanged; if 0<α2-α1<Δα, only the overlapping part of the space before and after the change needs to be calculated to detect the new intersection area A∩B.

[0168] Embodiment 2

[0169] See also Fig.10 , is a device for determining patrol points provided by a certain embodiment of the present application, comprising: a data acquisition module, a plane fitting module, a visual angle constraint determination module, a virtual field of view space determination module, a safe visual area determination module, a photographable area determination module and a final patrol point determination module;

[0170] The data acquisition module is used to acquire the geometric shape of the patrolled object, the type of the patrolled object, the type of patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle;

[0171] The plane fitting module is used to perform plane fitting on the geometric shape of the inspected object, and use the plane area obtained by fitting as a clipping viewport for representing the inspected object;

[0172] The visual angle constraint determination module is used to determine the visual angle constraint when the safety distance is met according to the type of the patrolled object, the type of the patrol device and the safety distance;

[0173] The virtual field of view space determination module is used to determine the projection center and the virtual field of view space formed when a light beam is emitted from the projection center to the clipping viewport under the visual angle constraint;

[0174] The safe visual area determination module is used to identify and remove the area that blocks the virtual visual field space to obtain a safe visual area;

[0175] The photographable area determination module is used to determine the photographable area of ​​the patrol device within the virtual field of view space under the premise of ensuring that the cropped viewport does not lose features, based on the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropped viewport;

[0176] The final patrol point determination module is used to determine the final patrol point of the patrolled object according to the intersection area when there is an intersection area between the safe visual area and the photographable area.

[0177] Embodiment 3

[0178] See also Fig.11 , is a method for optimizing attribute data of an inspected object provided in a certain embodiment of the present application, comprising:

[0179] D1. Calculate the spatial attribute between the inspected object and the final inspection point; wherein the final inspection point is determined by the inspection point determination method;

[0180] In a preferred embodiment, the spatial attribute is determined by the following method:

[0181] Calculate the geometric center of the outer bounding box generated by the face set to obtain a position attribute;

[0182] Calculating the distance between the final patrol point and the geometric center of the outer bounding box to obtain a distance attribute;

[0183] Calculate the direction from the geometric center of the outer bounding box to the final patrol point to obtain a direction attribute;

[0184] Determine the spatial attribute according to the position attribute, the distance attribute and the direction attribute;

[0185] Specifically, the final patrol point is the final patrol point determined by any of the patrol point determination method embodiments of the present invention;

[0186] Specifically, after obtaining the final patrol point through the first embodiment, it is necessary to first calculate the geometric center of the outer bounding box generated by the face set to obtain the position attribute; then, calculate the distance between the final patrol point and the geometric center of the outer bounding box to obtain the distance attribute; finally, it is necessary to calculate the direction from the geometric center of the outer bounding box to the final patrol point to obtain the direction attribute;

[0187] Specifically, the position attribute, the distance attribute, and the direction attribute together constitute the optimized spatial attribute.

[0188] D2. generating optimized attribute data of the inspected object according to the spatial attribute, the geometric shape of the inspected object and the type of the inspected object;

[0189] Specifically, see Fig.12 , generating optimized attribute data of the inspected object according to the spatial attribute, the geometric form of the inspected object and the type of the inspected object, wherein the geometric form of the inspected object and the type of the inspected object together constitute the inspected object data;

[0190] Therefore, the optimized attribute data of the inspected objects can help simplify subsequent application processes such as inspection route planning and improve the overall application effect. At the same time, it can better manage and maintain the inspected objects, reduce inspection costs, and improve efficiency and reliability.

[0191] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for determining patrol points, characterized in that: include: Obtain the geometry of the patrolled object, the type of the patrolled object, the type of patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle; Performing plane fitting on the geometric shape of the inspected object, and using the plane area obtained by fitting as a clipping viewport for characterizing the inspected object; Determining, according to the type of the patrolled object, the type of the patrol device and the safety distance, a viewing angle constraint when the safety distance is met; Under the visual angle constraint, determining a projection center and a virtual field of view space formed when a light beam is emitted from the projection center to the clipping viewport; Identify and eliminate areas that block the virtual visual field to obtain a safe visible area; Determine, based on the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropping viewport, a photographable area of ​​the patrol device within the virtual field of view, while ensuring that the cropping viewport does not lose features; In the case where there is an intersection area between the safe visible area and the photographable area, a final patrol point of the patrolled object is determined according to the intersection area.

2. The method for determining patrolled points according to claim 1, characterized in that: The determination of the projection center includes: Determine the geometric center of the clipping viewport; Draw a perpendicular line of the clipping viewport through the geometric center; A right-handed coordinate system is constructed by taking the intersection of the two sides of the clipping viewport as the coordinate origin O, taking any side of the intersection as the Z axis, taking a straight line perpendicular to the clipping viewport and passing through the coordinate origin as the X axis, and taking a straight line passing through the coordinate origin and perpendicular to both the Z axis and the X axis as the Y axis; Taking the coordinate origin as a fulcrum, according to the angle corresponding to the viewing angle constraint, the straight line located on the X-axis is rotated in the Y-axis direction on the XOY plane to obtain a rotated straight line; The intersection point of the rotated straight line and the vertical line is taken as the projection center.

3. The method for determining patrolled points according to claim 1, characterized in that: The identifying and eliminating the area that blocks the virtual visual field space to obtain a safe visual area includes: Obtain a set of patches used to reflect the surface of the inspected object; Calculating the geometric center point of each facet in the facet set, and constructing a corresponding unit circle with each geometric center point as the center of the circle; Sampling a local area of ​​each unit circle in the virtual field of view space to obtain a plurality of sampling points; Dividing the virtual visual field space into a multi-level grid, and setting the attributes of the multi-level grid as a visible grid; Taking each geometric center point as the starting point and each sampling point as the direction, project the light; Performing occlusion detection on each of the projection rays in turn, and after completing the occlusion detection on all the projection rays, taking the area corresponding to the multi-level grid whose attributes are adjusted to be an invisible grid as the area constituting occlusion of the virtual field of view space; According to the safety distance, determine the safety warning area corresponding to each obstacle; Eliminate the safety warning areas corresponding to the obstacles and the areas that block the virtual visual field from the virtual visual field to obtain a safe visual area; Wherein, the occlusion detection includes: Identify whether the current projection light is blocked by an obstacle. If yes, adjust the attributes of the multi-level grid whose attributes are visible grids behind the obstacle to invisible grids; If not, the properties of the multi-level grid are not adjusted.

4. The method for determining patrolled points according to claim 1, characterized in that: The determining, based on the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropping viewport, of a photographable area of ​​the patrol device within the virtual field of view, while ensuring that the cropping viewport does not lose features, includes: Determine the farthest shooting distance according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset first zoom ratio; wherein the first zoom ratio is the minimum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be fully displayed; Determine the minimum shooting distance according to the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and a preset second zoom ratio; wherein the second zoom ratio is the maximum ratio between the actual size of the cropped viewport and the imaging size under the premise that the imaging can be fully displayed; According to the shortest shooting distance and the farthest shooting distance, a shootable area of ​​the patrol device within the virtual field of view space is determined.

5. The method for determining patrolled points according to claim 3, characterized in that: When there is an intersection area between the safe visible area and the photographable area, determining the final patrol point of the patrolled object according to the intersection area includes: Calculating a normal vector for each face in the face set; Randomly generate several initial patrol points; For each initial patrol point, a relationship score of the current initial patrol point relative to each facet in the facet set is calculated; wherein the relationship score is determined by a balance angle and distance between the patrol point and the facet; The initial patrol point with the highest score above the preset threshold is taken as the final patrol point.

6. The method for determining patrolled points according to claim 5, characterized in that: Also includes: If there is no relationship score higher than the preset threshold, other different types of patrol devices are used instead.

7. The method for determining patrolled points according to claim 1, characterized in that: Also includes: When there is no intersection between the safe visual area and the photographable area, other different types of patrol equipment are used instead.

8. A device for determining patrol points, characterized in that: include: Data acquisition module, plane fitting module, visual angle constraint determination module, virtual field of view space determination module, safe visual area determination module, photographable area determination module and final patrol point determination module; The data acquisition module is used to acquire the geometric shape of the patrolled object, the type of the patrolled object, the type of patrol device, the focal length of the camera in the patrol device, the imaging angle of the camera in the patrol device, and the safe distance between the patrol device and each obstacle; The plane fitting module is used to perform plane fitting on the geometric shape of the inspected object, and use the plane area obtained by fitting as a clipping viewport for representing the inspected object; The visual angle constraint determination module is used to determine the visual angle constraint when the safety distance is met according to the type of the patrolled object, the type of the patrol device and the safety distance; The virtual field of view space determination module is used to determine the projection center and the virtual field of view space formed when a light beam is emitted from the projection center to the clipping viewport under the visual angle constraint; The safe visual area determination module is used to identify and remove the area that blocks the virtual visual field space to obtain a safe visual area; The photographable area determination module is used to determine the photographable area of ​​the patrol device within the virtual field of view space under the premise of ensuring that the cropped viewport does not lose features, based on the focal length of the camera in the patrol device, the imaging angle of view of the camera in the patrol device, and the cropped viewport; The final patrol point determination module is used to determine the final patrol point of the patrolled object according to the intersection area when there is an intersection area between the safe visual area and the photographable area.

9. A method for optimizing attribute data of an inspected object, characterized in that: include: Calculating the spatial attribute between the inspected object and the final inspection point; wherein the final inspection point is determined by the method for determining the inspected point according to any one of claims 1 to 7; Optimized attribute data of the inspected object is generated according to the spatial attributes, the geometric shape of the inspected object and the type of the inspected object.

10. The method for optimizing the attribute data of the inspected object according to claim 9, characterized in that: The spatial properties are determined by: Calculate the geometric center of the outer bounding box generated by the face set to obtain a position attribute; Calculating the distance between the final patrol point and the geometric center of the outer bounding box to obtain a distance attribute; Calculate the direction from the geometric center of the outer bounding box to the final patrol point to obtain a direction attribute; The spatial attribute is determined according to the position attribute, the distance attribute, and the direction attribute.

Images