Power plant air cooling island mobile robot control system

Abstract

The application relates to the technical field of mobile robot control, in particular to a power plant air cooling island mobile robot control system, and the following scheme is provided: images are collected at preset intervals, target structure edge features are detected, and a collection position is predicted based on structure integrity and extension direction; whether the images are included in a target image set to be inspected is judged by using image entropy features, a structure growth tree is constructed to dynamically generate a collection path, and target position information and structure features are extracted based on the target image set to be inspected, and an inspection control strategy is generated. The application realizes efficient modeling under the condition of no prior target information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile robot control, and particularly relates to a mobile robot control system for air cooling island of power plant. BACKGROUND

[0002] In the prior art, before performing a track inspection task, a mobile robot usually relies on preset inspection target coordinates, manually configured target regions or static image templates to complete path planning and initialization of a target recognition model. However, in a complex or dynamic field environment, the position distribution of an inspection target often has uncertainty, and the target structure has various forms, often showing fuzzy boundaries and strong spatial extension, and it is difficult to complete effective recognition and model configuration through static rules.

[0003] The prior art lacks a response mechanism for target structure changes, resulting in information redundancy or missing of key structures in an image set. To solve the above problems, the present application designs a mobile robot control system for air cooling island of power plant. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a mobile robot control system for air cooling island of power plant to collect images at preset intervals, detect edge features of a target structure, and predict a collection position based on structural integrity and extension direction, judge whether an image is included in a target image set to be inspected based on image entropy features, construct a structure growth tree to dynamically generate a collection path, and extract target position information and structural features based on the target image set to be inspected, and generate an inspection model.

[0005] To achieve the above purpose, the present application provides the following technical scheme:

[0006] A mobile robot control system for air cooling island of power plant, the mobile robot is configured with an image collection module, an instruction generation module, an iterative control module and a strategy generation module, wherein:

[0007] The image collection module is configured to control the mobile robot to move along a track and acquire images on both sides of the track at preset intervals until at least part of a pattern of a target to be inspected appears in the images on both sides of the track, and acquire a current image collection frame in which the at least part of the pattern appears;

[0008] The instruction generation module is configured to generate a control instruction according to edge structural features of the current image collection frame, the control instruction being used to predict a collection position of a next image collection frame and control the mobile robot to acquire the next image collection frame;

[0009] The iteration control module is configured to compare the entropy feature of the next image acquisition frame with the entropy feature of the current image acquisition frame, determine whether to add the next image acquisition frame to the preset target image set to be inspected, replace the current image acquisition frame with the next image acquisition frame, and iteratively perform the image acquisition frame acquisition operation according to the control instruction until a plurality of continuous image acquisition frames are not added to the target image set to be inspected, and terminate the iteration.

[0010] The policy generation module is configured to extract image position information and structural feature information of the target to be inspected according to the target image set to be inspected, generate corresponding control instructions according to the image position information and structural feature information of a plurality of targets to be inspected, and obtain an inspection control policy.

[0011] The predicted acquisition position of the next image acquisition frame includes:

[0012] The extension direction and the transverse structural line of the edge structure feature are extracted.

[0013] The structural integrity is calculated according to the closeness, continuity and image coverage of the transverse structural line.

[0014] The acquisition position of the next image acquisition frame is predicted according to the extension direction and the structural integrity.

[0015] The predicted acquisition position of the next image acquisition frame includes:

[0016] The structural integrity is divided into a first level and a second level.

[0017] When the structural integrity is in the first level, the acquisition position of the next image acquisition frame is predicted at a first interval along the extension direction.

[0018] When the structural integrity is in the second level, the acquisition position of the next image acquisition frame is predicted at a second interval along the extension direction, and the second interval is smaller than the first interval.

[0019] The first interval and the second interval are generated by a structure growth tree constructed based on the extension direction and the structural integrity, wherein:

[0020] The structure growth tree takes the starting position of the structural integrity in the image as a root node, recursively grows along the extension direction to form a plurality of branches, and each node of each branch corresponds to a candidate image acquisition position, and the depth of the node is associated with the level of the structural integrity at the node.

[0021] The first interval corresponds to the branch interval distance of the structure growth tree in which the structural integrity is in the first level, and the second interval corresponds to the branch interval distance of the structure growth tree in which the structural integrity is in the second level.

[0022] The entropy feature of the image acquisition frame is calculated as follows:

[0023] The image acquisition frame is subjected to edge enhancement processing to obtain a structure edge map.

[0024] The structure edge map is divided into multiple image regions, and the pixel gray level histogram entropy value and the edge direction distribution standard deviation in each image region are calculated respectively.

[0025] The pixel gray level histogram entropy value of each image region is multiplied by the corresponding edge direction distribution standard deviation to obtain a weighted entropy value.

[0026] The weighted entropy values of the image regions are summed to obtain the entropy feature of the image acquisition frame.

[0027] The judgment condition for whether to add the next image acquisition frame to the preset set of target images to be inspected includes at least one of the following:

[0028] The absolute value of the difference between the entropy feature of the next image acquisition frame and the entropy feature of the current image acquisition frame is greater than a first preset change threshold;

[0029] The entropy feature of the next image acquisition frame is greater than a second preset entropy threshold;

[0030] In the local region entropy feature of the next image acquisition frame, the weighted entropy value of at least one image region is greater than the entropy feature of the current image acquisition frame.

[0031] When the next image acquisition frame is added to the preset set of target images to be inspected, the next image acquisition frame is taken as the current image acquisition frame, and iteration is performed;

[0032] When the next image acquisition frame is not added to the preset set of target images to be inspected, the mobile robot is returned to the acquisition position of the current image acquisition frame, and the structure growth tree corresponding to the current image acquisition frame is updated according to the entropy feature of the next image acquisition frame, wherein the update includes adjusting the node depth mapping relationship of the first interval and the second interval, and calculating the interval distance of the corresponding branch according to the entropy feature change rate.

[0033] When the iteration of the image acquisition frame acquisition operation ends, it further includes:

[0034] The mobile robot is controlled to move along the track and acquire images on both sides of the track at a preset interval until at least part of the pattern of the next target to be inspected appears in the images on both sides of the track.

[0035] According to the set of target images to be inspected, the image position information and the structure feature information of the target to be inspected are extracted, including:

[0036] The adjacent image collection frames in the to-be-inspected target image set are spliced to generate a target image splicing result covering a complete inspection target area;

[0037] The target image splicing result is subjected to image segmentation to identify the inspection target area in the image;

[0038] For each inspection target area, the position index information thereof in the spliced image is extracted, and the image position information of the to-be-inspected target is determined in combination with the pose data of the mobile robot when the image collection frame is shot;

[0039] Edge analysis and structure feature extraction are performed on each inspection target area to obtain structure feature information.

[0040] The inspection control strategy comprises:

[0041] The image position information and the structure feature information of the plurality of to-be-inspected targets are used to plan an inspection path.

[0042] Compared with the prior art, the application has the following beneficial effects:

[0043] The application realizes dynamic identification of the inspection target and adaptive image set construction under the condition of lacking target prior information by introducing an image collection control mechanism combining structure integrity level, edge extension direction and image entropy feature. Compared with the prior art of fixed interval collection or static template identification, the application can adjust the collection interval in real time according to the target structure change, avoid information redundancy or missing of key frames, and thus improve the accuracy and efficiency of model construction. BRIEF DESCRIPTION OF DRAWINGS

[0044] Other features, objects and advantages of the application will become more apparent after reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 An exemplary inspection scene schematic diagram of an embodiment of the application;

[0046] Figure 2 A flowchart of a power plant air cooling island mobile robot control method of an embodiment of the application;

[0047] Figure 3 A collection position prediction flowchart of an embodiment of the application;

[0048] Figure 4 A construction example schematic diagram of a structure growth tree of an embodiment of the application;

[0049] Figure 5 A node depth expansion schematic diagram of an embodiment of the application;

[0050] Figure 6 This is a schematic diagram illustrating the principle of updating the image set of the target to be inspected in an embodiment of this application. Detailed Implementation

[0051] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0052] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0053] This application applies to track-mounted mobile inspection systems with fixed path deployment characteristics and data acquisition locations limited by structural wiring or fixed pose control. The characteristics of the application scenarios include, but are not limited to:

[0054] A mobile robot with a track-moving structure has an image acquisition frequency limited by a set step distance;

[0055] The targets to be inspected are arranged along the track direction, with unequal spacing between them, and do not have preset positioning mark information;

[0056] The target to be inspected is a surface structural unit of equipment with boundary contour features. In this embodiment, it may include large structures such as the fan unit of the air-cooled island, the radiator fins and their supporting frame, and the target has directional extension in space.

[0057] It should be noted that this application does not presuppose the coordinates of the inspection target or the static division of the region. Instead, it is designed for inspection scenarios where the target boundary is unclear, the distribution is uneven, or the spatial extension structure is difficult to predefine. It uses edge structure features and entropy features extracted from image acquisition to make dynamic judgments and builds an inspection model without the need for prior information about the target.

[0058] It is understood that this application does not require any image acquisition frame to be permanently mapped to a specific inspection target area. Instead, based on the calculation results of the structural extension direction and structural integrity, the target area to which the current image frame belongs is determined in real time during the image acquisition process. After the determination is valid, the frame is included in the corresponding image set of the target to be inspected as the input basis for model construction.

[0059] Please see Figure 1 This figure is a schematic diagram of an exemplary inspection scenario provided in an embodiment of this application.

[0060] As Figure 1 shown, the present application is applicable to a typical inspection environment with track layout features. Figure 1 As shown in the middle, a mobile robot is installed on the track, the track is arranged along the equipment area, and a plurality of target to be inspected (such as target one, target two and target three, the number in the figure is only an example, and the present application does not limit the number of targets) are distributed on both sides of the track. Each target to be inspected can be a common equipment unit of the air-cooled island, such as a fan unit, a radiator fin assembly, a pipeline and a support frame, etc., which usually presents a panel, a sheet or a directionally extended structure, the distribution positions of the targets are different, and they do not have preset anchor points such as two-dimensional codes and coded marks.

[0061] Figure 1 As shown, the mobile robot performs image acquisition operation along the track. In the present application, the acquisition process does not depend on target coordinate pre-configuration or manual annotation on site, but performs acquisition position prediction through target structure extension direction and image structure integrity to complete the coverage of the target.

[0062] It should be noted that the present application does not take the static segmentation of target area or inspection path as a prerequisite, but faces the inspection environment where the target boundary is not clear, the space arrangement is irregular, and the target distribution is uneven. As can be understood by those skilled in the art, in such an inspection environment, due to the lack of uniform size, standard anchor points or recognizable structure boundary marks in actual deployment, and the installation spacing between targets is limited by the on-site space conditions or the arrangement sequence, the mobile robot cannot rely on the given coordinates or prior models for target recognition and path planning when performing the task for the first time.

[0063] It can be understood that the method of the present application is used for the mobile robot to model the inspection target in advance before performing the task for the first time, to dynamically determine the target area boundary through image structure features, and to attribute the image frames to the corresponding target image set in real time during the image acquisition process, so as to realize the archiving of the modeling data by target. The present application does not require any image acquisition frame to be permanently bound to a certain target number, but analyzes the current acquisition frame through edge structure extension judgment and image information to identify whether it belongs to an existing target area or is a starting image segment of a new target, so as to realize the automatic recognition and model construction of the target structure in the scene without target unique identification information.

[0064] Figure 1 The target schematic in the figure is only an illustrative example, and the number, type and distribution mode of the target can be configured according to the scene requirements in actual application, and the present application does not make specific limitations on the structure form, communication protocol or identification machine.

[0065] In one example, the present application provides a power plant air cooling island mobile robot control method. Next, in combination with the drawings, a power plant air cooling island mobile robot control method provided by the embodiment of the present application is introduced. As shown in Figure 2 The control method comprises the following S1-S5, and the specific steps are as follows:

[0066] S1: controlling the mobile robot to move along the track and acquire images on both sides of the track at preset intervals until at least part of the pattern of the target to be inspected appears in the images on both sides of the track;

[0067] In the embodiment, the preset interval can be determined according to the specific track stepping accuracy, device density and shooting field of view. In most cases, in order to avoid missing or misjudging, the preset interval needs to be as short as possible to cover most of the inspection targets.

[0068] Further, the image acquisition process can be realized by an industrial camera installed at the front end of the mobile robot, and images on both sides in the track direction are acquired. It should be noted that the image acquisition of the present application is static acquisition, that is, only when the mobile robot stops at a fixed position according to the preset interval, the corresponding images on both sides of the track are acquired.

[0069] S2: acquiring an image acquisition frame in which the at least part of the pattern appears, predicting the acquisition position of the next image acquisition frame according to the edge structure features of the image acquisition frame, and acquiring the next image acquisition frame;

[0070] As can be understood by those skilled in the art, the calculation method of the edge structure features, the determination threshold and the acquisition prediction function form can be flexibly configured according to the image processing algorithm capability and the acquisition frequency, as long as the extension direction of the target region can be inferred on the basis of meeting the recognition accuracy, and the present application is not limited further.

[0071] S3: comparing the entropy feature of the next image acquisition frame with the entropy feature of the current image acquisition frame to determine whether to add the next image acquisition frame to the preset target to be inspected image set;

[0072] In the embodiment, the entropy feature is acquired by a local weighted image entropy algorithm, that is, the image is divided into multiple small blocks, the gray entropy value and the edge direction standard deviation of each small block are calculated respectively, a structure weighted entropy graph is formed, and then the sum is obtained to obtain the whole frame entropy feature. The condition for determining whether to add the target to be inspected image set is that the absolute value of the difference between the entropy feature and the entropy feature of the current frame exceeds the first preset change threshold, or the entropy feature is greater than the second preset entropy threshold, or the local region weighted entropy value exceeds the entropy feature of the current frame.

[0073] S4: iteratively performing the steps of S2 to S3 until a plurality of consecutive image acquisition frames are not added to the target to be inspected image set, and terminating the iteration;

[0074] In the embodiment, the number of the plurality of image acquisition frames is not further limited, and the number greater than or equal to 3 can meet the requirement, that is, it is determined that the target structure region acquisition is completed.

[0075] S5: According to the set of target images to be inspected, image position information and structure feature information of the target to be inspected are extracted, and corresponding control instructions are generated according to the image position information and structure feature information of the plurality of targets to be inspected, to obtain an inspection control strategy.

[0076] In the embodiment, the inspection control strategy is a structured modeling result used for completing target recognition, shooting position matching and path control decision in the execution of a specific inspection task of the mobile robot. The input of the inspection control strategy can be instruction information of a certain inspection task, including but not limited to a specified area, a target type and a sequence requirement of inspection, or a target instance to be inspected. It can be understood that the inspection control strategy outputs corresponding information according to the input.

[0077] For example, the output of the inspection control strategy can be a positioning interval of the current task, and the mobile robot can determine whether to enter a shooting preparation state according to the positioning interval. The output of the inspection control strategy can also be a structure contour and a feature hash value of a certain target to be inspected, which is used for the mobile robot to perform a matching operation in continuous frames. The output of the inspection control strategy can also be a recommended shooting angle and a focal length parameter, that is, related inspection parameters.

[0078] It can be understood by those skilled in the art that the output content of the inspection control strategy is not subject to a specific field format or identification method, and only needs to have an associated information expression capability that can support inspection path control and image recognition execution. The application does not limit the output structure or task calling method. In the application, the inspection control strategy can also be understood as an inspection model, and the inspection model is a daily inspection configuration of the mobile robot. For ease of understanding, the concept of the model is used to express the inspection control strategy below.

[0079] In the existing track inspection actual application, in the case where the inspection target information is lacking, the debugging work of the robot depends on manual configuration one by one. This process is not only extremely complex, but also needs to consume a large amount of manpower and time. It can be understood that this way is time-consuming and laborious and is only suitable for relatively small-scale scenes, and is prone to omissions and misjudgments. In addition, the data recorded by manual configuration is mostly paper documents, which is difficult to manage and analyze, and cannot realize instant transmission and processing of data.

[0080] The application constructs a dynamic adaptive image acquisition strategy, and solves the image selection problem in an unpredictable scenario where the target structure extends. Specifically, when a local target structure appears in the image, the equidistant acquisition or image fixed sliding window strategy is not immediately started, but the edge line direction, structure closure and coverage rate and other indicators in the image are extracted to determine the direction in which the target may extend. Then, image acquisition is performed at the predicted acquisition position, and whether to continue acquisition or terminate is determined in combination with the dynamic change trend of the entropy feature. As can be easily understood, edge prediction provides macroscopic directional perception, and entropy feature completes microscopic determination, which not only improves the efficiency of the acquired image, but also avoids target omission or redundant acquisition caused by single feature fluctuation.

[0081] As can be appreciated by those skilled in the art, the method provided by the application mainly focuses on the modeling phase of the inspection model, aims to solve the problem of image acquisition and target recognition of the mobile robot in the unknown target space under the condition of lacking prior information, and does not include subsequent daily inspection planning and model updating. In other words, whether the inspection model is iteratively adjusted in actual operation after being constructed, or is called and reconstructed in different tasks, belongs to another scheduling logic, which is beyond the modeling scope concerned by the application.

[0082] Next, the related technical content of the power plant air cooling island mobile robot control system provided by the application is further described.

[0083] In one example, the position calibration method of the mobile robot provided by the embodiment of the application is used to solve the problem of unstable initial pose or invalid positioning reference of the robot in the track inspection scene, and specifically includes:

[0084] The start position and the terminal position on the track are provided with signal positions, wherein the signal positions include but are not limited to limit switches, physical stop blocks or magnetic sensing tags;

[0085] When the mobile robot is ready to start working, it first moves to the left for several meters, and if it cannot contact the signal positions of the start position or the terminal position of the track, it moves to the other side until it contacts the signal positions of the start position or the terminal position;

[0086] The work starts from the start position or the terminal position.

[0087] As can be understood, the position calibration method of the application is not limited to the construction phase of the inspection model, and can also be repeatedly called in subsequent daily inspection. For example, when the robot restarts due to power failure, abnormally stops or deviates during the journey, the signal position calibration mechanism can be used to quickly restore the positioning reference. In addition, as can be appreciated by those skilled in the art, the type of signal position, the detection method and the specific threshold setting of the calibration action can be adjusted according to the deployment scene, and the application does not limit the hardware type or specific motion control algorithm.

[0088] Referring to Figure 3 , Figure 3 The acquisition position prediction flowchart provided for the embodiment of the application is shown in Figure 3 The method shown in the figure can be applied to S2 described above, and the specific steps are as follows:

[0089] S2.1: Extract the extension direction of the edge structure feature and the transverse structure line;

[0090] Specifically, the target to be inspected is usually a power equipment structure unit with spatial extension, and its image manifestation is usually a rectangular frame, a panel contour, a control interface, and other patterns with continuous edge features. In the limited conditions of the application, since the specific detail information is unknown, it is not possible to determine the specific form of the target to be inspected by fixed template matching, and therefore further image acquisition is required.

[0091] In the embodiment, the edge structure feature is calculated by image gradient analysis and Canny edge detection algorithm. Specifically, all edge pixel points are clustered according to the gradient direction, the average angle and distribution density of the main direction distribution are calculated, and the extension direction of the edge structure in the current image is determined. At the same time, the position and trend of the transverse structure line are determined in combination with the edge aggregation degree. In the application, the transverse structure line refers to the parallel boundary, transverse decoration or element arrangement line formed on the target cross section.

[0092] S2.2: Calculate the structural integrity according to the closure, continuity and image coverage of the transverse structure line;

[0093] In the embodiment, the closure refers to whether there is a closed shape edge structure in the image, such as a rectangular frame and a frame contour. In the specific implementation, Hough transform is used to detect closed curves, and a closed boundary angle closure threshold is set as a determination condition. Continuity is used to evaluate the extension continuity of the transverse line in the image. If the transverse line is interrupted too many times or the length difference of the line segment is too large, it is determined that the structure is interrupted and does not have integrity. The image coverage represents the proportion of the target structure in the current image.

[0094] S2.3: Predict the acquisition position of the next image acquisition frame according to the extension direction and the structural integrity;

[0095] Specifically, the traditional equidistant image acquisition method cannot adapt to the uncertainty of the target structure extension direction and the uneven distribution of the structure integrity, which is prone to cause over-sampling or missing-sampling problems. Especially in the mobile robot track linear movement scene, the target structure often presents in an irregular manner along the track direction, and its visible boundary and structure density are not uniform. If a uniform sampling step is still used, it may cause the target to be terminated prematurely before it is fully sampled, or a large number of repeated frames of the structure are collected, which seriously affects the modeling efficiency and recognition value.

[0096] In the present embodiment, the prediction logic is based on a virtual structure growth tree. In the detailed description of the present application, the structure growth tree takes the current image frame as the root node and gradually deduces the subsequent potential acquisition positions in the structure extension direction. Each depth of the growth path corresponds to an acquisition candidate point.

[0097] It can be understood that the structure extension direction and the structure integrity relied on by the present application in predicting the next image acquisition frame position are not only used as auxiliary indicators or empirical parameters for judgment, but also jointly constitute the decision logic of dynamic acquisition control. Among them, the extension direction is used to establish the directional constraint of the structure growth trunk, and limit the geometric trajectory of the acquisition advancement, while the structure integrity takes the structure coverage state reflected in the image semantic level as the evaluation index, and quantifies the saturation degree of the current target information, thereby guiding the adaptive adjustment of the acquisition interval. In other words, the position prediction is not based on fixed trajectory advancement or sliding window step selection, but driven by the spatial structure clues of the target image, and a set of image acquisition paths strongly associated with the target structure growth process is constructed in real time.

[0098] In one example, according to the extension direction and the structure integrity, the specific steps of predicting the acquisition position of the next image acquisition frame are as follows:

[0099] S2.3.1: divide the structure integrity into a first level and a second level;

[0100] In the foregoing supplementary description, the calculation of the structure integrity is derived from the closure, continuity and image coverage rate of the transverse structure line. As can be understood by those skilled in the art, the closure, continuity and image coverage rate can be processed by existing technology, which will not be repeated here.

[0101] Further, the processed closure, continuity and image coverage rate are normalized and then weighted and summed to obtain the structure integrity.

[0102] It can be understood that the structure integrity in the present application is theoretically within a continuous interval range, and the specific interval size can be set by those skilled in the art through a large number of experiments. For the convenience of explanation, the interval is first positioned to the open-closed interval of zero to one.

[0103] Further, the first level and the second level are distinguished according to a pre-set division threshold, in the embodiment of the present application, the overall interval is the open-interval of zero to one, and 0.5 can be defined as the specific division threshold naturally, in other words, (0.5, 1] corresponds to the first level, (0, 0.5] corresponds to the second level, and by analogy, the division threshold can also be set according to actual conditions, and the division standard of the threshold mainly lies in the performance density of the target structure in the image.

[0104] As preferred, when the target is generally in a longitudinal stretching structure, in order to avoid the early structure fragments being misjudged as complete, the division threshold can be appropriately adjusted upward, and vice versa. It is obvious to those skilled in the art that the setting of the division threshold is not limited to a single fixed value, but can be flexibly adjusted according to the specific application scene, target type and collection device precision and other parameters, provided that its distinguishing ability is sufficient to drive the level switching of the collection strategy, and the balance between the integrity of the structure collection area and the compactness of the image set is ensured.

[0105] Further, the first level and the second level can be further divided into multiple sub-levels according to the specific conditions of the closedness, continuity and image coverage rate. It is easy to understand that the closedness, continuity and image coverage rate are not linearly synchronous, in other words, although the three indicators are relatively consistent in overall trend as the information of the target to be inspected improves, there are still some cases where the edge lines in the image are highly continuous and the image coverage range is large, but due to the unsealed boundary, the structure cannot be considered complete.

[0106] Obviously, the first level and the second level can generally indicate the structure of the current image frame, but further refinement is more conducive to position prediction. In the embodiment of the present application, when two indicators meet the first level but one indicator does not meet, a first level sub-level can be further divided, which can be considered as the interval corresponding to (0.5, 0.75] in the numerical case, and the second level can also be screened by analogy. The present application does not repeat here. It should be noted that different levels have different prediction steps.

[0107] Taking the structure growth tree as an example, it can be understood with reference to Figure 4 Figure 4 An example of constructing a structure growth tree is shown, Figure 4 It shows that the collection position of the current image collection frame is taken as the starting position, that is, the root node, and from the root node, multiple child nodes are formed by growing along the structure extension direction level by level, and each child node corresponds to a candidate image collection position.

[0108] Figure 4 ​Further shown is an example of step-by-step growth, in the embodiments of the present application, the levels are specifically divided into a first level and a second level, and each level further includes a sub-level division line, combined with Figure 4 It can be understood that the area with lower structural integrity is classified into the second level, which corresponds to a dense distribution of sub-nodes, a short interval between predicted acquisition points, and is used to enhance the structural information capture capability; the area with higher structural integrity is classified into the first level, the sub-nodes are sparse, and the acquisition step is larger, so as to improve the acquisition efficiency.

[0109] In one example, the construction process of the structure growth tree is as follows:

[0110] In the first aspect, the structure analysis result of the current image acquisition frame is obtained, and the structure analysis result includes an edge extension main direction, a structural integrity level, a transverse structure closure index, an edge density atlas, and image pose information. Based on the frame position of the current image acquisition frame as a root node, the structure growth tree is initialized.

[0111] It should be noted that in order to adapt to the possible spatial irregular shape of the target structure, each structure growth tree is allowed to grow in multiple directions by default when initialized, the main direction is set as the forced advancing direction according to the edge main direction, and the remaining directions are auxiliary branch exploration paths, which are used to identify structural deviation or boundary turning.

[0112] It can be understood that since most of the target to be inspected is directly opposite to the track and the structure is a normal cube, the structure growth tree is grown from the edge main direction by default in the structure growth tree constructed in the first image acquisition frame, and if the structure analysis result of the subsequent image acquisition frame shows a significant deviation, the structure growth tree can be pruned.

[0113] In the second aspect, the structure growth tree is initialized according to the pre-defined levels, and in the present application, the first level and the second level are adopted. The definition and distinction of the first level and the second level have been described in the foregoing content, and the present application will not be repeated here.

[0114] As can be appreciated by those skilled in the art, the structure growth tree can also be branched according to the levels, for example, when the structural integrity is determined to be the first level, the three indicators described above can be further subdivided to control the number of branches to be between 1 and 2, that is, the main direction is continuously advanced, and an optional secondary direction branch is generated for structure extension characteristics. When the structural integrity is in the second level, considering that the target structure may not have been fully presented, or the extension direction at the boundary is uncertain, more number of lateral branches are allowed to be generated to improve the structure capture coverage and avoid missing target area recognition due to unstable structure.

[0115] It should be noted that in actual application, the generation of each branch is not divided at a fixed angle, but based on the direction probability distribution function of the structure edge main direction, combined with the edge density distribution trend in the image to control the direction deviation. For example, if the main direction of the structure edge is vertically upward, but there is a feature area with high-density edge extending right and upward in the image, then under the second level condition, the exploration branch can be generated right and upward based on the main stem. The node spacing in each branch path is also dynamically adjusted according to the level, and the node spacing is compressed synchronously under the second level, while the node spacing of the first level main stem path can be expanded to the maximum allowed spacing. The specific deviation can be further limited according to the track direction and the shooting angle.

[0116] In the third aspect, when the number of branches is determined, the generation strategy of the prediction node can be determined according to the structure integrity level corresponding to each branch and the dominant feature type (which specifically includes closed dominance, coverage dominance and continuous dominance in the present application).

[0117] In the present embodiment, the next candidate image acquisition position is generated along the preset direction with the current node as the reference point, and the coordinate of the position is not pushed forward by an absolute step, but a prediction step value is obtained by mapping the current structure level into an interval adjustment model.

[0118] As can be appreciated by those skilled in the art, the interval adjustment model in the embodiments of the present application refers to a dynamic step adjustment model driven by the joint of the structure integrity level and the dominant feature type, which calculates the prediction acquisition position interval in real time according to the local state difference of the image structure, avoiding the target structure truncation or redundant shooting caused by the lag or discontinuity of the image structure change.

[0119] Specifically, the interval adjustment model takes the structure integrity index corresponding to the current node as the main input parameter, introduces an adjustment coefficient in combination with the dominant feature type, and performs weighted correction on the basic step value. The basic step value can be understood as the maximum acquisition interval by default at the current level.

[0120] Further, if the dominant feature type is closed dominance, the step compression degree is relatively conservative, that is, the adjustment coefficient is fine-tuned, if it is coverage dominance, considering that the target may still be in the boundary spreading state, the step is further compressed, and the adjustment coefficient is correspondingly compressed; and in the continuous dominance case, in order to strengthen the edge tracking effect, the adjustment coefficient can be further compressed to meet the demand. The prediction step value is obtained by multiplying the basic step by the adjustment coefficient and rounding, and is mapped to the spatial coordinate of the next prediction acquisition node. The specific adjustment coefficient compression method can be determined by a large number of experiments by those skilled in the art.

[0121] For example, the second level path expansion can be understood with reference to Figure 5 ​Figure 5 To expand the intention of the node depth of the embodiment, Figure 5 One branch path of the second level of the image acquisition frame is shown, and the path is divided into two sub-nodes according to the basic step length.

[0122] Figure 5 In the case of the closed dominant as the dominant feature type, the path is divided into three sub-nodes by compression adjustment in combination with continuity and image coverage.

[0123] Figure 5 Further, in the case of the continuous dominant as the dominant feature type, the path is divided into four sub-nodes by compression adjustment in combination with closedness and image coverage.

[0124] Figure 5 Further, in the case of the coverage dominant as the dominant feature type, the path is divided into five sub-nodes by compression adjustment in combination with closedness and continuity.

[0125] Those skilled in the art can understand that, Figure 5 It is only a simple example provided for easy understanding, and does not represent a real scene. In a real scene, there can also be a case where the number of dominant sub-nodes is the same, and there can also be a case where the number of sub-nodes is reduced. The specific situation can be controlled by those skilled in the art, but in principle, the number of sub-nodes corresponding to the first level must be less than the number of sub-nodes corresponding to the second level, in other words, the second interval can be smaller than the first interval by controlling the basic step length.

[0126] It can be understood that, under the premise that the total path length is unchanged, according to the level and the corresponding dominant feature type, the structure growing tree can adaptively generate corresponding sub-nodes, and the interval between the sub-nodes is the first interval and the second interval of the present application.

[0127] It is easy to understand that, Figure 4 The structure growing tree in the above is not complete, and each branch theoretically includes three predicted branches, that is, the predicted branches under different dominant feature type conditions, Figure 4 Only one branch is explained in the above, and the present application will not be further described here.

[0128] S2.3.2: When the structure integrity is at the first level, predict the acquisition position of the next image acquisition frame along the extension direction at a first interval;

[0129] S2.3.3: When the structure integrity is at the second level, predict the acquisition position of the next image acquisition frame along the extension direction at a second interval, wherein the second interval is smaller than the first interval;

[0130] In the embodiment, the interval is determined according to the level range in the tree and the branch path where the current image acquisition frame is located, and the interval is determined according to the characteristics of the current image acquisition frame, which is not described herein.

[0131] In one example, the entropy feature of the image acquisition frame is calculated as follows:

[0132] The image acquisition frame is subjected to edge enhancement processing to obtain a structure edge map.

[0133] The structure edge map is divided into a plurality of image regions, and the pixel gray level histogram entropy value and the edge direction distribution standard deviation in each image region are calculated respectively.

[0134] The pixel gray level histogram entropy value of each image region is multiplied by the corresponding edge direction distribution standard deviation to obtain a weighted entropy value.

[0135] The weighted entropy values of the image regions are summed to obtain the entropy feature of the image acquisition frame.

[0136] Specifically, the calculation method of the entropy feature of the image acquisition frame includes the following technical process: first, the original image acquisition frame is subjected to edge enhancement processing, and algorithms such as Sobel and Canny are used to strengthen the structure contour information to obtain a structure edge map with high boundary response. Then, the structure edge map is divided into a plurality of fixed-size image regions, and in each image region, the histogram entropy value corresponding to the pixel gray value distribution is calculated respectively to measure the dispersion degree of the pixel brightness in the region; at the same time, the direction distribution of the edge pixels in the region is counted, and the standard deviation is calculated to represent the complexity and direction diversity of the edge direction in the region. Further, the gray level histogram entropy value of each image region is multiplied by the edge direction standard deviation to form a weighted entropy value, and finally the weighted entropy values of all image regions are weighted and summed to obtain the comprehensive entropy feature of the whole image.

[0137] In one example, the judgment condition for whether to add the next image acquisition frame to the preset set of target images to be inspected includes the following two aspects:

[0138] First, the absolute value of the difference between the entropy features of the two frames is used for judgment.

[0139] In one case, when the absolute value of the difference between the entropy feature of the next image acquisition frame and the entropy feature of the current image acquisition frame is greater than a first preset change threshold, it can be determined that the next image acquisition frame is added to the preset set of target images to be inspected, wherein the first preset change threshold can be obtained by experimental data fitting or historical running data statistics.

[0140] It can be understood that when the difference between the entropy features of two image frames is large, it usually means that the display state of the target structure in the image has changed significantly, such as the extension of the structure boundary, the disappearance of the occlusion, or the entry of a new region into the collection field of view. Such changes directly lead to the improvement of image information density or the reconstruction of distribution pattern, indicating that the structure information contained in the current frame has non-redundancy and significant increment compared with the last frame, and therefore has the value of being included in the target image set to be inspected.

[0141] In another case, when the absolute value of the difference between the entropy feature of the next image acquisition frame and the entropy feature of the current image acquisition frame is less than or equal to a first preset change threshold, it can be judged that the next image acquisition frame is not added to the preset target image set to be inspected.

[0142] It can be understood that when the change amplitude of the entropy features of two image frames is in a stable interval, it often means that the structure region has not substantially expanded, and the current frame and the last frame have a high degree of coincidence in structure information. At this time, continuing to include the image set will cause redundancy, neither improving the completeness of the structure modeling, nor occupying the collection resources. Therefore, the application skips the image collection operation of the frame when the difference is in the stable interval, in order to ensure the compactness and discriminability of the image set.

[0143] It should be noted that the application judges the absolute value of the difference, because when the structure changes dramatically, the information content in the image usually shows a significant increase or decrease in entropy feature, not limited to a one-way increase. Therefore, using the absolute value of the difference to judge can more comprehensively capture the information mutation caused by the structure change. Taking a typical case as an example, if the target just enters the field of view, the structure appears, and the image entropy value rises rapidly. At this time, the difference is positive, which has a judgment basis. On the contrary, when the target is about to exit the field of view or the structure occlusion appears, the image entropy value decreases rapidly, and the difference is negative, but it also reflects the dramatic change of the structure.

[0144] In a second aspect, judging according to the entropy feature of the next image acquisition frame;

[0145] In one case, when the entropy feature of the next image acquisition frame is greater than a second preset entropy threshold, it can be judged that the next image acquisition frame is added to the preset target image set to be inspected, wherein the second preset entropy threshold can be obtained by experimental data fitting or historical running data statistics.

[0146] It can be understood that when the entropy feature of the next image acquisition frame is greater than the second preset entropy threshold, it generally indicates that the image content contained in the current frame has a high information complexity, the edge structure in the image is rich, the texture changes significantly, and it may correspond to the first appearance of the target structure, the key area of the structure entering the field of view, or the clear presentation of local features. Such frames often play a core role in modeling, such as structure recognition and boundary fitting, so even if the entropy value of the last frame does not change much, it also has independent acquisition value, and the image acquisition frame should be included in the target image set to be inspected.

[0147] In another case, when the entropy feature of the next image acquisition frame is less than or equal to the second preset entropy threshold, it can be judged that the next image acquisition frame is not added to the preset target image set to be inspected.

[0148] It can be understood that when the entropy feature of the next image acquisition frame is less than or equal to the second preset entropy threshold, it generally indicates that the information density in the current image is low, and the structure content may be missing, the image may be blurred, the edge may be weakened, or the target may not be effectively entered into the picture due to shooting offset. At this time, the acquisition frame itself lacks sufficient structure description ability, and it is difficult to provide incremental image information for target modeling. If it is forcibly included in the image set, the recognition effectiveness and computational efficiency of the overall image set will be reduced, so the application chooses to skip this frame to maintain the quality of the image set in this case.

[0149] Further, the first aspect and the second aspect can be combined to determine whether to add the next image acquisition frame to the preset target image set to be inspected.

[0150] When the first aspect and the second aspect are combined to determine whether to add the next image acquisition frame to the preset target image set to be inspected, if the result based on the first aspect is inconsistent with the result based on the second aspect, the result based on the first aspect is given priority, that is, the judgment priority of the first aspect is higher than that of the first aspect. The reason for such setting is that the first aspect is based on the difference between the two entropy features for judgment, which essentially reflects the dynamic change trend of the structure information between the continuous image frames, and such trend can more accurately represent the states of the target structure expansion, turning, unblocking, or significant morphological change. In contrast, the second aspect only relies on the entropy feature of the current frame, which can reflect the complexity of the image content, but cannot determine whether the structure information has changed substantially, and there is a possibility of false judgment of high entropy without increment. Therefore, when the two judgment results are inconsistent, the difference judgment result is preferred in the embodiments of the application, which can effectively avoid redundant acquisition caused by accidental high entropy of local image features, and ensure the structural evolution integrity and frame difference of the target image set to be inspected.

[0151] In addition, in addition to the two aspects described above, in the embodiments of the present application, whether the next image acquisition frame is added to the preset target image set to be inspected can also be determined in other manners or in combination with other manners.

[0152] For example, the judgment is made according to the local region entropy feature of the next image acquisition frame.

[0153] Optionally, when the weighted entropy value of at least one image region in the local region entropy feature of the next image acquisition frame is greater than the entropy feature of the current image acquisition frame, the next image acquisition frame can be added to the preset target image set to be inspected.

[0154] In one example, S3 obviously includes two cases before iteration, and different iteration processing methods are used for different cases, which can be referred to as Figure 6 For understanding, Figure 6 The principle diagram for updating the target image set to be inspected in the embodiments of the present application is as follows:

[0155] In the first case, when the next image acquisition frame is added to the preset target image set to be inspected, it indicates that the next image acquisition frame has reference value, and therefore the next image acquisition frame can be used as the current image acquisition frame for iteration.

[0156] In the second case, when the next image acquisition frame is not added to the preset target image set to be inspected, it indicates that the next image acquisition frame has no obvious change, and therefore the mobile robot is returned to the collection position of the current image acquisition frame, the structure growth tree corresponding to the current image acquisition frame is updated according to the entropy feature of the next image acquisition frame, and the update includes adjusting the node depth mapping relationship of the first interval and the second interval, and calculating the interval distance of the corresponding branch according to the entropy feature change rate.

[0157] Specifically, the nodes of each branch in the structure growth tree are generated based on the structural integrity and the extension direction in the generation process. When the next image acquisition frame is not added to the preset target image set to be inspected, it actually means that the construction process of the structure growth tree has a false judgment. By comparing the entropy difference and its change rate between the next image acquisition frame and the current frame, it is determined whether the structure expansion of the current branch has entered the boundary stable region or the collection direction deviates from the main structure line.

[0158] If the result of the judgment is that the structural complexity is insufficient or the direction deviation is too large, step compression operations are performed on all candidate nodes under the branch path that have not been executed for collection, that is, the depth and interval relationship of the candidate nodes is remapped so that the subsequent node prediction interval is reduced by a compression factor, thereby increasing the density and correcting the collection accuracy. If the result of the judgment is that the structural error caused by the direction error causes the structure to deviate from the main line, the current branch is marked as a weak structure branch, the generation priority of subsequent nodes is reduced, or the branch is directly pruned to terminate the invalid expansion of the path.

[0159] In one example, when the iteration is completed, it is indicated that a complete image set of a target to be inspected has been collected. At this time, the mobile robot is controlled to move along the track and acquire images at preset intervals on both sides of the track until at least part of the pattern of the next target to be inspected appears in the images on both sides of the track.

[0160] It can be understood that the present application can directly identify the target after one iteration is completed, or can identify all targets at the same time after all iterations are completed until the mobile robot reaches the terminal position. The present application is not limited herein.

[0161] In one example, the present application embodiment introduces specific steps for identifying all targets at the same time:

[0162] S5.1: Splicing adjacent image acquisition frames in the image set of the target to be inspected to generate a target image splicing result covering the complete target inspection area;

[0163] S5.2: Image segmentation is performed on the target image splicing result to identify the target inspection area in the image;

[0164] S5.3: For each target inspection area, the position index information thereof in the spliced image is extracted, and the image position information of the target to be inspected is determined in combination with the pose data of the mobile robot when the image acquisition frame is photographed;

[0165] S5.4: Edge analysis and structural feature extraction are performed on each target inspection area to obtain structural feature information;

[0166] In the present application, SIFT algorithm is used to calculate the target features, all targets with the same SIFT features and similar frame size are marked as a group, and the number of targets in the group exceeding the range is reminded for manual inspection; if the frame needs to be supplemented, the features and frame size of the new frame are marked as a group; if no identification is required, the marked photos are discarded, and after training, the marked photos are identified, if the data is found to be different, the difference points are marked out for manual inspection and correction until the identification data and the marked data are the same.

[0167] In one example, the embodiment of the present application provides a power plant air cooling island mobile robot control system, the mobile robot is configured with an image acquisition module, an instruction generation module, an iterative control module and a strategy generation module, wherein:

[0168] The image acquisition module is configured to control the mobile robot to move along the track and acquire images on both sides of the track at preset intervals until at least part of the pattern of the target to be inspected appears in the images on both sides of the track, and acquire a current image acquisition frame in which the at least part of the pattern appears;

[0169] The instruction generation module is configured to generate a control instruction according to the edge structure features of the current image acquisition frame, the control instruction being used to predict the acquisition position of a next image acquisition frame and control the mobile robot to acquire the next image acquisition frame;

[0170] The iterative control module is configured to compare the entropy features of the next image acquisition frame and the current image acquisition frame, determine whether to add the next image acquisition frame to a preset target to be inspected image set, replace the current image acquisition frame with the next image acquisition frame, and iteratively execute the image acquisition frame acquisition operation according to the control instruction until a plurality of continuous image acquisition frames are not added to the target to be inspected image set, and terminate the iteration;

[0171] The strategy generation module is configured to extract image position information and structure feature information of the target to be inspected according to the target to be inspected image set, generate corresponding control instructions according to the image position information and structure feature information of a plurality of targets to be inspected, and obtain an inspection control strategy.

[0172] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A power plant air-cooled island mobile robot control system, characterized by, The mobile robot is configured with an image acquisition module, an instruction generation module, an iterative control module, and a strategy generation module, wherein: The image acquisition module is configured to control the mobile robot to move along the track and acquire images on both sides of the track at preset intervals until at least part of the pattern of the target to be inspected appears in the images on both sides of the track, and to acquire a current image acquisition frame in which the at least part of the pattern appears; The instruction generation module is configured to generate a control instruction according to the edge structure features of the current image acquisition frame, the control instruction being used to predict the acquisition position of the next image acquisition frame and control the mobile robot to acquire the next image acquisition frame; The iterative control module is configured to compare the entropy feature of the next image acquisition frame with the entropy feature of the current image acquisition frame, determine whether to add the next image acquisition frame to a preset target-to-be-inspected image set, replace the current image acquisition frame with the next image acquisition frame, and iteratively perform the image acquisition frame acquisition operation according to the control instruction until N consecutive image acquisition frames are not added to the target-to-be-inspected image set, and terminate the iteration, wherein N is not less than 3; The strategy generation module is configured to extract image position information and structure feature information of the target to be inspected according to the target-to-be-inspected image set, generate corresponding control instructions according to the image position information and structure feature information of multiple targets to be inspected, and obtain an inspection control strategy.

2. The power plant air cooling island mobile robot control system of claim 1, wherein, The prediction of the acquisition position of the next image acquisition frame comprises: extracting the extension direction and the transverse structure line of the edge structure features; calculating the structure integrity according to the closure, continuity, and image coverage of the transverse structure line; predicting the acquisition position of the next image acquisition frame according to the extension direction and the structure integrity.

3. The mobile robot control system for an air-cooled island of a power plant according to claim 2, characterized by, The prediction of the acquisition position of the next image acquisition frame according to the extension direction and the structure integrity comprises: dividing the structure integrity into a first level and a second level; when the structure integrity is in the first level, predicting the acquisition position of the next image acquisition frame at a first interval along the extension direction; when the structure integrity is in the second level, predicting the acquisition position of the next image acquisition frame at a second interval along the extension direction, wherein the second interval is smaller than the first interval.

4. The power plant air cooling island mobile robot control system of claim 3, wherein, The first interval and the second interval are generated by a structure growth tree constructed based on the extension direction and the structure integrity, wherein: the structure growth tree takes the starting position of the structure integrity in the image as the root node, recursively grows along the extension direction to form multiple branches, wherein each branch node corresponds to a candidate image acquisition position, and the depth of each branch node is associated with the level of the structure integrity at the node; the first interval corresponds to the branch interval distance of the structure growth tree in which the structure integrity is in the first level, and the second interval corresponds to the branch interval distance of the structure growth tree in which the structure integrity is in the second level.

5. The power plant air cooling island mobile robot control system of claim 1, wherein, The calculation method of the entropy feature of the image acquisition frame is as follows: performing edge enhancement processing on the image acquisition frame to obtain a structure edge map; dividing the structure edge map into multiple image regions, and calculating the pixel gray histogram entropy value and the edge direction distribution standard deviation in each image region, respectively; The weighted entropy value of each image region is obtained by multiplying the pixel gray histogram entropy value of the image region with the corresponding edge direction distribution standard deviation; The entropy feature of the image acquisition frame is obtained by summing the weighted entropy values of the image regions.

6. The power plant air cooling island mobile robot control system of claim 5, wherein, The judgment condition for whether to add the next image acquisition frame to the preset set of target images to be inspected includes at least one of the following: The absolute value of the difference between the entropy feature of the next image acquisition frame and the entropy feature of the current image acquisition frame is greater than a first preset change threshold; The entropy feature of the next image acquisition frame is greater than a second preset entropy threshold; The weighted entropy value of at least one image region in the local region entropy feature of the next image acquisition frame is greater than the entropy feature of the current image acquisition frame.

7. The mobile robot control system for an air-cooled island of a power plant of claim 4, wherein, When the next image acquisition frame is added to the preset set of target images to be inspected, the next image acquisition frame is taken as the current image acquisition frame for iteration; When the next image acquisition frame is not added to the preset set of target images to be inspected, the mobile robot is returned to the acquisition position of the current image acquisition frame, and the structure growing tree corresponding to the current image acquisition frame is updated according to the entropy feature of the next image acquisition frame, wherein the updating includes adjusting the node depth mapping relationship of the first interval and the second interval, and calculating the interval distance of the corresponding branch according to the entropy feature change rate.

8. The mobile robot control system for an air-cooled island of a power plant of claim 1, wherein, When the iteration of the image acquisition frame acquisition operation ends, the following steps are further included: The mobile robot is controlled to move along the track and acquire images on both sides of the track at a preset interval until at least part of the pattern of the next target to be inspected appears in the images on both sides of the track.

9. The mobile robot control system for an air-cooled island of an electric power plant according to claim 8, characterized by, According to the set of target images to be inspected, the image position information and the structural feature information of the target to be inspected are extracted, including: The adjacent image acquisition frames in the set of target images to be inspected are spliced to generate a target image splicing result covering the entire target inspection area; The target image splicing result is subjected to image segmentation to identify the target inspection area in the image; For each target inspection area, the position index information thereof in the spliced image is extracted, and the image position information of the target to be inspected is determined in combination with the pose data of the mobile robot when the image acquisition frame is captured; Edge analysis and structural feature extraction are performed on each target inspection area to obtain the structural feature information.

10. The power plant air cooling island mobile robot control system of claim 9, wherein, The inspection control strategy includes: According to the image position information and the structural feature information of the plurality of targets to be inspected, an inspection path is planned.

Images