Automatic preset bit configuration method and system for remote intelligent patrol system

Abstract

The invention provides a preset bit automatic configuration method and system for a remote intelligent patrol system, and relates to the technical field of intelligent patrol preset bit configuration.The method comprises the steps that a patrol video stream or a video file obtained from the remote intelligent patrol system is decoded, and a target point location detection model and a multi-target tracking algorithm are used for detecting a target point location; identifying each inspection target point location in the video, matching a unique track for the inspection target point location, and generating a point location list containing track information; based on the point location list, constructing a matrix model for describing a coverage relationship between video frames and inspection point locations, solving an optimal frame set covering all inspection point locations through an optimization algorithm, screening out a final optimal preset bit image and corresponding equipment control parameters from the optimal frame set according to a preset rule, and generating a preset bit list. According to the method, automatic identification, trajectory tracking and intelligent optimization of the preset position of the patrol point can be realized, and the configuration efficiency and accuracy of the preset position are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent patrol preset position configuration technology, and in particular to an automatic preset position configuration method and system for remote intelligent patrol systems. Background Technology

[0002] Remote intelligent inspection systems are comprehensive systems integrating the Internet of Things (IoT), artificial intelligence (AI), sensing technology, and communication technology. They are widely used in critical areas such as industrial sites, substations, and power distribution rooms to achieve remote, automatic, and intelligent monitoring of equipment status and fault early warning. In this system, preset positions refer to a set of specific positional parameters (such as horizontal angle, vertical angle, zoom level, etc.) pre-set for cameras or other inspection equipment, used to quickly locate the target area to be inspected. Proper configuration of preset positions directly affects the system's inspection efficiency, coverage, and reliability.

[0003] Currently, the configuration of preset positions mainly relies on manual methods. Operators need to set camera parameters one by one based on experience and manually associate them with the inspection points. This method has the following obvious drawbacks: (1) Inefficiency: Manual configuration is time-consuming, usually requiring several days or even weeks; (2) Inconsistent standards: Relying on personal experience can easily lead to inconsistent preset position settings, affecting the standardization level of the system; (3) Difficult maintenance: After preset positions fail due to equipment movement, environmental changes, etc., it is difficult to reconfigure them in a timely and accurate manner; (4) Lack of adaptive capability: It is impossible to dynamically optimize the layout of preset positions according to the actual inspection scenario.

[0004] To address the shortcomings of manual configuration, existing technologies have proposed several automated or semi-automated solutions, such as: (1) SIFT-based method: A robot inspection method based on automatic visual scanning to generate inspection tasks. It uses the video stream of the robot scanning environment to send all images into the YOLOv4 target detection network for parsing and filtering, and uses SIFT matching to remove duplicate targets and automatically generate inspection points for complex field equipment. However, this method still relies on preset images and is suitable for correcting existing preset positions, but cannot achieve automatic configuration from scratch.

[0005] (2) Image exploration and neural network matching method: The edge computing gateway explores the field of view that the camera device can capture based on the preset position exploration strategy corresponding to the camera device and the preset initial field of view of the camera device. The neural network model outputs the result labels of the explored images, and the images selected by the model are matched with preset images. The camera parameters corresponding to the successfully matched images are used as the preset positions of the camera device. However, this method determines the preset positions by exploring the field of view and matching preset images. It also requires manually preset images as matching benchmarks and cannot completely get rid of the dependence on human prior knowledge.

[0006] In summary, existing technologies primarily employ a matching method, requiring pre-set images. This necessitates manual configuration of preset positions and acquisition of preset position images beforehand. While suitable for correction and adjustment after preset position failures, they cannot fully automate preset position configuration and fail to address the problems of low efficiency and inconsistent configuration standards. Therefore, current technologies still cannot achieve truly automated, standardized, and highly efficient preset position configuration. There is an urgent need for a technical method that can automatically identify and track patrol points from raw video data and intelligently generate the optimal preset position configuration scheme. Summary of the Invention

[0007] To address the technical problems of existing remote intelligent inspection systems, such as reliance on manual configuration, low efficiency, inconsistent standards, and inability to adaptively optimize preset locations, this invention provides a method and system for automatic preset location configuration in remote intelligent inspection systems, enabling automatic identification, trajectory tracking, and intelligent optimization of preset locations for inspection points.

[0008] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides, in a first aspect, a method for automatically configuring preset positions for a remote intelligent inspection system, comprising: S1. Decoding an inspection video stream or video file obtained from the remote intelligent inspection system, identifying each inspection target position in the video and matching it with a unique trajectory through a target position detection model and a multi-target tracking algorithm, and generating a position list containing trajectory information; S2. Based on the position list, constructing a matrix model describing the coverage relationship between video frames and inspection positions, solving for the optimal frame set covering all inspection positions through an optimization algorithm, and selecting the final optimal preset position image and corresponding device control parameters from the optimal frame set according to preset rules, and generating a preset position list.

[0009] The present invention provides a preferred solution in its first aspect, wherein step S1 specifically includes: S11. Decoding the inspection video stream or video file obtained from the remote intelligent inspection system into video frames in the form of multiple consecutive images; S12. Constructing a target point detection model based on a deep neural network model, and training and fine-tuning the target detection model using the target point image set of the remote intelligent inspection system; S13. Using the fine-tuned target point detection model to identify points in each video frame, obtaining the point information of all inspection target points in each video frame; S14. For all video frames, based on the obtained point information, performing trajectory matching using a multi-target tracking algorithm to obtain the trajectory information of all inspection target points; S15. Filtering the trajectory information to obtain video frame-point information with trajectory information, which serves as the point list. This preferred solution can achieve accurate identification of inspection targets in specific scenarios, improve detection accuracy and generalization ability, and ensure the integrity and reliability of the trajectory of each target in the point list.

[0010] Further preferably, in step S12, the target point detection model is established based on the YOLOv8 architecture; in step S14, the multi-target tracking algorithm adopts the ByteTrack multi-target tracking algorithm. This preferred scheme can achieve high-precision identification and stable trajectory tracking of multiple types of patrol targets in complex scenarios.

[0011] Further preferably, the point information includes: point type, point frame coordinates, and confidence level; step S14 specifically includes: S141. Based on whether the confidence level in the point information of each video frame reaches a preset first confidence threshold, classify the point frame of the current video frame into high-confidence detection boxes and low-confidence detection boxes, and create trajectory information for the first-time appearing point, assigning a unique trajectory ID; S142. Based on the existing trajectory information, predict the trajectory of each point in the current frame, and obtain the current state of each predicted trajectory; S143. According to a preset priority... The process proceeds sequentially, performing multi-stage matching to associate and match different categories of detection boxes with predicted trajectories in different states; S144. Based on the matching results, assign or continue the corresponding trajectory ID to the successfully matched points, and update the status and information of the corresponding trajectories; Step S15 specifically includes: S151. Processing the trajectory information obtained in the previous step, unifying the point type within a single trajectory, and filtering out abnormal data in the trajectory based on the area size of the point bounding box, finally outputting structured video frame-point information containing trajectory ID, point bounding box coordinates, and confidence level.

[0012] Further preferably, in step S142, the trajectory state is divided into an active state and an inactive state; step S143 specifically includes: S143a. Matching the high-confidence detection box of the current frame in step S141 with the trajectory predicted in step S142 that is in the active state. When the match is successful, the trajectory state is set to active, and the corresponding detection box state is set to tracked; when the match fails, the active but unmatched trajectory and the unmatched detection box are marked respectively; S143b. Matching the inactive trajectory predicted in step S142 with the unmatched detection box in step S143a. When the match is successful, the trajectory state is set to active, and the corresponding detection box state is set to tracked; when the match fails, further... Step 1: Determine whether the confidence level of the detection box reaches a preset second confidence threshold. If it does, add a new trajectory, assign a trajectory ID, and set the detection box status to "tracked." If not, keep the trajectory status inactive and the detection box status unmatched. Step 2: Match the low-confidence detection boxes in the current frame from step 141 with the active but unmatched trajectories from step 143a. If the match is successful, set the trajectory status to "activated" and the corresponding detection box status to "tracked." If the match fails, keep the trajectory status active and the detection box status untracked. Step 3: Keep the unmatched trajectories remaining inactive from step 143a inactive in the current frame. This preferred solution uses a multi-stage matching strategy to prioritize associating high-confidence detection boxes with active trajectories, ensuring continuous tracking of stable targets. Secondly, it uses low-confidence detection boxes to supplement potentially missed targets, improving the recall rate for occluded or blurred targets.

[0013] More preferably, in step S151, the step of unifying the point type within a single trajectory includes: obtaining the trajectory information of each point, determining whether the same trajectory contains different point types, and if there are multiple point types in a trajectory, selecting the point type with the highest average confidence as the point type of the entire trajectory.

[0014] The present invention provides a preferred solution in the first aspect, wherein step S2 specifically includes: S21. Based on the point list, construct a binary matrix model as a first matrix, with video frames as rows and patrol point IDs as columns, to describe the coverage relationship between video frames and patrol points, wherein the element values ​​indicate whether the corresponding frame contains the corresponding point; perform preprocessing on the first matrix including row deduplication and zero-row removal to obtain a second matrix; S22. With minimizing the number of preset points as the objective and covering all patrol points as the constraint, model the row selection problem of the second matrix as a 0-1 integer programming problem to form an optimization algorithm, and use a CBC solver to solve it to obtain the optimal row vector combination, which is the optimal frame set covering all patrol points; S23. For each row in the optimal row vector combination, find all frames with the same point coverage pattern in the original video frames, calculate the minimum confidence value of all covered points in each frame, select the frame with the largest minimum value as the final optimal preset point image, and record its timestamp and corresponding device control parameters. This preferred solution can further ensure that the remote intelligent inspection system achieves optimal configuration with limited resources.

[0015] In a second aspect, this invention provides an automatic preset position configuration system for a remote intelligent patrol system, used to implement the above-mentioned method, comprising: S1. a point tracking module, used to decode the patrol video stream or video file acquired from the remote intelligent patrol system, identify each patrol target point in the video and match it with a unique trajectory through a target point detection model and a multi-target tracking algorithm, generating a point list containing trajectory information; S2. a preset position optimization module, used to construct a matrix model describing the coverage relationship between video frames and patrol points based on the point list, solve for the optimal frame set covering all patrol points through an optimization algorithm, and select the final optimal preset position image and corresponding device control parameters from the optimal frame set according to preset rules, generating a preset position list. This preferred solution can enhance the visual integrity and recognition robustness of selected frames in complex environments while ensuring full coverage.

[0016] The present invention provides a preferred embodiment in a second aspect, wherein the point tracking module comprises: a decoding unit, configured to decode the inspection video stream or video file obtained from the remote intelligent inspection system into video frames in the form of multiple consecutive images; a modeling unit, configured to construct a target point detection model based on a deep neural network model, and to train and fine-tune the target detection model using the target point image set of the remote intelligent inspection system; a detection unit, configured to perform point identification on each video frame using the fine-tuned target point detection model, and obtain point information of all inspection target points in each video frame; a tracking unit, configured to perform trajectory matching on all video frames according to the obtained point information using a multi-target tracking algorithm, to obtain trajectory information of all inspection target points, and obtain video frame-point information with trajectory information after filtering, as the point list; and a filtering unit, configured to filter the trajectory information to obtain video frame-point information with trajectory information, as the point list. The preset position optimization module includes: a matrix construction unit, used to construct a binary matrix model, a first matrix, based on the point list, with video frames as rows and patrol point IDs as columns, to describe the coverage relationship between video frames and patrol points, where the element values ​​indicate whether the corresponding frame contains the corresponding point; the first matrix is ​​preprocessed to remove duplicate rows and remove rows with all zeros to obtain a second matrix; an optimization solution unit, used to model the row selection problem of the second matrix as a 0-1 integer programming problem with the objective of minimizing the number of preset positions and the constraint of covering all patrol points, to form an optimization algorithm, and to solve it using a CBC solver to obtain the optimal row vector combination, which is the optimal frame set covering all patrol points; and a preset position selection unit, used to find all frames with the same point coverage pattern in the original video frames for each row in the optimal row vector combination, calculate the minimum confidence value of all covered points in each frame, select the frame with the largest minimum value as the final optimal preset position image, and record its timestamp and corresponding device control parameters.

[0017] Compared with the prior art, the present invention has the following advantages: This invention analyzes decoded video frames frame by frame using a target point detection model to extract patrol target points. It then combines this with a multi-target tracking algorithm to achieve trajectory association and unique identifier matching, ensuring the continuous and stable trajectory of each point. Simultaneously, it generates a list of points containing trajectory information for subsequent preset position optimization. By constructing a matrix model describing the coverage relationship between video frames and patrol points, and using an optimization algorithm to solve for the optimal preset position image, it achieves automatic identification, trajectory tracking, and intelligent preset position selection for patrol points. This process requires no manual image pre-setting or prior knowledge intervention, relying solely on the algorithm's autonomous identification and tracking. It automates the conversion from raw video to structured point information, effectively improving the comprehensiveness and objectivity of point discovery and configuration. This solves the technical problems of existing remote intelligent patrol systems, such as reliance on manual configuration, low efficiency, inconsistent standards, and inability to adaptively optimize preset positions. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an automatic configuration method for preset positions in a remote intelligent patrol system according to a specific embodiment of the present invention; Figure 2 This is a block diagram of a preset position automatic configuration system for a remote intelligent inspection system provided in a specific embodiment of the present invention; Figure 3 This is a flowchart illustrating the method for automatically configuring preset positions in a remote intelligent patrol system according to a specific embodiment of the present invention, in which video frame-point information is obtained through point tracking. Figure 4 This is an architecture diagram of an automatic configuration system for a remote intelligent inspection system provided in a specific embodiment of the present invention. Figure 5 This is a flowchart of the internal algorithm of the preset position optimization module in the preset position automatic configuration method for a remote intelligent inspection system provided in a specific embodiment of the present invention. Figure 6 This is a schematic diagram of the output result of the point tracking module when the preset position automatic configuration system for a remote intelligent inspection system provided by a specific embodiment of the present invention is applied to the point configuration of the control cabinet panel in the control room of a substation. Figure 7The preset position automatic configuration system for a remote intelligent inspection system provided in a specific embodiment of the present invention is applied to the position configuration of the control cabinet panel in the control room of a substation, and the point matrix diagram after deduplication by the preset position optimization module is shown. Figure 8 The optimal preset position number diagram output by the preset position optimization module when the preset position automatic configuration system for a remote intelligent inspection system provided in a specific embodiment of the present invention is applied to the position configuration of the control cabinet panel in the control room of a substation. Figure 9 The diagram shows the preset position configuration result when the preset position automatic configuration system for a remote intelligent inspection system provided by a specific embodiment of the present invention is applied to the position configuration of the control cabinet panel in the control room of a substation. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please refer to Figure 1 In one optional implementation, a method for automatically configuring preset positions for a remote intelligent inspection system is provided, mainly through steps S1 (position tracking) and S2 (preset position optimization): Point Tracking: S1. Decode the patrol video stream or video file obtained from the remote intelligent patrol system. Using a target point detection model and a multi-target tracking algorithm, identify each patrol target point in the video and match it with a unique trajectory, generating a point list containing trajectory information. Preset Position Optimization: S2. Based on the point list, construct a matrix model describing the coverage relationship between video frames and patrol points. Solve for the optimal frame set covering all patrol points using an optimization algorithm. Then, according to preset rules, select the final optimal preset position image and corresponding equipment control parameters from the optimal frame set, generating a preset position list.

[0022] Please refer to Figure 2Corresponding to the above embodiments, an automatic preset position configuration system for a remote intelligent inspection system is provided to implement the above method. It mainly consists of the following modules: a point tracking module 1, used to decode the inspection video stream or video file obtained from the remote intelligent inspection system, identify each inspection target point in the video and match it with a unique trajectory using a target point detection model and a multi-target tracking algorithm, generating a point list containing trajectory information; and a preset position optimization module 2, used to construct a matrix model describing the coverage relationship between video frames and inspection points based on the point list, solve for the optimal frame set covering all inspection points using an optimization algorithm, and select the final optimal preset position image and corresponding device control parameters from the optimal frame set according to preset rules, generating a preset position list.

[0023] This embodiment demonstrates that a target point detection model is used to analyze decoded video frames frame by frame to extract patrol target points. A multi-target tracking algorithm is combined to achieve trajectory association and unique identifier matching, ensuring the continuous and stable trajectory of each point. Simultaneously, a list of points containing trajectory information is generated for subsequent preset position optimization. By constructing a matrix model describing the coverage relationship between video frames and patrol points, and combining it with an optimization algorithm to obtain the optimal preset position image, automatic identification, trajectory tracking, and intelligent preset position selection for patrol points are achieved. This process requires no manual image pre-setting or prior knowledge intervention, relying solely on the algorithm for autonomous identification and tracking. It achieves automated conversion from raw video to structured point information, effectively improving the comprehensiveness and objectivity of point discovery and configuration. It solves the technical problems of existing remote intelligent patrol systems, such as reliance on manual configuration, low efficiency, inconsistent standards, and inability to adaptively optimize preset positions.

[0024] Based on the above implementation methods, more preferred and specific implementation methods will be given below: For step S1, the inspection video stream or video file obtained from the remote intelligent inspection system is decoded. Using a target point detection model and a multi-target tracking algorithm, each inspection target point in the video is identified and matched with a unique trajectory, generating a list of points containing trajectory information. This is specifically achieved through the following sub-steps (S11 to S15): Decoding: S11. Decode the inspection video stream or video file obtained from the remote intelligent inspection system into video frames in the form of multiple consecutive images.

[0025] Modeling: S12. Construct a target location detection model based on a deep neural network model, and train and fine-tune the target detection model using the target location image set of the remote intelligent patrol system. In a preferred embodiment, the target location detection model is built based on the YOLOv8 architecture (model). This step uses the target location image set of the patrol system to train a fine-tuned model for detecting points.

[0026] Detection: S13. Using the fine-tuned target point detection model, point identification is performed on each video frame to obtain the point information of all patrol target points in each video frame. This step uses the fine-tuned model as a detector to identify the target points in each frame and output the point information related to each frame. In a preferred embodiment, the point information includes: point type, point bounding box coordinates, and confidence level.

[0027] Tracking: S14. For all video frames, based on the obtained point information, trajectory matching is performed using a multi-target tracking algorithm to obtain the trajectory information of all patrol target points. The multi-target tracking algorithm adopted is the ByteTrack multi-target tracking algorithm. This step designs a tracker based on the ByteTrack algorithm and filters the point trajectory information. This preferred scheme establishes a target point detection model based on the YOLOv8 architecture and combines it with the ByteTrack multi-target tracking algorithm, which can achieve high-precision identification and stable trajectory tracking of multiple types of patrol targets in complex scenes.

[0028] Filtering: S15. After filtering the trajectory information, obtain video frame-point information (in tabular form) with trajectory information, as the point list. Step S15 is specifically implemented through the following sub-steps: S151. Process each trajectory information obtained in the previous step, unify the point type within a single trajectory, and filter out abnormal data in the trajectory based on the area size of the point bounding box, finally outputting structured video frame-point information (in tabular form) containing trajectory ID, point bounding box coordinates, and confidence level. In step S151, the step of unifying the point type within a single trajectory is specifically as follows: obtain the trajectory information of each point, determine whether the same trajectory contains different point types, and if a trajectory contains multiple point types, select the point type with the highest average confidence level as the point type of the entire trajectory. More specifically, the trajectory information for each point is obtained, and it is determined whether different point types are contained within the same trajectory. If multiple point types exist within a trajectory, the point type with the highest average confidence score is selected as the point type for the entire trajectory. Within the same trajectory, point information with a bounding box area less than 80% of the maximum point area is deleted. Finally, the video frame-point information, including trajectory ID, bounding box coordinates, and confidence score, is retained. By selecting the point type with the highest average confidence score from the point types within the same trajectory as the final identifier for that trajectory, the risk of type drift caused by misjudgment in a single frame is effectively reduced, enhancing the consistency of trajectory semantics.

[0029] This preferred embodiment uses a target location detection model built on a deep neural network model for fine-tuning, enabling accurate identification of patrol targets in specific scenarios and improving detection accuracy and generalization ability. Furthermore, by filtering trajectory information to eliminate abnormal trajectories introduced by occlusion, false detection, or noise, the integrity and reliability of each target trajectory in the location list are ensured, thus providing high-quality input for subsequent preset location selection.

[0030] Step S14 is specifically implemented through the following sub-steps (S141 to S144): S141. Based on whether the confidence level of the point information in each video frame reaches a preset first confidence threshold, the point map of the current video frame is classified into high-confidence detection boxes and low-confidence detection boxes, and trajectory information is created for the first occurrence of the point, and a unique trajectory ID is assigned. This step obtains the image point information of each frame obtained in step S13 according to the time sequence. If the confidence level of the point is greater than the preset first confidence threshold, it is classified as a high-confidence detection box; if the confidence level of the point is less than the first confidence threshold, it is classified as a low-confidence detection box; if the point is the first occurrence, a trajectory information is created and a trajectory ID is assigned.

[0031] S142. Based on the existing trajectory information, predict the trajectory of each point in the current frame and obtain the current state of each predicted trajectory. Further, this step predicts the coordinate trajectory of the point in the current frame image based on the trajectory information of the point, and distinguishes between active and inactive trajectories, that is, the trajectory state is divided into active state and inactive state.

[0032] S143. Perform multi-stage matching according to the preset priority order, and sequentially associate and match different types of detection boxes with predicted trajectories of different states.

[0033] Please refer to Figure 3 This process mainly involves three stages of matching, specifically implemented through the following sub-steps (S143a to S143d): Phase 1 matching: S143a. Match the high-confidence detection box in the current frame in step S141 with the trajectory predicted in step S142 that is in an active state. When the match is successful, set the trajectory state to active and the corresponding detection box state to tracked. When the match fails, mark the active but unmatched trajectory and the unmatched (untracked) detection box respectively. Second-stage matching: S143b. Match the inactive trajectory predicted in step S142 with the unmatched (untracked) detection boxes from step S143a. If the match is successful, set the trajectory status to active and the corresponding detection box status to tracked. If the match fails, further determine whether the confidence level of the detection box reaches a preset second confidence threshold. If it does, add a new trajectory, assign a trajectory ID, and set the detection box status to tracked. If it does not reach the threshold, keep the trajectory status inactive and the detection box status unmatched (untracked). S143c. Match the low-confidence detection box of the current frame in step S141 with the active but unmatched trajectory in step S143a. If the match is successful, set the trajectory state to active and the corresponding detection box state to tracked. If the match fails, keep the trajectory state active and the detection box state untracked (unmatched). S143d. The unmatched trajectories that were left over from step S143a and were inactive are kept inactive in the current frame.

[0034] S144. Based on the matching results, assign or continue the corresponding trajectory ID to the successfully matched points, and update the status and information of the corresponding trajectories. That is, after trajectory matching in steps S143a to S143d, the trajectory information of each point is obtained for each frame of the image. The preferred embodiment described above employs a multi-stage matching strategy, prioritizing the association of high-confidence detection boxes with active trajectories to ensure continuous tracking of stable targets. Secondly, it utilizes low-confidence detection boxes to supplement potentially missed targets, improving recall for occluded or blurred targets. Simultaneously, it effectively suppresses noisy trajectories introduced by false detections, ensuring the integrity and spatiotemporal continuity of the output trajectory sequence. Furthermore, by introducing an area filtering mechanism, it further eliminates abnormal points caused by environmental interference or detection errors, thereby improving the accuracy of the trajectory data. The resulting structured information not only includes the position and confidence of each target in each frame but also maintains consistency and logical coherence across frames, providing reliable input for subsequent advanced applications such as behavior analysis and path prediction.

[0035] Correspondingly, this embodiment provides a preset position automatic configuration system for a remote intelligent patrol system, wherein the position tracking module includes: a decoding unit, used to decode the patrol video stream or video file obtained from the remote intelligent patrol system into video frames in the form of multiple consecutive images; a modeling unit, used to construct a target position detection model based on a deep neural network model, and to train and fine-tune the target detection model using the target position image set of the remote intelligent patrol system; a detection unit, used to perform position identification on each video frame using the fine-tuned target position detection model, and obtain position information of all patrol target positions in each video frame; a tracking unit, used to perform trajectory matching on all video frames according to the obtained position information using a multi-target tracking algorithm, to obtain trajectory information of all patrol target positions, and obtain video frame-position information with trajectory information (in tabular form) after filtering, as the position list; and a filtering unit, used to filter the trajectory information to obtain video frame-position information with trajectory information (in tabular form), as the position list. The preset position optimization module includes: a matrix construction unit, used to construct a binary matrix model, a first matrix, based on the point list, with video frames as rows and patrol point IDs as columns, to describe the coverage relationship between video frames and patrol points, where the element values ​​indicate whether the corresponding frame contains the corresponding point; the first matrix is ​​preprocessed to remove duplicate rows and remove rows with all zeros to obtain a second matrix; an optimization solution unit, used to model the row selection problem of the second matrix as a 0-1 integer programming problem with the objective of minimizing the number of preset positions and the constraint of covering all patrol points, to form an optimization algorithm, and to solve it using a CBC solver to obtain the optimal row vector combination, which is the optimal set of frames covering all patrol points; and a preset position selection unit, used to find all frames with the same point coverage pattern in the original video frames for each row in the optimal row vector combination, calculate the minimum confidence value of all covered points in each frame, select the frame with the largest minimum value as the final optimal preset position image, and record its timestamp and corresponding device control parameters.

[0036] For specific implementation, please refer to Figure 4 The point tracking module uses detectors (modeling units and detection units) to identify points and coordinate bounding boxes in each frame of the image. It then uses a tracker (tracking unit) to associate points across frames, assigning a unique ID to each point and ultimately generating a point list. The preset point optimization module preprocesses the point data, generating a preset point list through an optimizer (matrix construction unit and optimization solution unit) and a selector (preset point selection unit).

[0037] In a preferred embodiment, step S2 is specifically implemented through the following steps (S21 to S23): Matrix Construction: S21. Based on the point list, with video frames as rows and patrol point IDs as columns, construct a binary matrix model to describe the coverage relationship between video frames and patrol points as the first matrix, where the element value indicates whether the corresponding frame contains the corresponding point; perform preprocessing on the first matrix including row deduplication and all-zero row removal to obtain the second matrix.

[0038] Specifically, a matrix is ​​constructed using video frames as rows and inspection point IDs as columns. ,in Indicates the number of video frames. This indicates the number of inspection points. Merging matrices For rows with identical element values, remove rows where all elements are 0, and generate a new matrix. , Representation matrix The number of distinct row vectors in the array. Representation matrix The first in row vectors Indicates the first The first row of the non-repeating vector The values ​​at each inspection point. .

[0039] Optimization solution: S22. With the goal of minimizing the number of preset positions and the constraint of covering all inspection points, the row selection problem of the second matrix is ​​modeled as a 0-1 integer programming problem to form an optimization algorithm, and the CBC solver is used to solve it to obtain the optimal row vector combination, which is the optimal frame set covering all inspection points.

[0040] Specifically, based on the matrix obtained in step S21 The optimal row vector combination is calculated using the 0-1 integer programming method.

[0041] The 0-1 integer programming problem is expressed as: st Where S is the number of rows in the second matrix, and N is the total number of inspection points. For the elements of the second matrix, As decision variables, This indicates whether to select row i.

[0042] The CBC (COIN-OR Branch-and Cut) solver was used to solve the above optimization problem, and the optimal solution was obtained. . It is a binary column vector. ,element , making Each column in The inner product of all of them is greater than zero, that is, through The selected row vector combination covers all inspection points.

[0043] Preset position selection: S23. For each row in the optimal row vector combination, find all frames with the same point coverage pattern in the original video frame, calculate the minimum confidence value of all covered points in each frame, select the frame with the largest minimum value as the final optimal preset position image, and record its timestamp and corresponding device control parameters.

[0044] Specifically, based on the optimal solution obtained in step S22 Construct a selector to select the optimal preset bit image and parameter combination.

[0045] Optimal solution The non-zero values ​​correspond to the matrix Use the row index to obtain a series of vectors. Then retrieve the row vectors in the matrix The row index. Assume a vector. In the matrix Retrieve the indices of rows with the same row vector. , This indicates the number of identical row vectors. Each index corresponds to the confidence level of a set of identification points. , , This indicates the number of identifiable points corresponding to this set of vectors. Indicates the subscript is The recognition confidence score of the first point corresponding to the row vector. The minimum confidence score of the point position corresponding to each index is calculated, and the index corresponding to the maximum value among all minimums is selected as the preset position. For related procedures, please refer to [link / reference]. Figure 5 .

[0046] Using the above method, vector groups Convert into a set of preset indexes The timestamp corresponding to the index can be further converted into a set of preset position images and a set of control parameters for the gimbal, camera, drone, and robot.

[0047] This preferred solution constructs a binary matrix model to describe the coverage relationship between video frames and inspection points, and performs deduplication and constraint optimization based on this model. This transforms the selection of a set of frames covering all inspection points into a 0-1 integer programming problem, significantly improving the compactness and representativeness of the preset position selection. By introducing the minimum confidence value as the selection criterion, the reliability and observability of the selected frames in the actual monitoring environment are further enhanced, ensuring that the remote intelligent inspection system achieves optimal configuration under limited resources.

[0048] This preferred implementation constructs a binary matrix model to describe the coverage relationship between video frames and patrol points, and performs deduplication and constraint optimization based on this model. This transforms the selection of a set of frames covering all patrol points into a 0-1 integer programming problem, significantly improving the compactness and representativeness of the preset position selection. By introducing the minimum confidence value as the selection criterion, the reliability and observability of the selected frames in the actual monitoring environment are further enhanced, ensuring that the remote intelligent inspection system achieves optimal configuration under limited resources.

[0049] The following example illustrates the automatic configuration method and system for preset positions in a remote intelligent inspection system, using the control panel of a substation control room as an example. Figure 6 This is the output of the point tracking module. Figure 6 In the diagram, (a), (b), and (c) represent (a) the original video frame, the point detected by the detector, and the point ID assigned by the tracker, respectively. The point tracking module parses the video stream or video file, which contains 7704 frames, and extracts the video frame images, such as... Figure 6 As shown in (a), the detector model detects target points in each frame of the image. The point category number is the category number output by the detection model, such as... Figure 6 As shown in (b), the tracker then assigns an ID to each location, such as... Figure 6 As shown in (c), the point positions of the two consecutive frames are kept consistent through ID, as shown in Table 1.

[0050] Table 1 shows the point data for the two frames. The preset position optimization module converts the position data into a matrix through preprocessing. The dimension is [7704, 486], indicating that there are 7704 image frames and 486 points, and the matrix is ​​obtained after removing duplicates using row vectors. The dimension is [878, 486], indicating that there are 878 unique row vectors, such as... Figure 7 As shown, the deduplicated row vectors do not follow the exact numbering order, and the ID numbers of the points in the matrix do not appear in perfect order.

[0051] Solving matrices using an optimizer Given a constrained optimization problem, a set of optimal solutions is obtained. ,like Figure 8 As shown, the product of the x and y coordinates corresponds to the matrix. The row vector indices. There are a total of 123 non-zero values, corresponding to the matrix. 123 row vectors.

[0052] Through matrix The 123 row vectors in the matrix Search and select 123 preset positions, some of which are as follows: Figure 9 As shown in the diagram. Here, 00-14-6 represents the 6th frame at 0 minutes and 14 seconds; other representations follow the same logic. The selected preset positions can cover all points, and the most suitable combination of preset positions can be chosen by modifying the selector. The coordinates of the points are configured simultaneously with the preset positions, simplifying preset and point configuration and improving efficiency.

[0053] Figure 9 Preset bit configuration results: Figure 9 (a) is 00-14-6. Figure 9 (b) is 00-55-10. Figure 9 (c) is 1-32-7. Figure 9 (d) is 02-27-28. Figure 9 (e) is 03-09-6. Figure 9 It is (f)02-33-29.

[0054] The point tracking module processed 7704 frames of images from a 4-minute and 17-second video in a total time of 357.7 seconds, while the preset position optimization module took only 0.01 seconds, meeting the automatic preset position configuration requirements of the remote intelligent inspection system. Compared to the current situation where manual configuration of preset positions and inspection points takes several days or even tens of days, the above method completes the preset position optimization configuration and the marking of inspection points on preset positions in minutes, greatly improving efficiency.

[0055] This invention enables automatic configuration of preset positions in a remote intelligent inspection system through a point tracking module composed of detectors and trackers and a preset position optimization module composed of optimizers and selectors. This allows for manual replacement and standardized operations, reducing the workload from 15-30 days to less than 1 hour, thus greatly improving work efficiency.

[0056] This invention converts video frame-position IDs into a non-repeating row vector matrix, applies a 0-1 integer programming optimization design optimizer, designs a selector based on confidence, and finally selects the optimal combination of preset positions, thus achieving the optimization and standardized configuration of preset positions. It is practical and interpretable.

[0057] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0058] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for automatically configuring preset positions in a remote intelligent inspection system, characterized in that, include: S1. Decode the patrol video stream or video file obtained from the remote intelligent patrol system, identify each patrol target point in the video and match it with a unique trajectory through the target point detection model and multi-target tracking algorithm, and generate a list of points containing trajectory information. S2. Based on the location list, construct a matrix model describing the coverage relationship between video frames and patrol locations, solve for the optimal set of frames covering all patrol locations using an optimization algorithm, and select the final optimal preset location image and corresponding equipment control parameters from the optimal frame set according to preset rules to generate a preset location list.

2. The method for automatic configuration of preset positions for a remote intelligent inspection system according to claim 1, characterized in that, Step S1 specifically includes: S11. Decode the inspection video stream or video file obtained from the remote intelligent inspection system into video frames in the form of multiple consecutive images; S12. Construct a target location detection model based on a deep neural network model, and train and fine-tune the target detection model using the target location image set of the remote intelligent patrol system; S13. Use the fine-tuned target point detection model to identify points in each video frame and obtain the point information of all patrol target points in each video frame. S14. For all video frames, based on the obtained point information, use a multi-target tracking algorithm to perform trajectory matching to obtain the trajectory information of all patrol target points; S15. After filtering the trajectory information, obtain the video frame-location information with trajectory information, which is used as the location list.

3. The method for automatically configuring preset positions for a remote intelligent inspection system according to claim 2, characterized in that, In step S12, the target point detection model is established based on the YOLOv8 architecture; in step S14, the multi-target tracking algorithm adopts the ByteTrack multi-target tracking algorithm.

4. The method for automatically configuring preset positions for a remote intelligent inspection system according to claim 2, characterized in that, The location information includes: location type, location map coordinates, and confidence level; step S14 specifically includes: S141. Based on whether the confidence level in the point information of each video frame reaches the preset first confidence threshold, classify the point block diagram of the current video frame into high confidence detection boxes and low confidence detection boxes, and create trajectory information for the first point to appear, and assign a unique trajectory ID. S142. Based on the existing trajectory information, predict the trajectory of each point in the current frame, and obtain the current state of each predicted trajectory; S143. Perform multi-stage matching according to the preset priority order, and sequentially associate and match different categories of detection boxes with predicted trajectories of different states; S144. Based on the matching results, assign or continue the corresponding trajectory ID to the successfully matched points, and update the status and information of the corresponding trajectory; Step S15 specifically includes: S151. Processing the trajectory information obtained in the previous step, unifying the point type within a single trajectory, and filtering out abnormal data in the trajectory based on the area size of the point bounding box, and finally outputting structured video frame-point information containing trajectory ID, point bounding box coordinates and confidence level.

5. The method for automatically configuring preset positions for a remote intelligent inspection system according to claim 2, characterized in that, In step S142, the trajectory state is divided into an active state and an inactive state; step S143 specifically includes: S143a. Match the high-confidence detection box in the current frame in step S141 with the trajectory predicted in step S142 that is in an active state. When the match is successful, set the trajectory state to active and the corresponding detection box state to tracked. When the match fails, mark the active but unmatched trajectory and the unmatched detection box respectively. S143b. Match the inactive trajectory predicted in step S142 with the unmatched detection box in step S143a. If the match is successful, set the trajectory status to active and the corresponding detection box status to tracked. If the match fails, further determine whether the confidence level of the detection box reaches the preset second confidence threshold. If it does, add a new trajectory, assign a trajectory ID, and set the detection box status to tracked. If it does not reach the threshold, keep the trajectory status inactive and the detection box status unmatched. S143c. Match the low-confidence detection box of the current frame in step S141 with the active but unmatched trajectory in step S143a. If the match is successful, set the trajectory state to active and the corresponding detection box state to tracked. If the match fails, keep the trajectory state active and the detection box state unmatched. S143d. The unmatched trajectories that were left over from step S143a and were inactive are kept inactive in the current frame.

6. The method for automatically configuring preset positions for a remote intelligent inspection system according to claim 4, characterized in that, In step S151, the step of unifying the point type within a single trajectory includes: obtaining the trajectory information of each point, determining whether the same trajectory contains different point types, and if there are multiple point types in a trajectory, selecting the point type with the highest average confidence as the point type of the entire trajectory.

7. The method for automatically configuring preset positions for a remote intelligent inspection system according to claim 1, characterized in that, Step S2 specifically includes: S21. Based on the list of locations, construct a binary matrix model as the first matrix, with video frames as rows and patrol location IDs as columns, to describe the coverage relationship between video frames and patrol locations, where the element values ​​indicate whether the corresponding frame contains the corresponding location; perform preprocessing on the first matrix, including row deduplication and zero-row removal, to obtain the second matrix; S22. With the goal of minimizing the number of preset positions and the constraint of covering all inspection points, the row selection problem of the second matrix is ​​modeled as a 0-1 integer programming problem to form an optimization algorithm, and the CBC solver is used to solve it to obtain the optimal row vector combination, which is the optimal frame set covering all inspection points. S23. For each row in the optimal row vector combination, find all frames with the same point coverage pattern in the original video frame, calculate the minimum confidence value of all covered points in each frame, select the frame with the largest minimum value as the final optimal preset position image, and record its timestamp and corresponding device control parameters.

8. The method for automatically configuring preset positions for a remote intelligent inspection system according to claim 6, characterized in that, In step S22, the 0-1 integer programming problem is expressed as: s.t. Where S is the number of rows in the second matrix, and N is the total number of inspection points. For the elements of the second matrix, As decision variables, This indicates whether to select row i.

9. A preset position automatic configuration system for a remote intelligent inspection system, used to implement the method described in any one of claims 1 to 8, characterized in that, include: The point tracking module is used to decode the patrol video stream or video file obtained from the remote intelligent patrol system, identify each patrol target point in the video and match it with a unique trajectory through the target point detection model and multi-target tracking algorithm, and generate a point list containing trajectory information. The preset position optimization module is used to construct a matrix model describing the coverage relationship between video frames and patrol points based on the point list, solve for the optimal set of frames covering all patrol points through an optimization algorithm, and select the final optimal preset position image and corresponding equipment control parameters from the optimal frame set according to preset rules to generate a preset position list.

10. The preset position automatic configuration system for a remote intelligent inspection system according to claim 9, characterized in that, The point tracking module includes: The decoding unit is used to decode the inspection video stream or video file obtained from the remote intelligent inspection system into video frames in the form of multiple consecutive images; The modeling unit is used to build a target location detection model based on a deep neural network model, and to train and fine-tune the target detection model using the target location image set of the remote intelligent inspection system. The detection unit is used to identify points in each video frame using the fine-tuned target point detection model, and obtain the point information of all patrol target points in each video frame. The tracking unit is used to perform trajectory matching based on the obtained point information for all video frames using a multi-target tracking algorithm to obtain the trajectory information of all patrol target points. After filtering, the video frame-point information with trajectory information is obtained as the point list. The filtering unit is used to filter the trajectory information to obtain video frame-point information with trajectory information, which serves as the point list. The preset position optimization module includes: The matrix construction unit is used to construct a binary matrix model, namely the first matrix, based on the point list, with video frames as rows and patrol point IDs as columns, to describe the coverage relationship between video frames and patrol points. The element values ​​indicate whether the corresponding frame contains the corresponding point. The first matrix is ​​preprocessed to remove duplicate rows and remove rows with all zeros to obtain the second matrix. The optimization unit is used to model the row selection problem of the second matrix as a 0-1 integer programming problem with the objective of minimizing the number of preset positions and the constraint of covering all inspection points. The optimization algorithm is formed and solved by CBC solver to obtain the optimal row vector combination, which is the optimal frame set covering all inspection points. The preset position selection unit is used to find all frames with the same point coverage pattern in the original video frame for each row in the optimal row vector combination, calculate the minimum confidence value of all covered points in each frame, select the frame with the largest minimum value as the final optimal preset position image, and record its timestamp and corresponding device control parameters.

Images