An operation and maintenance platform management system and method based on semantic trajectory analysis

Abstract

The application discloses an operation and maintenance platform management system and method based on semantic trajectory analysis, belongs to the technical field of semantic trajectory analysis, and comprises an operation and maintenance platform server and a plurality of terminal operation and maintenance devices; basic semantic trajectory data and computer control operation instructions are associated to construct a semantic trajectory analysis model; an exclusive semantic trajectory data database is constructed for each operation and maintenance personnel on the operation and maintenance platform server, and the semantic trajectory analysis model is optimized based on the exclusive semantic trajectory data database, so that different exclusive semantic trajectory analysis models are formed for different operation and maintenance personnel; the application improves the accuracy of operation and maintenance personnel semantic trajectory data control instruction recognition and the convenience of operation and maintenance personnel control of the operation and maintenance system; the exclusive semantic trajectory analysis model is constructed in combination with the semantic trajectory habits of the operation and maintenance personnel, and the accuracy of operation and maintenance personnel semantic trajectory data control instruction recognition is effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of semantic trajectory analysis technology, and in particular relates to an operation and maintenance platform management system and method based on semantic trajectory analysis. Background Technology

[0002] Traditional semantic trajectory recognition control typically relies on model training based on large amounts of data, demonstrating good recognition and control performance for common, general vocabulary. However, in specialized fields with high safety requirements, such as the operation and maintenance management of power equipment, specialized maintenance personnel are usually assigned to perform maintenance tasks. These personnel are few in number, and each individual has significant differences in their semantic habits and control command delivery habits. Traditional semantic trajectory recognition control methods have relatively low adaptability in such scenarios, requiring personalized settings for maintenance personnel to improve the ease of control they can achieve with the maintenance system. Summary of the Invention

[0003] The purpose of this invention is to provide an operation and maintenance platform management system and method based on semantic trajectory analysis to address the shortcomings of the prior art and solve the problems mentioned in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] An operation and maintenance platform management system based on semantic trajectory analysis includes an operation and maintenance platform server and multiple terminal operation and maintenance devices connected to it;

[0006] The operation and maintenance platform server is used to pre-build and store a semantic trajectory database and a computer control operation instruction database that stores computer control operation instructions, and associate the semantic trajectory data and computer control operation instructions to build a semantic analysis model; a dedicated semantic trajectory database is built for each operation and maintenance personnel, and the semantic analysis model is optimized based on the dedicated semantic trajectory database to form different dedicated semantic trajectory analysis models for different operation and maintenance personnel.

[0007] The terminal maintenance device is used to upload the semantic trajectory data of the currently operating maintenance personnel to the maintenance platform server and input it into the personnel's dedicated semantic trajectory analysis model. It then obtains computer control operation instructions with a keyword relevance greater than a preset threshold, sorts them based on their relevance, creates a basic computer control operation instruction list, and transmits it to the terminal maintenance device. The terminal maintenance device executes the computer control operation instructions selected by the currently operating maintenance personnel from the basic list and uploads these instructions to the maintenance platform server to optimize the personnel's dedicated semantic trajectory analysis model, thereby improving the relevance between the keywords and the selected computer control operation instructions.

[0008] As a further preferred embodiment of the operation and maintenance platform management system based on semantic trajectory analysis of the present invention, the construction and storage of a semantic trajectory database specifically includes the following steps;

[0009] Step 1: Starting from the features of semantic trajectory, construct an abstract conceptual model of semantic trajectory; where the abstract conceptual model includes trajectory, trajectory segment, stay, travel, travel chain, and activity.

[0010] Step 2: Formalize the semantic trajectory;

[0011] Step 3, support for spatial data types and open-source extensibility, enables the use of semantic trajectory data types and other native PostgreSQL data types in the database;

[0012] Step 4: Implement operations related to the semantic trajectory data type, achieve effective storage of trajectory data, and build a semantic trajectory database.

[0013] As a further preferred embodiment of the operation and maintenance platform management system based on semantic trajectory analysis of the present invention, step 1 is specifically as follows:

[0014] Step 1.1: Acquire the original image and use image semantic segmentation technology in deep learning to segment the dynamic objects in the original image to obtain a semantic image containing pixel-level semantic information of the dynamic objects;

[0015] Step 1.2: Extract candidate points from the original image, and remove candidate points in dynamic regions based on the semantic image obtained in Step 1, retaining only candidate points in static regions;

[0016] Step 1.3: Based on the retained candidate points, a pyramid model that integrates image semantic information is used to estimate the camera pose;

[0017] Step 1.4: Finally, based on sliding window optimization and combined with image semantic information, the pose of keyframes is optimized, thereby constructing an abstract conceptual model of semantic trajectory.

[0018] As a further preferred embodiment of the operation and maintenance platform management system based on semantic trajectory analysis of the present invention, step 1.3 specifically includes the following steps:

[0019] Step 1.31, for keyframe I i The image was scaled with a scaling factor of 0.5 to obtain the results relative to I. i resolution The image, will I i An image pyramid is constructed using the three scaled images in ascending order of resolution. The image at the k-th level of the image pyramid is denoted as... And perform static region candidate point extraction for each layer of the pyramid as described in step 2;

[0020] Step 1.32, for subsequent frame I j and its corresponding semantic image I jsem Construct the same image pyramid as in step 3.1. The images at the k-th level of the image pyramid are denoted as and respectively. and

[0021] Step 1.33, for For a single candidate point p in the image, calculate its projection onto the image. The resulting photometric error:

[0022]

[0023] Where p′ is p in The projection point on t i and t j These are images I i and I j Exposure time, a i ,a j b i ,b j N is the photometric transfer function parameter of the image. p It is a set of 8 points including point p and its surrounding neighbors, w p It is a weighting factor, ||.|| γ It is the Huber norm;

[0024] Step 1.34: For each candidate point p, based on its position in... Calculate a removal label based on the position of the projection point p′ in the middle. Determine whether the projection residual at this point should be removed:

[0025]

[0026] Step 1.35, will The projection residuals at each point are summed, and the residuals projected onto the dynamic region are removed to obtain the sum of the projection residuals of the k-th layer of the pyramid:

[0027]

[0028] Step 1.36: Optimize the relative pose T at layer k-1. k-1 As the initial values ​​for this optimization, the Gauss-Newton method was used to... Optimization can yield keyframes. and subsequent frames The relative pose T between k ;

[0029] Step 1.37: Repeat steps 3.3-3.6 for all layers of the pyramid in a top-to-bottom order to finally obtain keyframe I. i and subsequent frames I j The relative pose T between k .

[0030] As a further preferred embodiment of the operation and maintenance platform management system based on semantic trajectory analysis of the present invention, in step 1.4, the keyframe pose is optimized using a sliding window and semantic information, specifically including the following steps:

[0031] Step 1.41, for keyframe I i A single point p in the sliding window projects onto another keyframe I. l The resulting photometric error is:

[0032]

[0033] Where p′ is p in I l The projection point on t i and t l These are images I i and I l Exposure time, a i ,a l b i ,b l N is the photometric transfer function parameter of the image. p It is a set of 8 points including point p and its surrounding neighbors, w p It is a weighting factor, ||.|| γ It is the Huber norm;

[0034] Step 1.42, for keyframe I i Each candidate point p, based on its position in I... lCalculate a removal label C based on the position of the projection point p′ in the middle. pl Determine whether the projection residual at this point should be removed:

[0035] C pl =I lsem (p′);

[0036] Step 1.43: Traverse all keyframes within the sliding window, project all candidate points from each keyframe onto other keyframes within the window, and count and accumulate all photometric errors.

[0037]

[0038] Where F is the set of all keyframes within the sliding window, and P... i It is keyframe I i The set of all candidate points in the dataset, where obs(p) refers to the set of keyframes from which point p can be observed;

[0039] Step 1.44: Use Gauss-Newton's method to analyze E. full By performing optimization, the optimized poses of all keyframes can be obtained, thus completing the tracking of camera motion.

[0040] As a further preferred embodiment of the operation and maintenance platform management system based on semantic trajectory analysis of the present invention,

[0041] The method of using semantic trajectory analysis to assist in operation and maintenance management also includes the following steps:

[0042] If the number of computer control operation commands with a keyword correlation greater than a preset threshold analyzed by the current operation and maintenance personnel's dedicated semantic trajectory analysis model is 0, then the keyword will be uploaded sequentially to the dedicated semantic trajectory analysis models of other operation and maintenance personnel.

[0043] The computer control operation instructions with the highest keyword relevance are obtained by using the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel.

[0044] The computer control operation instructions obtained from the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel are integrated and sorted to form a list of candidate computer control operation instructions, which is then pushed to the terminal operation and maintenance equipment for the current operation and maintenance personnel to select.

[0045] If a computer operation control instruction is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation instruction selected by the current operation and maintenance personnel from the list of alternative computer control operation instructions, and uploads the computer control operation instruction selected by the current operation and maintenance personnel to the operation and maintenance platform server to optimize the exclusive semantic trajectory analysis model of the current operation and maintenance personnel, add the correlation between the keywords and the selected computer control operation instruction and improve the correlation.

[0046] If no computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command found and selected by the current operation and maintenance personnel, and uploads the computer control operation command found and selected by the current operation and maintenance personnel to the operation and maintenance platform server to optimize the exclusive semantic trajectory analysis model of the current operation and maintenance personnel, add the correlation between the keyword and the selected computer control operation command and improve the correlation.

[0047] As a further preferred embodiment of the operation and maintenance platform management system based on semantic trajectory analysis of the present invention, if the difference between the correlation degree of the computer control operation instruction list ranked first in the basic computer control operation instruction list and the correlation degree of the computer control operation instruction list ranked second in the basic computer control operation instruction list is greater than a preset difference, the computer control operation instruction ranked first in the correlation degree is broadcast to the operation and maintenance personnel currently operating and a request is made to confirm whether to execute the computer control operation instruction.

[0048] When the terminal maintenance device receives confirmation information input by the maintenance personnel currently operating through semantic trajectory data, it is considered that the maintenance personnel currently operating have selected a computer control operation instruction from the basic computer control operation instruction list, and the terminal maintenance device executes the selected computer control operation instruction.

[0049] If the basic computer control operation instruction list contains only one computer control operation instruction, then the correlation of the computer control operation instruction list ranked second in correlation is considered to be 0.

[0050] As a further preferred embodiment of the operation and maintenance platform management system based on semantic trajectory analysis of the present invention,

[0051] The terminal maintenance device transmits the login information of the maintenance personnel to the maintenance platform server for identity verification. Once the identity verification is successful, the maintenance personnel are allowed to log in.

[0052] After receiving the semantic trajectory data control command from the currently operating maintenance personnel, the terminal maintenance device first confirms through voiceprint recognition whether the semantic trajectory data control command comes from the corresponding maintenance personnel whose identity has been verified. If so, the terminal maintenance device converts the semantic trajectory data control command from the currently operating maintenance personnel into text.

[0053] An operation and maintenance management method based on semantic trajectory analysis includes the following steps:

[0054] A basic semantic trajectory data database storing basic semantic trajectory data and a computer control operation instruction database storing computer control operation instructions are pre-built on the operation and maintenance platform server, and the basic semantic trajectory data and computer control operation instructions are associated to construct a semantic trajectory analysis model;

[0055] On the operations and maintenance platform server, a dedicated semantic trajectory data database is built for each operations and maintenance personnel, and the semantic trajectory analysis model is optimized based on the dedicated semantic trajectory data database to form different dedicated semantic trajectory analysis models for different operations and maintenance personnel.

[0056] The terminal maintenance equipment converts the semantic trajectory data control commands of the current maintenance personnel into text, extracts keywords, and uploads the keywords to the maintenance platform server. These keywords are then fed into the current maintenance personnel's exclusive semantic trajectory analysis model. The system obtains computer control operation commands with a correlation greater than a preset threshold based on the correlation of the keywords, sorts them according to their correlation, creates a basic list of computer control operation commands, and transmits it to the terminal maintenance equipment.

[0057] The terminal maintenance equipment executes the computer control operation instruction selected by the maintenance personnel from the basic computer control operation instruction list, and uploads the selected computer control operation instruction to the maintenance platform server to optimize the exclusive semantic trajectory analysis model of the maintenance personnel, thereby improving the correlation between the keywords and the selected computer control operation instruction.

[0058] As a further preferred embodiment of the operation and maintenance management method based on semantic trajectory analysis of the present invention,

[0059] The operation and maintenance management method based on semantic trajectory analysis also includes the following steps:

[0060] If the number of computer control operation commands with a keyword correlation greater than a preset threshold analyzed by the current operation and maintenance personnel's dedicated semantic trajectory analysis model is 0, then the keyword will be uploaded sequentially to the dedicated semantic trajectory analysis models of other operation and maintenance personnel.

[0061] The computer control operation instructions with the highest keyword relevance are obtained by using the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel.

[0062] The computer control operation instructions obtained from the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel are integrated and sorted to form a list of candidate computer control operation instructions, which is then pushed to the terminal operation and maintenance equipment for the current operation and maintenance personnel to select.

[0063] If a computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command selected by the current operation and maintenance personnel from the list of candidate computer control operation command commands, and uploads the selected computer control operation command to the operation and maintenance platform server to optimize the current operation and maintenance personnel's exclusive semantic trajectory analysis model, adding the correlation between the keywords and the selected computer control operation command and improving the correlation. If no computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command found and selected by the current operation and maintenance personnel, and uploads the found and selected computer control operation command to the operation and maintenance platform server to optimize the current operation and maintenance personnel's exclusive semantic trajectory analysis model, adding the correlation between the keywords and the selected computer control operation command and improving the correlation.

[0064] If the difference between the correlation of the computer control operation instruction list ranked first and the correlation of the computer control operation instruction list ranked second in the basic computer control operation instruction list is greater than a preset difference, the computer control operation instruction ranked first in the correlation will be broadcast to the current operation and maintenance personnel and a request will be made to confirm whether to execute the computer control operation instruction.

[0065] When the terminal maintenance equipment receives confirmation information input by the maintenance personnel currently operating through semantic trajectory data, it is considered that the maintenance personnel currently operating have selected a computer control operation instruction from the basic computer control operation instruction list, and the terminal maintenance equipment executes the selected computer control operation instruction.

[0066] If the basic computer control operation instruction list contains only one computer control operation instruction, then the correlation of the computer control operation instruction list ranked second in correlation is considered to be 0.

[0067] Compared with the prior art, the beneficial effects of the present invention are:

[0068] 1. This invention constructs a unique semantic trajectory analysis model for each operations and maintenance personnel, thereby improving the accuracy of recognizing control commands based on semantic trajectory data of operations and maintenance personnel, and thus improving the convenience of operations and maintenance personnel in controlling the operations and maintenance system;

[0069] 2. The semantic trajectory database constructed in this invention fills the gap in spatial database support for trajectory data types, ensuring the integrity and consistency of trajectory data in space and time; organizing trajectory data by aggregation transforms the original low-correlation spatiotemporal point data into highly correlated fragmented and block data storage, which can significantly improve the storage efficiency of large trajectory data; the native trajectory database built based on an abstract conceptual model is suitable for use on various cross-platforms, is easy and quick to operate, and has strong practicality.

[0070] 3. This invention uses supervised learning in deep learning to segment dynamic objects in the original image to obtain a semantic image; on this basis, candidate points are extracted from the original image and dynamic object candidate points are removed according to the semantic image, thereby improving the positioning accuracy and robustness of traditional visual odometry based on direct methods in dynamic scenes.

[0071] 4. Based on basic semantic trajectory data, this invention constructs a dedicated semantic trajectory analysis model by combining the semantic trajectory habits of operation and maintenance personnel, which effectively improves the accuracy of recognizing control commands based on semantic trajectory data of operation and maintenance personnel; the dedicated semantic trajectory analysis model can be repeatedly optimized based on the semantic trajectory habits of operation and maintenance personnel, and the dedicated semantic trajectory analysis model is optimized once after each operation command is issued;

[0072] 5. For newly joined operations and maintenance personnel, this invention utilizes keywords to input into the exclusive semantic trajectory analysis models of other operations and maintenance personnel, thereby leveraging the exclusive semantic trajectory analysis models of other operations and maintenance personnel to assist in optimizing their own exclusive semantic trajectory analysis models, thus rapidly improving the recognition accuracy of the exclusive semantic trajectory analysis models.

[0073] 6. In the process of repeatedly optimizing the exclusive semantic trajectory analysis model, the computer control operation instruction with the highest correlation will be much higher than other computer control operation instructions, thus controlling the execution of the computer control operation instruction;

[0074] 7. This invention uses voiceprint recognition technology to confirm whether the semantic trajectory data control command comes from the corresponding operation and maintenance personnel whose identity has been verified, thereby ensuring the security of semantic trajectory data control;

[0075] 8. This invention uses terminal maintenance equipment to perform voiceprint recognition, semantic trajectory data conversion, and keyword extraction, thereby reducing the computational burden on the server. Attached Figure Description

[0076] Figure 1 This is a structural block diagram of an operation and maintenance management system based on semantic trajectory analysis according to the present invention;

[0077] Figure 2 This is a flowchart of an operation and maintenance management method based on semantic trajectory analysis according to the present invention;

[0078] Figure 3 This is a schematic diagram of the semantic trajectory database construction method of the present invention;

[0079] Figure 4 This is a schematic diagram of the abstract conceptual model for constructing semantic trajectories in this invention. Detailed Implementation

[0080] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0081] like Figure 1 , Figure 2 , Figure 3 and Figure 4 As shown in the embodiments of the present invention,

[0082] An operation and maintenance management method based on semantic trajectory analysis, and an operation and maintenance platform management system based on semantic trajectory analysis, including an operation and maintenance platform server and multiple terminal operation and maintenance devices connected thereto;

[0083] The operation and maintenance platform server is used to pre-build and store a semantic trajectory database and a computer control operation instruction database that stores computer control operation instructions, and associate the semantic trajectory data and computer control operation instructions to build a semantic analysis model; a dedicated semantic trajectory database is built for each operation and maintenance personnel, and the semantic analysis model is optimized based on the dedicated semantic trajectory database to form different dedicated semantic trajectory analysis models for different operation and maintenance personnel.

[0084] The terminal maintenance device is used to upload the semantic trajectory data of the currently operating maintenance personnel to the maintenance platform server and input it into the personnel's dedicated semantic trajectory analysis model. It then obtains computer control operation instructions with a keyword relevance greater than a preset threshold, sorts them based on their relevance, creates a basic computer control operation instruction list, and transmits it to the terminal maintenance device. The terminal maintenance device executes the computer control operation instructions selected by the currently operating maintenance personnel from the basic list and uploads these instructions to the maintenance platform server to optimize the personnel's dedicated semantic trajectory analysis model, thereby improving the relevance between the keywords and the selected computer control operation instructions.

[0085] like Figure 3 As shown, the construction of a semantic trajectory database includes the following steps;

[0086] Step 1: Starting from the features of semantic trajectory, construct an abstract conceptual model of semantic trajectory; where the abstract conceptual model includes trajectory, trajectory segment, stay, travel, travel chain, and activity.

[0087] Step 2: Formalize the semantic trajectory;

[0088] Step 3, support for spatial data types and open-source extensibility, enables the use of semantic trajectory data types and other native PostgreSQL data types in the database;

[0089] Step 4: Implement operations related to the semantic trajectory data type, achieve effective storage of trajectory data, and build a semantic trajectory database.

[0090] like Figure 4 As shown, step 1 is as follows:

[0091] Step 1.1: Acquire the original image and use image semantic segmentation technology in deep learning to segment the dynamic objects in the original image to obtain a semantic image containing pixel-level semantic information of the dynamic objects;

[0092] Step 1.2: Extract candidate points from the original image, and remove candidate points in dynamic regions based on the semantic image obtained in Step 1, retaining only candidate points in static regions;

[0093] Step 1.3: Based on the retained candidate points, a pyramid model that integrates image semantic information is used to estimate the camera pose;

[0094] Step 1.4: Finally, based on sliding window optimization and combined with image semantic information, the pose of keyframes is optimized, thereby constructing an abstract conceptual model of semantic trajectory.

[0095] Step 1.3 specifically includes the following steps:

[0096] Step 1.31, for keyframe I i The image was scaled with a scaling factor of 0.5 to obtain the results relative to I. i resolution The image, will I i An image pyramid is constructed using the three scaled images in ascending order of resolution. The image at the k-th level of the image pyramid is denoted as... Then, static region candidate points are extracted for each layer of the pyramid as described in step 1.2.

[0097] Step 1.32, for subsequent frame I j and its corresponding semantic image I jsem Construct the same image pyramid as in step 3.1. The images at the k-th level of the image pyramid are denoted as and respectively. and

[0098] Step 1.33, for For a single candidate point p in the image, calculate its projection onto the image. The resulting photometric error:

[0099]

[0100] Where p′ is p in The projection point on t i and t j These are images I i and I j Exposure time, a i ,a j b i ,b j N is the photometric transfer function parameter of the image. p It is a set of 8 points including point p and its surrounding neighbors, w p It is a weighting factor, ||.|| γ It is the Huber norm;

[0101] Step 1.34: For each candidate point p, based on its position in... Calculate a removal label based on the position of the projection point p′ in the middle. Determine whether the projection residual at this point should be removed:

[0102]

[0103] Step 1.35, The projection residuals at each point are summed, and the residuals projected onto the dynamic region are removed to obtain the sum of the projection residuals of the k-th layer of the pyramid:

[0104]

[0105] Step 1.36: Optimize the relative pose T at layer k-1. k-1 As the initial values ​​for this optimization, the Gauss-Newton method was used to... Optimization can yield keyframes. and subsequent frames The relative pose T between k ;

[0106] Step 1.37: Repeat steps 3.3-3.6 for all layers of the pyramid in a top-to-bottom order to finally obtain keyframe I. i and subsequent frames I j The relative pose T between k .

[0107] In step 1.4, the keyframe pose is optimized using a sliding window and semantic information, specifically including the following steps:

[0108] Step 1.41, for keyframe I i A single point p in the sliding window projects onto another keyframe I. l The resulting photometric error is:

[0109]

[0110] Where p′ is p in I l The projection point on t i and t l These are images I i and I l Exposure time, a i ,a l b i ,b l N is the photometric transfer function parameter of the image. p It is a set of 8 points including point p and its surrounding neighbors, w p It is a weighting factor, ||.|| γ It is the Huber norm;

[0111] Step 1.42, for keyframe I i Each candidate point p, based on its position in I... l Calculate a removal label C based on the position of the projection point p′ in the middle. pl Determine whether the projection residual at this point should be removed:

[0112] C pl =Ilsem (p′);

[0113] Step 1.43: Traverse all keyframes within the sliding window, project all candidate points from each keyframe onto other keyframes within the window, and count and accumulate all photometric errors.

[0114]

[0115] Where F is the set of all keyframes within the sliding window, and P... i It is keyframe I i The set of all candidate points in the dataset, where obs(p) refers to the set of keyframes from which point p can be observed;

[0116] Step 1.44: Use Gauss-Newton's method to analyze E. full By performing optimization, the optimized poses of all keyframes can be obtained, thus completing the tracking of camera motion.

[0117] As a preferred implementation, an operation and maintenance management method based on semantic trajectory analysis further includes the following steps:

[0118] If the number of computer control operation commands with a keyword correlation greater than a preset threshold analyzed by the current operation and maintenance personnel's dedicated semantic trajectory analysis model is 0, then the keyword will be uploaded sequentially to the dedicated semantic trajectory analysis models of other operation and maintenance personnel.

[0119] The computer control operation instructions with the highest keyword relevance are obtained by using the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel.

[0120] The computer control operation instructions obtained from the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel are integrated and sorted to form a list of candidate computer control operation instructions, which is then pushed to the terminal operation and maintenance equipment for the current operation and maintenance personnel to select. The integration and sorting of computer control operation instructions includes deduplication of duplicate data and sorting based on the number of times the duplicate data is repeated.

[0121] If a computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command selected by the current operation and maintenance personnel from the list of candidate computer control operation command commands. The selected command is then uploaded to the operation and maintenance platform server to optimize the personnel's dedicated semantic trajectory analysis model, adding keywords and improving the correlation between keywords and the selected command. If no computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command found and selected by the current operation and maintenance personnel. This command is then uploaded to the operation and maintenance platform server to optimize the personnel's dedicated semantic trajectory analysis model, adding keywords and improving the correlation between keywords and the selected command.

[0122] As a preferred implementation, if the difference between the correlation of the computer control operation instruction list ranked first and the correlation of the computer control operation instruction list ranked second in the basic computer control operation instruction list is greater than a preset difference, the computer control operation instruction ranked first in the correlation is broadcast to the current operation maintenance personnel through semantic trajectory broadcasting and a request is made to confirm whether to execute the computer control operation instruction.

[0123] When the terminal maintenance equipment receives confirmation information from the maintenance personnel currently performing the operation via semantic trajectory input, it is considered that the maintenance personnel currently performing the operation have selected a computer control operation instruction from the list of basic computer control operation instructions, and the terminal maintenance equipment executes the selected computer control operation instruction.

[0124] As a preferred implementation, if the basic computer control operation instruction list contains only one computer control operation instruction, then the correlation of the computer control operation instruction list ranked second in correlation is considered to be 0.

[0125] As a preferred implementation, the terminal maintenance device transmits the login information of the maintenance personnel to the maintenance platform server for authentication. After successful authentication, the maintenance personnel are allowed to log in.

[0126] After receiving the semantic trajectory control command from the currently operating maintenance personnel, the terminal maintenance equipment first uses voiceprint recognition to confirm whether the semantic trajectory control command comes from the corresponding maintenance personnel whose identity has been verified. If so, the terminal maintenance equipment converts the semantic trajectory control command from the currently operating maintenance personnel into text.

[0127] An operation and maintenance management system based on semantic trajectory analysis includes: an operation and maintenance platform server and multiple terminal operation and maintenance devices; the terminal operation and maintenance devices communicate with the operation and maintenance platform server.

[0128] An operation and maintenance management system based on semantic trajectory analysis, the method of using semantic trajectory analysis to assist operation and maintenance management includes the following steps:

[0129] A basic semantic trajectory data database storing basic semantic trajectory data and a computer control operation instruction database storing computer control operation instructions are pre-built on the operation and maintenance platform server, and the basic semantic trajectory data and computer control operation instructions are associated to construct a semantic trajectory analysis model;

[0130] On the operations and maintenance platform server, a dedicated semantic trajectory data database is built for each operations and maintenance personnel, and the semantic trajectory analysis model is optimized based on the dedicated semantic trajectory data database to form different dedicated semantic trajectory analysis models for different operations and maintenance personnel.

[0131] The terminal maintenance equipment converts the semantic trajectory control commands of the current maintenance personnel into text, extracts keywords, and uploads the keywords to the maintenance platform server. These keywords are then fed into the current maintenance personnel's exclusive semantic trajectory analysis model. The system obtains computer control operation commands with a correlation greater than a preset threshold based on the correlation of the keywords, sorts them according to their correlation, creates a basic list of computer control operation commands, and transmits it to the terminal maintenance equipment.

[0132] The terminal maintenance equipment executes the computer control operation instructions selected by the maintenance personnel from the basic computer control operation instruction list, and uploads the selected computer control operation instructions to the maintenance platform server to optimize the exclusive semantic trajectory analysis model of the maintenance personnel, thereby improving the correlation between keywords and the selected computer control operation instructions.

[0133] As a preferred implementation method, the method of using semantic trajectory analysis to assist in operation and maintenance management further includes the following steps:

[0134] If the number of computer control operation commands with a keyword correlation greater than a preset threshold analyzed by the current operation and maintenance personnel's dedicated semantic trajectory analysis model is 0, then the keyword will be uploaded sequentially to the dedicated semantic trajectory analysis models of other operation and maintenance personnel.

[0135] The computer control operation instructions with the highest keyword relevance are obtained by using the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel.

[0136] The computer control operation instructions obtained from the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel are integrated and sorted to form a list of candidate computer control operation instructions, which is then pushed to the terminal operation and maintenance equipment for the current operation and maintenance personnel to select.

[0137] If a computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command selected by the current operation and maintenance personnel from the list of alternative computer control operation commands, and uploads the computer control operation command selected by the current operation and maintenance personnel to the operation and maintenance platform server to optimize the exclusive semantic trajectory analysis model of the current operation and maintenance personnel, add the correlation between keywords and the selected computer control operation command, and improve the correlation.

[0138] If no computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command found and selected by the current operation and maintenance personnel, and uploads the computer control operation command found and selected by the current operation and maintenance personnel to the operation and maintenance platform server to optimize the exclusive semantic trajectory analysis model of the current operation and maintenance personnel, add the correlation between keywords and the selected computer control operation command, and improve the correlation.

[0139] As a preferred implementation, if the difference between the correlation of the computer control operation instruction list ranked first and the correlation of the computer control operation instruction list ranked second in the basic computer control operation instruction list is greater than a preset difference, the computer control operation instruction ranked first in the correlation is broadcast to the current operation maintenance personnel through semantic trajectory broadcasting and a request is made to confirm whether to execute the computer control operation instruction.

[0140] When the terminal maintenance equipment receives confirmation information from the maintenance personnel currently performing the operation via semantic trajectory input, it is considered that the maintenance personnel currently performing the operation have selected a computer control operation instruction from the list of basic computer control operation instructions, and the terminal maintenance equipment executes the selected computer control operation instruction.

[0141] As a preferred implementation, if the basic computer control operation instruction list contains only one computer control operation instruction, then the correlation of the computer control operation instruction list ranked second in correlation is considered to be 0.

[0142] As a preferred implementation, the terminal maintenance device transmits the login information of the maintenance personnel to the maintenance platform server for authentication. After successful authentication, the maintenance personnel are allowed to log in.

[0143] After receiving the semantic trajectory control command from the currently operating maintenance personnel, the terminal maintenance equipment first uses voiceprint recognition to confirm whether the semantic trajectory control command comes from the corresponding maintenance personnel whose identity has been verified. If so, the terminal maintenance equipment converts the semantic trajectory control command from the currently operating maintenance personnel into text.

[0144] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0145] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An operation and maintenance platform management system based on semantic trajectory analysis, characterized in that, This includes the operation and maintenance platform server and multiple terminal operation and maintenance devices connected to it; The operation and maintenance platform server is used to pre-build and store a semantic trajectory database and a computer control operation instruction database that stores computer control operation instructions, and associate the semantic trajectory data and computer control operation instructions to build a semantic analysis model; a dedicated semantic trajectory database is built for each operation and maintenance personnel, and the semantic analysis model is optimized based on the dedicated semantic trajectory database to form different dedicated semantic trajectory analysis models for different operation and maintenance personnel. Construct a semantic trajectory database, including: Step 1: Starting from the features of semantic trajectories, construct an abstract conceptual model of semantic trajectories; whereby the abstract conceptual model includes trajectory, trajectory segments, stops, trips, trip chains, and activities; Step 2: Formalize the semantic trajectory; Step 3, support for spatial data types and open-source extensibility, enables the use of semantic trajectory data types and other native PostgreSQL data types in the database; Step 4: Implement operations related to the semantic trajectory data type, achieve effective storage of trajectory data, and build a semantic trajectory database; Step 1 includes: Step 1.1: Acquire the original image and use image semantic segmentation technology in deep learning to segment the dynamic objects in the original image to obtain a semantic image containing pixel-level semantic information of the dynamic objects; Step 1.2: Extract candidate points from the original image, and remove candidate points in dynamic regions based on the semantic image obtained in Step 1, retaining only candidate points in static regions; Step 1.3: Based on the retained candidate points, a pyramid model that integrates image semantic information is used to estimate the camera pose; Step 1.4: Finally, the pose of the keyframe is optimized based on the sliding window and combined with the semantic information of the image, thereby constructing an abstract conceptual model of the semantic trajectory. The terminal maintenance device is used to upload the semantic trajectory data of the currently operating maintenance personnel to the maintenance platform server and input it into the personnel's dedicated semantic trajectory analysis model. It then obtains computer control operation instructions with a keyword relevance greater than a preset threshold, sorts them based on their relevance, creates a basic computer control operation instruction list, and transmits it to the terminal maintenance device. The terminal maintenance device executes the computer control operation instructions selected by the currently operating maintenance personnel from the basic list and uploads these instructions to the maintenance platform server to optimize the personnel's dedicated semantic trajectory analysis model, thereby improving the relevance between the keywords and the selected computer control operation instructions.

2. The operation and maintenance platform management system based on semantic trajectory analysis according to claim 1, characterized in that, Step 1.3 specifically includes the following steps: Step 1.31, for keyframes The image was scaled with a scaling factor of 0.5 to obtain the results relative to the original image. resolution The image will An image pyramid is constructed using the three scaled images in ascending order of resolution. The image pyramid is then... k Layer image denoted as And extract static region candidate points for each layer of the pyramid as described in step 2; Step 1.32, for subsequent frames and its corresponding semantic image Construct the same image pyramid as in step 1.31, the image pyramid of the [missing information] generation. k The images of the layers are denoted as and respectively. and ; Step 1.33, for A single candidate point in Calculate its projection onto the image The resulting photometric error: in, yes exist Projection points on, and These are images and Exposure time, , , , These are the parameters of the image's photometric transfer function. It includes The set of 8 points including the point and its surrounding neighboring points. It is a weighting factor. It is the Huber norm; Step 1.34: For each candidate point According to its Mid-projection point Calculate a removal label based on the position. Determine whether the projection residual at this point should be removed: ； Step 1.35, The projection residuals at each point in the pyramid are accumulated, and the residuals projected onto the dynamic region are removed to obtain the pyramid's first... k The sum of the projection residuals of the layers: ； Step 1.36, in Relative pose optimization results of the layer As the initial values ​​for this optimization, the Gauss-Newton method was used to... Optimization can yield keyframes. and subsequent frames Relative pose between ; Step 1.37: Repeat steps 1.33-1.36 for all layers of the pyramid in a top-to-bottom order to obtain the keyframes. and subsequent frames Relative pose between .

3. The operation and maintenance platform management system based on semantic trajectory analysis according to claim 2, characterized in that: In step 1.4, the keyframe pose is optimized using a sliding window and semantic information, specifically including the following steps: Step 1.41, for keyframes a single point in It projects onto another keyframe in the sliding window. The resulting photometric error is: in yes exist The projection point on the surface, and These are images and Exposure time, , , , These are the parameters of the photometric transfer function of the image. It includes The set of 8 points including the point and its surrounding neighbors. It is a weighting factor. It is the Huber norm; Step 1.42, for keyframes Each candidate point According to its Mid-projection point Calculate a removal label based on the position. Determine whether the projection residual at this point should be removed: ;in, Keyframe Semantic images; Step 1.43: Traverse all keyframes within the sliding window, project all candidate points from each keyframe onto other keyframes within the window, and count and accumulate all photometric errors. in, F It is the collection of all keyframes within the sliding window. Keyframe The set of all candidate points in the set. refers to being able to observe The set of keyframes for a point; Step 1.44: Use Gauss-Newton's method to... By performing optimization, the optimized poses of all keyframes can be obtained, thus completing the tracking of camera motion.

4. The operation and maintenance management system based on semantic trajectory analysis according to claim 3, characterized in that, The method of using semantic trajectory analysis to assist in operation and maintenance management also includes the following steps: If the number of computer control operation commands with a keyword correlation greater than a preset threshold analyzed by the current operation and maintenance personnel's dedicated semantic trajectory analysis model is 0, then the keyword will be uploaded sequentially to the dedicated semantic trajectory analysis models of other operation and maintenance personnel. The computer control operation instructions with the highest keyword relevance are obtained by using the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel. The computer control operation instructions obtained from the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel are integrated and sorted to form a list of candidate computer control operation instructions, which is then pushed to the terminal operation and maintenance equipment for the current operation and maintenance personnel to select. If a computer operation control instruction is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation instruction selected by the current operation and maintenance personnel from the list of alternative computer control operation instructions, and uploads the computer control operation instruction selected by the current operation and maintenance personnel to the operation and maintenance platform server to optimize the exclusive semantic trajectory analysis model of the current operation and maintenance personnel, add the correlation between the keywords and the selected computer control operation instruction and improve the correlation. If no computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command found and selected by the current operation and maintenance personnel, and uploads the computer control operation command found and selected by the current operation and maintenance personnel to the operation and maintenance platform server to optimize the exclusive semantic trajectory analysis model of the current operation and maintenance personnel, add the correlation between the keyword and the selected computer control operation command and improve the correlation.

5. The operation and maintenance management system based on semantic trajectory analysis according to claim 4, characterized in that, If the difference between the correlation of the computer control operation instruction list ranked first and the correlation of the computer control operation instruction list ranked second in the basic computer control operation instruction list is greater than a preset difference, the computer control operation instruction ranked first in the correlation will be broadcast to the current operation and maintenance personnel and a request will be made to confirm whether to execute the computer control operation instruction. When the terminal maintenance device receives confirmation information input by the maintenance personnel currently operating through semantic trajectory data, it is considered that the maintenance personnel currently operating have selected a computer control operation instruction from the basic computer control operation instruction list, and the terminal maintenance device executes the selected computer control operation instruction. If the basic computer control operation instruction list contains only one computer control operation instruction, then the correlation of the computer control operation instruction list ranked second in correlation is considered to be 0.

6. The operation and maintenance management system based on semantic trajectory analysis according to claim 5, characterized in that, The terminal maintenance device transmits the login information of the maintenance personnel to the maintenance platform server for identity verification. Once the identity verification is successful, the maintenance personnel are allowed to log in. After receiving the semantic trajectory data control command from the currently operating maintenance personnel, the terminal maintenance equipment first confirms through voiceprint recognition whether the semantic trajectory data control command comes from the corresponding maintenance personnel whose identity has been verified.

7. An operation and maintenance management method based on semantic trajectory analysis, implementing the operation and maintenance management system based on semantic trajectory analysis as described in any one of claims 1 to 6, characterized in that, Includes the following steps: A basic semantic trajectory data database storing basic semantic trajectory data and a computer control operation instruction database storing computer control operation instructions are pre-built on the operation and maintenance platform server, and the basic semantic trajectory data and computer control operation instructions are associated to construct a semantic trajectory analysis model; On the operations and maintenance platform server, a dedicated semantic trajectory data database is built for each operations and maintenance personnel, and the semantic trajectory analysis model is optimized based on the dedicated semantic trajectory data database to form different dedicated semantic trajectory analysis models for different operations and maintenance personnel. The terminal maintenance equipment converts the semantic trajectory data control commands of the current maintenance personnel into text, extracts keywords, and uploads the keywords to the maintenance platform server. These keywords are then fed into the current maintenance personnel's exclusive semantic trajectory analysis model. The system obtains computer control operation commands with a correlation greater than a preset threshold based on the correlation of the keywords, sorts them according to their correlation, creates a basic list of computer control operation commands, and transmits it to the terminal maintenance equipment. The terminal maintenance equipment executes the computer control operation instruction selected by the maintenance personnel from the basic computer control operation instruction list, and uploads the selected computer control operation instruction to the maintenance platform server to optimize the exclusive semantic trajectory analysis model of the maintenance personnel, thereby improving the correlation between the keywords and the selected computer control operation instruction.

8. The operation and maintenance management method based on semantic trajectory analysis according to claim 7, characterized in that, The operation and maintenance management method based on semantic trajectory analysis also includes the following steps: If the number of computer control operation commands with a keyword correlation greater than a preset threshold analyzed by the current operation and maintenance personnel's dedicated semantic trajectory analysis model is 0, then the keyword will be uploaded sequentially to the dedicated semantic trajectory analysis models of other operation and maintenance personnel. The computer control operation instructions with the highest keyword relevance are obtained by using the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel. The computer control operation instructions obtained from the dedicated semantic trajectory analysis model corresponding to other operation and maintenance personnel are integrated and sorted to form a list of candidate computer control operation instructions, which is then pushed to the terminal operation and maintenance equipment for the current operation and maintenance personnel to select. If a computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command selected by the current operation and maintenance personnel from the list of candidate computer control operation command commands, and uploads the selected computer control operation command to the operation and maintenance platform server to optimize the current operation and maintenance personnel's exclusive semantic trajectory analysis model, adding the correlation between the keywords and the selected computer control operation command and improving the correlation. If no computer operation control command is selected by the current operation and maintenance personnel, the terminal operation and maintenance device executes the computer control operation command found and selected by the current operation and maintenance personnel, and uploads the found and selected computer control operation command to the operation and maintenance platform server to optimize the current operation and maintenance personnel's exclusive semantic trajectory analysis model, adding the correlation between the keywords and the selected computer control operation command and improving the correlation. If the difference between the correlation of the computer control operation instruction list ranked first and the correlation of the computer control operation instruction list ranked second in the basic computer control operation instruction list is greater than a preset difference, the computer control operation instruction ranked first in the correlation will be broadcast to the current operation and maintenance personnel and a request will be made to confirm whether to execute the computer control operation instruction. When the terminal maintenance equipment receives confirmation information input by the maintenance personnel currently operating through semantic trajectory data, it is considered that the maintenance personnel currently operating have selected a computer control operation instruction from the basic computer control operation instruction list, and the terminal maintenance equipment executes the selected computer control operation instruction. If the basic computer control operation instruction list contains only one computer control operation instruction, then the correlation of the computer control operation instruction list ranked second in correlation is considered to be 0.

Images