A camera intelligent operation and maintenance method and system based on deep learning

Abstract

The present invention discloses a camera intelligent operation and maintenance method and system based on deep learning, including: S1: collecting camera historical fault data and preprocessing to obtain preprocessed camera historical fault data; S2: according to the preprocessed camera historical fault data, counting the number of occurrences of each historical fault type and historical fault cause, and extracting the historical fault type feature vector and the historical fault cause feature vector; S3: generating camera operation and maintenance text and performing data enhancement, extracting the historical fault description feature vector, the historical maintenance measure description feature vector, the first relationship feature vector and the second relationship feature vector; S4: collecting camera current fault data, and calculating the current fault type, the first fault cause and the second fault cause; S5: generating maintenance suggestions and work orders, and selecting maintenance personnel to dispatch work orders. The present invention can solve the problem that the fault location of the traditional camera operation and maintenance method depends on manual experience.

Description

Technical Field

[0001] The present invention relates to the field of camera operation and maintenance technology, and in particular to a camera intelligent operation and maintenance method and system based on deep learning. Background Art

[0002] The intelligent operation and maintenance of cameras aims to monitor the operating status of cameras in real time, predict faults, and perform efficient maintenance. Traditional camera operation and maintenance methods mainly rely on manual inspections and expert experience for fault diagnosis, which has many disadvantages. First, fault diagnosis is highly dependent on expert experience. Limited by the individual knowledge level and experience accumulation of experts, the accuracy and efficiency of the diagnosis results are often not guaranteed. Secondly, the traditional work order processing process is cumbersome and inefficient. From fault discovery to work order generation to maintenance processing, the entire process takes a long time, which seriously affects the fault response speed and operation and maintenance efficiency. Therefore, there is an urgent need for a new method that can intelligently predict camera faults and automatically generate corresponding work orders to improve the intelligence level and response speed of operation and maintenance. Summary of the invention

[0003] In view of this, the present invention aims to provide a camera intelligent operation and maintenance method and system based on deep learning, so as to solve the problem that fault location of traditional camera operation and maintenance methods relies on manual experience and the work order generation efficiency is low.

[0004] A camera intelligent operation and maintenance method based on deep learning, comprising:

[0005] S1: Collect historical camera fault data and perform preprocessing to obtain preprocessed historical camera fault data;

[0006] S2: According to the pre-processed camera historical fault data, the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause are counted, and the historical fault type feature vector and the historical fault cause feature vector are extracted, including:

[0007] First, based on the pre-processed camera historical fault data, the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause are counted;

[0008] Secondly, the preprocessed camera historical fault data is grouped with the historical fault type as the index, and the time decay factor is calculated for each group of preprocessed camera historical fault data in combination with the maintenance start time, and feature extraction is performed to obtain the historical fault type feature vector;

[0009] Finally, for each set of preprocessed camera historical fault data, the mutual information method is used to calculate the mutual information matrix of fault causes and fault types, and the feature vector of historical fault causes is calculated by combining the number of occurrences of each historical fault cause.

[0010] S3: For the preprocessed camera historical fault data, combine the historical fault type feature vector and the historical fault cause feature vector to generate the camera operation and maintenance text and perform data enhancement, and then extract the historical fault description feature vector, the historical maintenance measure description feature vector, the first relationship feature vector and the second relationship feature vector;

[0011] S4: collecting the current fault data of the camera and extracting the current fault description feature vector; then combining the first relationship feature vector to calculate the current fault type, the first fault cause and the second fault cause;

[0012] S5: Generate maintenance suggestions and work orders based on the current fault type, the first fault cause, the second fault cause, and the second relationship feature vector, and select maintenance personnel to dispatch the work orders.

[0013] Furthermore, the S1 step further includes:

[0014] S11: Collect historical fault data of the camera, each piece of historical fault data of the camera includes camera number, installation location, historical fault type, historical fault cause, fault duration, historical fault description, work order number, maintenance personnel, maintenance start time, maintenance duration, and maintenance measure description;

[0015] S12: For the camera historical fault data, delete the camera historical fault data with a fault duration of less than 5 seconds, and use a word segmentation tool to delete stop words in the historical fault description and the repair measure description to obtain the preprocessed camera historical fault data.

[0016] Furthermore, the S2 step further includes:

[0017] S21: Counting the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause according to the pre-processed historical fault data of the camera;

[0018] S22: Using the historical fault type as an index, grouping the preprocessed camera historical fault data, and extracting features from each group of preprocessed camera historical fault data based on the number of occurrences of each historical fault type, to obtain a historical fault type feature vector, which is calculated as follows:

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] in, is the preliminary feature vector of the i-th historical fault type, i is the first index of the historical fault type, MLP( ) is a multi-layer perceptron, data is the historical fault data of each group of preprocessed cameras, sim i,j is the similarity between the preliminary feature vectors of the i-th and j-th historical fault types, cossim() is the cosine similarity, is the preliminary feature vector of the jth historical fault type, is the time attenuation factor of the jth historical fault type, j is the second index of the historical fault type, i≠j, is the number of occurrences of the jth historical fault type, e is a natural constant, λ is the attenuation coefficient, t is the time difference between the start time of maintenance and the current time, is the feature vector of the i-th historical fault type, and m is the number of historical fault types;

[0024] S23: For each set of pre-processed camera historical fault data, combined with the number of occurrences of each historical fault cause, a historical fault cause feature vector is calculated in the following manner:

[0025] ;

[0026] ;

[0027] ;

[0028] in, For the The preliminary feature vector of historical fault causes, An index of historical fault causes. For the The historical fault cause feature vectors are Concat( ) and CNN( ) respectively. A is the mutual information matrix between the historical fault causes and the historical fault types. For the The number of times the historical fault cause occurred, For mutual information calculation, reason and type are the historical fault cause and historical fault type in the preprocessed camera historical fault data, respectively.

[0029] Traditional methods usually only consider the number of occurrences of fault types or simple statistical features, while ignoring the similarity between fault types and the time decay effect; step S22 of the present invention uses a multi-layer perceptron to perform preliminary feature extraction on each group of pre-processed camera historical fault data, and then calculates the cosine similarity between these preliminary feature vectors, and combines the time decay factor to perform weighted summation on the feature vectors, thereby obtaining the final historical fault type feature vector; the introduction of the time decay factor makes the fault type that has occurred recently occupy a larger weight in the feature vector, which is more in line with the characteristics of actual operation and maintenance;

[0030] Step S23 of the present invention reflects the degree of correlation between the cause of the fault and the type of the fault by calculating the mutual information matrix between the historical fault causes and the historical fault types; then, the mutual information matrix is ​​used as the input of the convolutional neural network, and the local perception and weight sharing characteristics of the convolutional neural network are used to extract the high-order features of the cause of the fault; finally, these high-order features are spliced ​​with the preliminary feature vector and the number of occurrences of the cause of the fault, and further feature fusion is performed through a multi-layer perceptron to obtain the final feature vector of the historical cause of the fault, which provides more accurate and comprehensive information for subsequent fault location, prediction and work order generation.

[0031] Furthermore, the S3 step further includes:

[0032] S31: Generate camera operation and maintenance text by combining the preprocessed camera historical fault data with the historical fault type feature vector and the historical fault cause feature vector. The calculation method is:

[0033] ;

[0034] Among them, text k is the camera operation and maintenance text of the kth preprocessed camera historical fault data, k is the data index, For the improved Transformer block, data k is the kth preprocessed camera historical fault data; the improved Transformer block uses two encoders to encode the historical fault type feature vector and the historical fault cause feature vector respectively, each encoder includes a self-attention mechanism and a feedforward neural network, and the input of each encoder is data k In addition, it also includes the output of another encoder;

[0035] S32: introducing a semantic enhancement module to perform data enhancement on the camera operation and maintenance text to obtain an enhanced camera operation and maintenance text, wherein the data enhancement method includes synonym replacement and sentence transformation, wherein the sentence transformation includes active-passive transformation, affirmative-negative transformation, and causal transformation;

[0036] S33: For the enhanced camera operation and maintenance text, extract the historical fault description feature vector and the historical maintenance measure description feature vector, and the calculation method is:

[0037] ;

[0038] in, , are the historical fault description feature vector and the historical maintenance measure description feature vector respectively, The k-th enhanced camera operation and maintenance text;

[0039] S34: extracting the first relationship feature vector between the historical fault description feature vector, the historical fault type feature vector and the historical fault cause feature vector, and calculating the first relationship feature vector as follows:

[0040] ;

[0041] ;

[0042] in, , , are the description weight, type first weight and cause first weight respectively, sigmoid() is the sigmoid function, r 1 is the first relation eigenvector;

[0043] S35: extracting the second relationship feature vector between the historical fault type feature vector, the historical fault cause feature vector and the historical maintenance measure description feature vector, and calculating the second relationship feature vector as follows:

[0044] ;

[0045] ;

[0046] in, , , , respectively, the second weight of type, the second weight of cause and the maintenance weight, r 2 is the second relation eigenvector.

[0047] The S3 step of the present invention introduces the idea of ​​natural language processing, uses the pre-processed historical fault data of the camera to generate operation and maintenance text, and uses the Transformer model's reasoning ability for semantic information to extract historical fault description feature vectors and historical maintenance measures description feature vectors. The Transformer model is a deep learning architecture proposed by Vaswani et al. in 2017; the improved Transformer block uses two independent encoders. This dual encoding mechanism enables the model to more carefully capture and distinguish the subtle differences between fault types and fault causes; in addition, each encoder introduces the output of another encoder as additional input information, so that the two encoders can exchange and share information, thereby enhancing the model's ability to understand the correlation between fault data and fault types and causes; finally, the present invention calculates the first relationship feature vector and the second relationship feature vector, comprehensively considers various historical fault handling experiences, and realizes in-depth mining and analysis of the camera's historical fault data, providing strong support for subsequent fault prediction, diagnosis and maintenance.

[0048] Furthermore, the S4 step further includes:

[0049] S41: Collect the current fault data of the camera, including the camera number, installation location and fault description; and extract the current fault description feature vector, which is calculated as follows:

[0050] ;

[0051] in, is the feature vector describing the current fault, data now It is the current fault data of the camera;

[0052] S42: Based on the first relationship feature vector and the current fault description feature vector, the fault type and fault cause are inferred, and the probability distribution of the current fault type and the current fault cause is output. The calculation method is:

[0053] ;

[0054] ;

[0055] Among them, p 1 With p 2 are the probability distributions of the current fault type and the current fault cause, respectively; softmax() is the softmax function operation; and W is the fault cause weight matrix;

[0056] S43: Take the fault type with the highest probability in the current fault type probability distribution as the current fault type; take the two fault types with the highest probability in the probability distribution of the current fault cause, and record them as the first fault cause and the second fault cause respectively in descending order of probability.

[0057] The present invention combines the extracted first relationship feature vector to locate the fault type and infer the cause, wherein the fault cause weight matrix W is used to adjust the relative importance between different fault causes, so that the model pays more attention to those fault causes that are more common or have a greater impact in historical data.

[0058] Furthermore, the step S5 further includes:

[0059] S51: Generate maintenance suggestions according to the current fault type, the first fault cause, the second fault cause, and the second relationship feature vector. The calculation method is:

[0060] ;

[0061] Among them, text maintenance is the maintenance suggestion, B is the current fault type, C is 1 The first fault cause is C 2 The second fault cause;

[0062] S52: Generate a work order based on the current fault type, the first fault cause and the maintenance suggestion. The calculation method is:

[0063] ;

[0064] Among them, table is the work order;

[0065] S53: For the current fault type, the frequency of occurrence of maintenance personnel is counted and sorted in descending order, and the maintenance personnel with the highest frequency of occurrence is selected as the maintenance personnel of the work order, and the work order is dispatched.

[0066] The S5 step of the present invention combines the second relationship feature vector to generate maintenance suggestions; the second fault cause introduced in the maintenance suggestion is used to generate alternative solutions in the maintenance suggestion, informing the maintenance personnel of the possible causes of the fault, thereby increasing the flexibility and comprehensiveness of the maintenance suggestion, so that the maintenance personnel can have more response plans when facing complex faults; in addition, the present invention integrates the current fault type, the first fault cause and the maintenance suggestion, and uses a multi-layer perceptron and a dual Transformer model to realize the automatic generation of work orders, which helps the maintenance personnel to quickly understand the fault situation and prepare corresponding maintenance tools and materials.

[0067] The present invention also discloses a camera intelligent operation and maintenance system based on deep learning, comprising:

[0068] Camera historical fault data collection and preprocessing module: collects camera historical fault data and preprocesses it to obtain preprocessed camera historical fault data;

[0069] Historical fault type and fault cause analysis module: based on the pre-processed camera historical fault data, the module counts the number of occurrences of each historical fault type and each historical fault cause, and extracts the historical fault type feature vector and the historical fault cause feature vector;

[0070] Relationship feature vector extraction module: for the preprocessed camera historical fault data, the historical fault type feature vector and the historical fault cause feature vector are combined to generate the camera operation and maintenance text and perform data enhancement, and then extract the historical fault description feature vector, the historical maintenance measure description feature vector, the first relationship feature vector and the second relationship feature vector;

[0071] Current fault analysis module: collects the current fault data of the camera and extracts the current fault description feature vector; then combines the first relationship feature vector to calculate the current fault type, the first fault cause and the second fault cause;

[0072] Work order generation and dispatching module: Generates maintenance suggestions and work orders based on the current fault type and the first fault cause, the second fault cause, and the second relationship feature vector, and selects maintenance personnel to dispatch the work orders.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] (1) The present invention first uses a deep learning algorithm to automatically extract key information such as historical fault type feature vectors and historical fault cause feature vectors; secondly, the present invention combines natural language processing technology to enhance the reasoning ability of fault types and fault causes, and enriches the data; finally, based on the improved Transformer block, the present invention automatically locates the current fault of the camera and generates maintenance suggestions and work orders, realizing the automation and intelligence of the operation and maintenance process and greatly improving the efficiency of work order generation.

[0075] (2) In terms of algorithm, the present invention constructs an improved Transformer block and adopts two independent encoders, so that the model can capture and distinguish the subtle differences between fault types and fault causes in a more detailed manner; in addition, each encoder introduces the output of the other encoder as additional input information, so that the two encoders can exchange and share information, thereby enhancing the model's ability to understand the correlation between fault data and fault types and causes.

[0076] (3) In terms of algorithms, the present invention combines the time decay factor to calculate the historical fault type feature vector, so that the fault type that occurred recently has a larger weight in the feature vector; combines the mutual information matrix with the convolutional neural network to calculate the historical fault cause feature vector, which provides more accurate and comprehensive information for subsequent fault location, prediction and work order generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A flowchart of a camera intelligent operation and maintenance method based on deep learning provided by the present invention;

[0078] Figure 2 A schematic diagram of the algorithm flow of the improved Transformer block provided by the present invention;

[0079] Figure 3 A schematic diagram of the algorithm flow for generating a work order provided by the present invention. DETAILED DESCRIPTION

[0080] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0081] Embodiment 1: A camera intelligent operation and maintenance method based on deep learning, such as Figure 1 As shown, the following steps are included:

[0082] S1: Collect historical camera fault data and perform preprocessing to obtain preprocessed historical camera fault data;

[0083] S11: Collect historical fault data of the camera, each piece of historical fault data of the camera includes camera number, installation location, historical fault type, historical fault cause, fault duration, historical fault description, work order number, maintenance personnel, maintenance start time, maintenance duration, and maintenance measure description;

[0084] S12: For the camera historical fault data, delete the camera historical fault data with a fault duration of less than 5 seconds, and use a word segmentation tool to delete stop words in the historical fault description and the repair measure description to obtain the preprocessed camera historical fault data.

[0085] S2: According to the pre-processed camera historical fault data, count the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause, and extract the historical fault type feature vector and the historical fault cause feature vector;

[0086] S21: Counting the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause according to the pre-processed historical fault data of the camera;

[0087] S22: Using the historical fault type as an index, grouping the preprocessed camera historical fault data, and extracting features from each group of preprocessed camera historical fault data based on the number of occurrences of each historical fault type, to obtain a historical fault type feature vector, which is calculated as follows:

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] in, is the preliminary feature vector of the i-th historical fault type, i is the first index of the historical fault type, MLP( ) is a multi-layer perceptron, data is the historical fault data of each group of preprocessed cameras, sim i,j is the similarity between the preliminary feature vectors of the i-th and j-th historical fault types, cossim() is the cosine similarity, is the preliminary feature vector of the jth historical fault type, is the time attenuation factor of the jth historical fault type, j is the second index of the historical fault type, i≠j, is the number of occurrences of the jth historical fault type, e is a natural constant, λ is the attenuation coefficient, t is the time difference between the start time of maintenance and the current time, is the feature vector of the i-th historical fault type, and m is the number of historical fault types;

[0093] S23: For each set of pre-processed camera historical fault data, combined with the number of occurrences of each historical fault cause, a historical fault cause feature vector is calculated in the following manner:

[0094] ;

[0095] ;

[0096] ;

[0097] in, For the The preliminary feature vector of historical fault causes, An index of historical fault causes. For the The historical fault cause feature vectors are Concat( ) and CNN( ) respectively. A is the mutual information matrix between the historical fault causes and the historical fault types. For the The number of times the historical fault cause occurred, For mutual information calculation, reason and type are the historical fault cause and historical fault type in the preprocessed camera historical fault data, respectively.

[0098] Traditional methods usually only consider the number of occurrences of fault types or simple statistical features, while ignoring the similarity between fault types and the time decay effect; Step S22 of the present invention uses a multi-layer perceptron (MLP) to perform preliminary feature extraction on each set of pre-processed camera historical fault data, and then calculates the cosine similarity between these preliminary feature vectors, and combines the time decay factor to perform weighted summation on the feature vectors, thereby obtaining the final historical fault type feature vector; The introduction of the time decay factor makes the fault type that has occurred recently occupy a larger weight in the feature vector, which is more in line with the characteristics of actual operation and maintenance;

[0099] Step S23 of the present invention reflects the degree of correlation between the cause of the fault and the type of the fault by calculating the mutual information matrix between the historical fault causes and the historical fault types; then, the mutual information matrix is ​​used as the input of the convolutional neural network, and the local perception and weight sharing characteristics of the convolutional neural network are used to extract the high-order features of the cause of the fault; finally, these high-order features are spliced ​​with the preliminary feature vector and the number of occurrences of the cause of the fault, and further feature fusion is performed through a multi-layer perceptron to obtain the final feature vector of the historical cause of the fault, which provides more accurate and comprehensive information for subsequent fault location, prediction and work order generation.

[0100] S3: For the preprocessed camera historical fault data, combine the historical fault type feature vector and the historical fault cause feature vector to generate the camera operation and maintenance text and perform data enhancement, and then extract the historical fault description feature vector, the historical maintenance measure description feature vector, the first relationship feature vector and the second relationship feature vector;

[0101] S31: Generate camera operation and maintenance text by combining the preprocessed camera historical fault data with the historical fault type feature vector and the historical fault cause feature vector. The calculation method is:

[0102] ;

[0103] Among them, text k is the camera operation and maintenance text of the kth preprocessed camera historical fault data, k is the data index, For the improved Transformer block, data k is the kth preprocessed camera historical fault data; the improved Transformer block uses two encoders to encode the historical fault type feature vector and the historical fault cause feature vector respectively, each encoder includes a self-attention mechanism and a feedforward neural network, and the input of each encoder is data k In addition, it also includes the output of another encoder;

[0104] For example, the generated camera operation and maintenance text is "The camera has a fuzzy image failure. After inspection, it was found that this was caused by lens contamination. The failure lasted for about 2 hours."

[0105] S32: introducing a semantic enhancement module to perform data enhancement on the camera operation and maintenance text to obtain an enhanced camera operation and maintenance text, wherein the data enhancement method includes synonym replacement and sentence transformation, and the sentence transformation includes active-passive transformation, affirmative-negative transformation, and causal transformation;

[0106] For example:

[0107] Original sentence (active sentence): "The operation and maintenance personnel checked the connection status of the camera."

[0108] After transformation (passive sentence): "The connection status of the camera was checked by the operation and maintenance personnel."

[0109] Original sentence (affirmative sentence): "The cause of the failure is poor contact of the camera power cord."

[0110] After transformation (negative sentence): "The cause of the fault is not the poor contact of the camera power cord."

[0111] Original sentence (reason first): "Because the camera was eroded by rain, the image was blurred."

[0112] After the transformation (reasons below): "The camera was eroded by rain. Therefore, the image is blurred."

[0113] S33: For the enhanced camera operation and maintenance text, extract the historical fault description feature vector and the historical maintenance measure description feature vector, and the calculation method is:

[0114] ;

[0115] in, , are the historical fault description feature vector and the historical maintenance measure description feature vector respectively, The k-th enhanced camera operation and maintenance text;

[0116] S34: extracting the first relationship feature vector between the historical fault description feature vector, the historical fault type feature vector and the historical fault cause feature vector, and calculating the first relationship feature vector as follows:

[0117] ;

[0118] ;

[0119] in, , , are the description weight, type first weight and cause first weight respectively, sigmoid() is the sigmoid function, r 1 is the first relation eigenvector;

[0120] S35: extracting the second relationship feature vector between the historical fault type feature vector, the historical fault cause feature vector and the historical maintenance measure description feature vector, and calculating the second relationship feature vector as follows:

[0121] ;

[0122] ;

[0123] in, , , , respectively, the second weight of type, the second weight of cause and the maintenance weight, r 2 is the second relation eigenvector.

[0124] The S3 step of the present invention introduces the idea of ​​natural language processing, uses the pre-processed historical fault data of the camera to generate operation and maintenance text, and uses the Transformer model's reasoning ability for semantic information to extract historical fault description feature vectors and historical maintenance measures description feature vectors. The Transformer model is a deep learning architecture proposed by Vaswani et al. in 2017; the improved Transformer block uses two independent encoders. This dual encoding mechanism enables the model to more carefully capture and distinguish the subtle differences between fault types and fault causes; in addition, each encoder introduces the output of another encoder as additional input information, so that the two encoders can exchange and share information, thereby enhancing the model's ability to understand the correlation between fault data and fault types and causes; finally, the present invention calculates the first relationship feature vector and the second relationship feature vector, comprehensively considers various historical fault handling experiences, and realizes in-depth mining and analysis of the camera's historical fault data, providing strong support for subsequent fault prediction, diagnosis and maintenance.

[0125] S4: collecting the current fault data of the camera and extracting the current fault description feature vector; then combining the first relationship feature vector to calculate the current fault type, the first fault cause and the second fault cause;

[0126] S41: Collect the current fault data of the camera, including the camera number, installation location and fault description; and extract the current fault description feature vector, which is calculated as follows:

[0127] ;

[0128] in, is the feature vector describing the current fault, data now It is the current fault data of the camera;

[0129] S42: Based on the first relationship feature vector and the current fault description feature vector, the fault type and fault cause are inferred, and the probability distribution of the current fault type and the current fault cause is output. The calculation method is:

[0130] ;

[0131] ;

[0132] Among them, p 1 With p 2 are the probability distributions of the current fault type and the current fault cause, respectively; softmax() is the softmax function operation; and W is the fault cause weight matrix;

[0133] S43: Take the fault type with the highest probability in the current fault type probability distribution as the current fault type; take the two fault types with the highest probability in the probability distribution of the current fault cause, and record them as the first fault cause and the second fault cause respectively in descending order of probability.

[0134] The present invention combines the extracted first relationship feature vector to locate the fault type and infer the cause, wherein the fault cause weight matrix W is used to adjust the relative importance between different fault causes, so that the model pays more attention to those fault causes that are more common or have a greater impact in historical data.

[0135] S5: Generate maintenance suggestions and work orders based on the current fault type, the first fault cause, the second fault cause, and the second relationship feature vector, and select maintenance personnel to dispatch the work orders;

[0136] The work order generation algorithm process is as follows Figure 3 As shown, the following steps are included:

[0137] S51: Generate maintenance suggestions according to the current fault type, the first fault cause, the second fault cause, and the second relationship feature vector. The calculation method is:

[0138] ;

[0139] Among them, text maintenance is the maintenance suggestion, B is the current fault type, C is 1 The first fault cause is C 2 The second fault cause;

[0140] S52: Generate a work order based on the current fault type, the first fault cause and the maintenance suggestion. The calculation method is:

[0141] ;

[0142] Among them, table is the work order;

[0143] S53: For the current fault type, the frequency of occurrence of maintenance personnel is counted and sorted in descending order, and the maintenance personnel with the highest frequency of occurrence is selected as the maintenance personnel of the work order, and the work order is dispatched.

[0144] For example, the generated work order is:

[0145] Work Order Number: WD20230401001

[0146] Camera ID: CAM123456

[0147] Installation location: Entrance on the first floor of XX Building

[0148] Fault type: Blurred image (B)

[0149] Cause of failure: The first cause of failure: lens contamination, the second cause of failure: improper focus adjustment

[0150] Fault start time: 2024-01-01 12:00

[0151] Fault description:

[0152] Camera CAM123456 has been experiencing image blur since 8:30 on April 1, 2023. There may be stains on the lens surface and the focus adjustment may be inaccurate, resulting in unclear surveillance images.

[0153] Maintenance personnel: Zhang Wei (selected based on historical maintenance frequency)

[0154] Repair suggestions: (generated by Transformer_double model):

[0155] It is recommended to first use a professional lens cleaner to clean the lens surface to ensure that there is no stain left; then adjust the camera focus to the best state to restore a clear monitoring image. If the problem persists, please further check the internal circuit and connection of the camera.

[0156] In step S5 of the present invention, the second relationship feature vector is combined to generate maintenance suggestions; the second fault cause introduced in the maintenance suggestion is used to generate alternative solutions in the maintenance suggestion, informing maintenance personnel of what other causes the fault may be, thereby increasing the flexibility and comprehensiveness of the maintenance suggestion, so that maintenance personnel can have more coping plans when facing complex faults; in addition, the present invention integrates the current fault type, the first fault cause and the maintenance suggestion, and uses a multi-layer perceptron (MLP) and a dual Transformer model to realize the automatic generation of work orders, which helps maintenance personnel quickly understand the fault situation and prepare corresponding maintenance tools and materials;

[0157] In particular, the following steps may be selected to replace step S53:

[0158] For the current fault type, the occurrence frequency of maintenance personnel is counted and sorted in descending order to obtain the maintenance personnel sequence. Then, maintenance personnel are selected and work orders are dispatched in the following sampling method. The sampling steps are as follows:

[0159]

[0160]

[0161] in, is the sampling probability threshold, GRU ( ) is the gated recurrent unit, D is the maintenance personnel sequence, and P is the maintenance personnel probability sequence;

[0162] In the maintenance personnel probability sequence P, select the sum of probabilities less than or equal to the sampling probability threshold The maintenance personnel are taken as the candidate sequence; then a maintenance personnel is randomly selected from the candidate sequence to assign a work order.

[0163] Embodiment 2: The present invention also discloses a camera intelligent operation and maintenance system based on deep learning, comprising:

[0164] Camera historical fault data collection and preprocessing module: collects camera historical fault data and preprocesses it to obtain preprocessed camera historical fault data;

[0165] Historical fault type and fault cause analysis module: based on the pre-processed camera historical fault data, the module counts the number of occurrences of each historical fault type and each historical fault cause, and extracts the historical fault type feature vector and the historical fault cause feature vector;

[0166] Relationship feature vector extraction module: for the preprocessed camera historical fault data, the historical fault type feature vector and the historical fault cause feature vector are combined to generate the camera operation and maintenance text and perform data enhancement, and then extract the historical fault description feature vector, the historical maintenance measure description feature vector, the first relationship feature vector and the second relationship feature vector;

[0167] Current fault analysis module: collects the current fault data of the camera and extracts the current fault description feature vector; then combines the first relationship feature vector to calculate the current fault type, the first fault cause and the second fault cause;

[0168] Work order generation and dispatching module: Generates maintenance suggestions and work orders based on the current fault type and the first fault cause, the second fault cause, and the second relationship feature vector, and selects maintenance personnel to dispatch the work orders.

[0169] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in multiple embodiments of the present invention.

[0171] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A camera intelligent operation and maintenance method based on deep learning, characterized in that: The following steps are involved: S1: Collect historical camera fault data and perform preprocessing to obtain preprocessed historical camera fault data; S2: According to the pre-processed camera historical fault data, the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause are counted, and the historical fault type feature vector and the historical fault cause feature vector are extracted, including: First, based on the pre-processed camera historical fault data, the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause are counted; Secondly, the preprocessed camera historical fault data is grouped with the historical fault type as the index, and the time decay factor is calculated for each group of preprocessed camera historical fault data in combination with the maintenance start time, and feature extraction is performed to obtain the historical fault type feature vector; Finally, for each set of preprocessed camera historical fault data, the mutual information method is used to calculate the mutual information matrix of fault causes and fault types, and the feature vector of historical fault causes is calculated by combining the number of occurrences of each historical fault cause. S3: For the preprocessed camera historical fault data, combine the historical fault type feature vector and the historical fault cause feature vector to generate the camera operation and maintenance text and perform data enhancement, and then extract the historical fault description feature vector, the historical maintenance measure description feature vector, the first relationship feature vector and the second relationship feature vector; S4: collecting the current fault data of the camera and extracting the current fault description feature vector; then combining the first relationship feature vector to calculate the current fault type, the first fault cause and the second fault cause; S5: Generate maintenance suggestions and work orders based on the current fault type, the first fault cause, the second fault cause, and the second relationship feature vector, and select maintenance personnel to dispatch the work orders.

2. The camera intelligent operation and maintenance method based on deep learning according to claim 1 is characterized in that: The S1 step includes: S11: Collect historical fault data of the camera, each piece of historical fault data of the camera includes: camera number, installation location, historical fault type, historical fault cause, fault duration, historical fault description, work order number, maintenance personnel, maintenance start time, maintenance duration, and maintenance measure description; S12: the camera historical fault data, deletes the camera historical fault data with a fault duration of less than 5 seconds, and uses a word segmentation tool to delete stop words in the historical fault description and the repair measure description to obtain the pre-processed camera historical fault data.

3. The camera intelligent operation and maintenance method based on deep learning according to claim 1 is characterized in that: The S2 step includes: S21: Counting the number of occurrences of each historical fault type and the number of occurrences of each historical fault cause according to the pre-processed historical fault data of the camera; S22: Group the preprocessed camera historical fault data using the historical fault type as an index, and extract features from each group of preprocessed camera historical fault data based on the number of occurrences of each historical fault type to obtain a historical fault type feature vector, which is calculated as follows: ； ； ； ； in, is the preliminary feature vector of the i-th historical fault type, i is the first index of the historical fault type, MLP( ) is a multi-layer perceptron, data is the historical fault data of each group of preprocessed cameras, sim i,j is the similarity between the preliminary feature vectors of the i-th and j-th historical fault types, cossim() is the cosine similarity, is the preliminary feature vector of the jth historical fault type, is the time attenuation factor of the jth historical fault type, j is the second index of the historical fault type, i≠j, is the number of occurrences of the jth historical fault type, e is a natural constant, λ is the attenuation coefficient, t is the time difference between the start time of maintenance and the current time, is the feature vector of the i-th historical fault type, and m is the number of historical fault types; S23: For each set of pre-processed camera historical fault data, combined with the number of occurrences of each historical fault cause, a historical fault cause feature vector is calculated in the following manner: ； ； ； in, For the The preliminary feature vector of historical fault causes, An index of historical fault causes. For the The historical fault cause feature vectors are Concat( ) and CNN( ) respectively. A is the mutual information matrix between the historical fault causes and the historical fault types. For the The number of times the historical fault cause occurred, For mutual information calculation, reason and type are the historical fault cause and historical fault type in the preprocessed camera historical fault data, respectively.

4. The camera intelligent operation and maintenance method based on deep learning according to claim 3 is characterized in that: The S3 step includes: S31: Generate camera operation and maintenance text by combining the preprocessed camera historical fault data with the historical fault type feature vector and the historical fault cause feature vector. The calculation method is: ； Among them, text k is the camera operation and maintenance text of the kth preprocessed camera historical fault data, k is the data index, For the improved Transformer block, data k is the kth preprocessed camera historical fault data; the improved Transformer block uses two encoders to encode the historical fault type feature vector and the historical fault cause feature vector respectively, each encoder includes a self-attention mechanism and a feedforward neural network, and the input of each encoder is data k In addition, it also includes the output of another encoder; S32: introducing a semantic enhancement module to perform data enhancement on the camera operation and maintenance text to obtain an enhanced camera operation and maintenance text, wherein the data enhancement method includes synonym replacement and sentence transformation, wherein the sentence transformation includes active-passive transformation, affirmative-negative transformation, and causal transformation; S33: For the enhanced camera operation and maintenance text, extract the historical fault description feature vector and the historical maintenance measure description feature vector, and the calculation method is: ； in, , are the historical fault description feature vector and the historical maintenance measure description feature vector respectively, The k-th enhanced camera operation and maintenance text; S34: extracting the first relationship feature vector between the historical fault description feature vector, the historical fault type feature vector and the historical fault cause feature vector, and calculating the first relationship feature vector as follows: ； ； in, , , are the description weight, the first type weight and the first cause weight respectively, sigmoid() is the sigmoid function, and r1 is the first relational eigenvector; S35: extracting the second relationship feature vector between the historical fault type feature vector, the historical fault cause feature vector and the historical maintenance measure description feature vector, and calculating the second relationship feature vector as follows: ； ； in, , , , are the second weight of type, the second weight of cause and the maintenance weight respectively, and r2 is the second relationship feature vector.

5. The camera intelligent operation and maintenance method based on deep learning according to claim 4 is characterized in that: The S4 step includes: S41: Collect the current fault data of the camera, including the camera number, installation location and fault description; and extract the current fault description feature vector, which is calculated as follows: ； in, is the feature vector describing the current fault, data now It is the current fault data of the camera; S42: Based on the first relationship feature vector and the current fault description feature vector, the fault type and fault cause are inferred, and the probability distribution of the current fault type and the current fault cause is output. The calculation method is: ； ； Among them, p1 and p2 are the probability distribution of the current fault type and the current fault cause respectively, softmax() is the softmax function operation, and W is the fault cause weight matrix; S43: Take the fault type with the highest probability in the current fault type probability distribution as the current fault type; take the two fault types with the highest probability in the probability distribution of the current fault cause, and record them as the first fault cause and the second fault cause respectively in descending order of probability.

6. The camera intelligent operation and maintenance method based on deep learning according to claim 5 is characterized in that: The S5 step includes: S51: Generate maintenance suggestions according to the current fault type, the first fault cause, the second fault cause, and the second relationship feature vector. The calculation method is: ； Among them, text maintenance is a maintenance suggestion, B is the current fault type, C1 is the first fault cause, and C2 is the second fault cause; S52: Generate a work order based on the current fault type, the first fault cause and the maintenance suggestion. The calculation method is: ； Among them, table is the work order; S53: For the current fault type, the frequency of occurrence of maintenance personnel is counted and sorted in descending order, and the maintenance personnel with the highest frequency of occurrence is selected as the maintenance personnel of the work order, and the work order is dispatched.

7. A camera intelligent operation and maintenance system based on deep learning, characterized in that: include: Camera historical fault data collection and preprocessing module: collects camera historical fault data and preprocesses it to obtain preprocessed camera historical fault data; Historical fault type and fault cause analysis module: based on the pre-processed camera historical fault data, the module counts the number of occurrences of each historical fault type and each historical fault cause, and extracts the historical fault type feature vector and the historical fault cause feature vector; Relationship feature vector extraction module: for the preprocessed camera historical fault data, the historical fault type feature vector and the historical fault cause feature vector are combined to generate the camera operation and maintenance text and perform data enhancement, and then extract the historical fault description feature vector, the historical maintenance measure description feature vector, the first relationship feature vector and the second relationship feature vector; Current fault analysis module: collects the current fault data of the camera and extracts the current fault description feature vector; then combines the first relationship feature vector to calculate the current fault type, the first fault cause and the second fault cause; Work order generation and dispatching module: Generates maintenance suggestions and work orders based on the current fault type, the first fault cause, the second fault cause, and the second relationship feature vector, and selects maintenance personnel to dispatch the work orders; To realize the camera intelligent operation and maintenance method based on deep learning as described in any one of claims 1-6.

Images