An image processing method and system for a dynamic monitoring instrument

Abstract

The application discloses an image processing method and system for a dynamic monitoring instrument, and relates to the technical field of image processing, and comprises the following steps: exporting image frames in a monitoring window through a target dynamic monitoring instrument to obtain a monitoring image frame sequence; traversing the monitoring image frame sequence to perform image quality scoring and identification, obtaining an identified monitoring image frame sequence, performing semantic conversion analysis and multi-scale feature recognition to obtain a structured description text sequence and an identified multi-scale feature group sequence; performing intra-group feature interaction on the identified multi-scale feature group sequence to determine an identified interactive multi-scale feature sequence; performing double memory iteration on the identified interactive multi-scale feature sequence and the structured description text sequence to determine target iterative memory, and performing identification to determine an image processing result of the monitoring window. The application solves the technical problems of low dynamic image monitoring recognition accuracy and poor reliability in the prior art, and achieves the technical effect of improving abnormal recognition accuracy and stability.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an image processing method and system for a dynamic monitoring instrument. Background Technology

[0002] With the widespread application of intelligent monitoring equipment, dynamic monitoring instruments are playing an increasingly important role in industrial inspection, safety protection and other fields. However, the acquired dynamic images are often affected by factors such as clarity, brightness, exposure and contrast, resulting in unstable image quality. Furthermore, a single frame image is difficult to fully express the deep semantics and multi-scale feature associations in the dynamic change process, which leads to insufficient accuracy and reliability of anomaly identification and makes it difficult to meet the needs of high-precision dynamic monitoring. Summary of the Invention

[0003] This application provides an image processing method and system for a dynamic monitoring instrument, which addresses the technical problems of low accuracy and poor reliability in dynamic image monitoring and recognition in the prior art.

[0004] In view of the above problems, this application provides an image processing method and system for a dynamic monitoring instrument.

[0005] A first aspect of this application provides an image processing method for a dynamic monitoring instrument, the method comprising:

[0006] The image frames within the monitoring window are exported through the SDK driver interface of the target dynamic monitoring instrument to obtain a monitoring image frame sequence. The monitoring image frame sequence is then traversed to perform image quality scoring according to preset scoring indicators, and the results are used to identify the identified monitoring image frame sequence. Semantic transformation analysis and multi-scale feature recognition are performed on the identified monitoring image frame sequence to obtain a structured description text sequence and an identified multi-scale feature group sequence. Intra-group feature interaction is performed based on the identified multi-scale feature group sequence to determine the identified interactive multi-scale feature sequence. Dual-memory iteration is performed on the identified interactive multi-scale feature sequence and the structured description text sequence to determine the target iterative memory. The target iterative memory is then identified to determine the image processing result of the monitoring window.

[0007] A second aspect of this application provides an image processing system for a dynamic monitoring instrument, the system comprising:

[0008] The system comprises the following modules: an image frame export module, used to export image frames within the monitoring window via the SDK driver interface of the target dynamic monitoring instrument, to obtain a monitoring image frame sequence; a scoring module, used to traverse the monitoring image frame sequence, score the image quality according to preset scoring indicators, and identify the images based on the scoring results, to obtain an identified monitoring image frame sequence; an analysis and recognition module, used to traverse the identified monitoring image frame sequence, perform semantic transformation analysis and multi-scale feature recognition, to obtain a structured description text sequence and an identified multi-scale feature group sequence; an intra-group feature interaction module, used to perform intra-group feature interaction based on the identified multi-scale feature group sequence, to determine the identified interactive multi-scale feature sequence; a dual-memory iteration module, used to perform dual-memory iteration on the identified interactive multi-scale feature sequence and the structured description text sequence, to determine the target iterative memory; and a recognition module, used to recognize the target iterative memory, to determine the image processing result of the monitoring window.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0010] This application uses the SDK driver interface of a target dynamic monitoring instrument to export image frames within a monitoring window, obtaining a sequence of monitoring image frames. It then iterates through the monitoring image frame sequence, scoring image quality according to preset scoring indicators, and identifies the frames based on the scoring results, obtaining an identified monitoring image frame sequence. Next, it iterates through the identified monitoring image frame sequence, performing semantic transformation analysis and multi-scale feature recognition to obtain a structured description text sequence and an identified multi-scale feature group sequence. Based on the identified multi-scale feature group sequence, it performs intra-group feature interaction to determine the identified interactive multi-scale feature sequence. Finally, it performs dual-memory iteration on the identified interactive multi-scale feature sequence and the structured description text sequence to determine the target iterative memory. Finally, it identifies the target iterative memory to determine the image processing result of the monitoring window. This invention solves the technical problems of low accuracy and poor reliability in dynamic image monitoring and recognition in existing technologies. By introducing image quality scoring, multi-scale feature interaction, and dual-memory iterative processing, it achieves the technical effect of improving the accuracy and stability of anomaly recognition. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of an image processing method for a dynamic monitoring instrument provided in an embodiment of this application;

[0013] Figure 2This is a schematic diagram of an image processing system for a dynamic monitoring instrument provided in an embodiment of this application.

[0014] Figure labeling: Image frame export module 11, scoring module 12, analysis and recognition module 13, intra-group feature interaction module 14, dual memory iteration module 15, recognition module 16. Detailed Implementation

[0015] This application provides an image processing method and system for dynamic monitoring instruments, which addresses the technical problems of low accuracy and poor reliability in dynamic image monitoring and recognition in the prior art. By introducing image quality scoring, multi-scale feature interaction, and dual memory iterative processing, it achieves the technical effect of improving the accuracy and stability of anomaly recognition.

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0017] It should be noted that any variation of the terms "comprising" and "having" is intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0018] Example 1, as Figure 1 As shown, this application provides an image processing method for a dynamic monitoring instrument, the method comprising:

[0019] Step S100: Export the image frames within the monitoring window through the SDK driver interface of the target dynamic monitoring instrument to obtain the monitoring image frame sequence.

[0020] In this embodiment, the dynamic images within the monitoring window are accessed and invoked through the SDK driver interface of the target dynamic monitoring instrument. The SDK driver interface is a software development kit interface provided by the dynamic monitoring instrument, used to establish a data interaction channel between the device and the application, converting the acquired monitoring screen data into processable image frames. During this process, the dynamic images within the monitoring window are decomposed into consecutive image frames, each corresponding to a specific moment in the monitoring screen, containing complete spatial pixel information and retaining temporal continuity. These image frames are then exported sequentially to form a monitoring image frame sequence.

[0021] Step S200: Traverse the monitoring image frame sequence, score the image quality according to the preset scoring index, and mark it according to the scoring result to obtain the marked monitoring image frame sequence.

[0022] Furthermore, the method provided in the application embodiments also includes:

[0023] The preset scoring indicators include peak signal-to-noise ratio, sharpness, brightness, exposure, and contrast.

[0024] In this embodiment, when traversing the sequence of monitored image frames and scoring image quality according to preset scoring indicators, quality analysis is performed independently for each frame. The preset scoring indicators include peak signal-to-noise ratio (PSNR), sharpness, brightness, exposure, and contrast. When scoring the image quality of each frame, the PSNR is first calculated using mean squared error; then, an edge detection operator, such as the Sobel operator, is used to extract gradient information and calculate gradient energy to obtain sharpness; simultaneously, the pixel values ​​of the image brightness channel are averaged to obtain brightness; exposure is extracted by statistically analyzing the proportion of pixels with grayscale values ​​in the overly dark and overly bright ranges; finally, the overall image contrast is calculated based on the standard deviation or grayscale variance of the brightness distribution.

[0025] After obtaining the raw values ​​of the above five indicators, they are each normalized to map them to the same numerical range. Then, the normalized indicators are weighted and summed according to preset weights to obtain the image quality score for that frame. Finally, the corresponding image frames are labeled based on the quality scores, resulting in a sequence of labeled monitoring image frames with quality score tags.

[0026] Step S300: Traverse the sequence of monitored image frames to perform semantic transformation analysis and multi-scale feature recognition, and obtain a structured description text sequence and a multi-scale feature group sequence of the identifier.

[0027] In this embodiment, when traversing the sequence of identification monitoring image frames, a semantic converter is first invoked to perform semantic transformation analysis on each image frame, mapping low-level pixel information to high-level semantic expressions to obtain a structured descriptive text sequence reflecting the semantic features of the scene and the target. Subsequently, a multi-scale set is obtained, and a multi-scale feature analyzer set is constructed based on this set. Finally, the multi-scale feature analyzer set is used to perform multi-scale feature recognition on the identification monitoring image frame sequence to obtain a multi-scale feature group sequence for the identification.

[0028] Furthermore, the method provided in the application embodiment, which involves traversing the sequence of identified monitoring image frames to perform semantic transformation analysis and multi-scale feature recognition to obtain a structured description text sequence and an identified multi-scale feature group sequence, also includes:

[0029] The semantic converter is invoked to perform semantic conversion analysis on the identification monitoring image frame sequence to obtain a structured description text sequence; a multi-scale set is obtained, and a multi-scale feature analyzer set is constructed; the multi-scale feature analyzer set is used to perform multi-scale feature recognition on the identification monitoring image frame sequence to obtain the identification multi-scale feature group sequence.

[0030] In this embodiment, during semantic transformation analysis, a semantic converter is invoked to process the image frame sequence for identification monitoring frame by frame. The semantic converter employs an encoder-decoder structure combining a convolutional neural network and a Transformer. During training, a large-scale image-text pairing dataset is used, and the difference between the model's predicted text and the actual labeled text is optimized through a cross-entropy loss function, thereby obtaining a semantic mapping model with cross-modal representation capabilities. The image-text pairing dataset consists of tens of thousands of images and their corresponding text descriptions, with each image accompanied by natural language annotations. The semantic converter performs convolutional feature extraction on the input image frames and then captures global contextual relationships through a multi-head self-attention mechanism, outputting a structured descriptive text sequence.

[0031] To obtain the multi-scale set, the historical image anomaly log set of the dynamic monitoring instrument is first obtained. This set is then divided into multiple partitioned historical image anomaly log sets based on anomaly type. Subsequently, the anomaly interval duration within each partitioned historical image anomaly log set is extracted, resulting in multiple partitioned anomaly interval duration sets. These partitioned anomaly interval duration sets are then filtered to ultimately obtain the multi-scale set.

[0032] Finally, when performing multi-scale feature recognition on the identification monitoring image frame sequence using a multi-scale feature analyzer set, convolutional feature extraction is performed on each frame of the identification monitoring image frame sequence using the constructed multi-scale feature analyzer set. Pooling and normalization operations are then performed on the feature maps output at each scale to obtain feature vector groups at different scales. Subsequently, the feature vectors at each scale are combined through feature concatenation to form an identification multi-scale feature group sequence. This identification multi-scale feature group sequence reflects the feature differences of the same frame image at different spatial scales.

[0033] Furthermore, the method provided in the application embodiments, in obtaining the multi-scale set and constructing the multi-scale feature analyzer set, further includes:

[0034] Obtain the historical image anomaly log set of the dynamic monitoring instrument; classify the historical image anomaly log set according to the anomaly type to obtain multiple partitioned historical image anomaly log sets; extract the anomaly interval duration within the logs of the multiple partitioned historical image anomaly log sets to obtain multiple partitioned anomaly interval duration sets; filter based on the multiple partitioned anomaly interval duration sets to obtain the multi-scale set.

[0035] In this embodiment, a pre-stored set of historical image anomaly logs from the dynamic monitoring instrument is first obtained from the historical database. The set of historical image anomaly logs is a dataset formed by the dynamic monitoring instrument recording abnormal events detected during operation. Each log entry contains a timestamp, anomaly type, and index information of the corresponding image frame.

[0036] Subsequently, the historical image anomaly log sets are divided into similar categories according to the anomaly type. Based on the anomaly type field in the historical image anomaly logs, events of the same type are grouped into the same subset, thus forming multiple sets of historical image anomaly logs.

[0037] Next, the abnormal interval duration within the logs is extracted from multiple sets of historical image abnormal logs. That is, the events are sorted in the order of timestamps within each set, and the interval duration between two consecutive abnormalities is calculated by the difference between adjacent timestamps, thus obtaining multiple sets of abnormal interval durations.

[0038] Finally, filtering is performed based on multiple sets of anomaly interval durations. This process begins by extracting the maximum and minimum anomaly interval durations from each set, resulting in multiple maximum and minimum anomaly interval durations. Subsequently, the union of these maximum and minimum anomaly interval durations is calculated to obtain a multi-scale set.

[0039] Furthermore, in the method provided in the application embodiments, the multi-scale set is obtained by filtering based on the plurality of sets of abnormal interval durations, and further includes:

[0040] Extract the maximum and minimum partitioning abnormal interval durations from the multiple partitioning abnormal interval duration sets to obtain multiple maximum and minimum partitioning abnormal interval durations; perform a union of the multiple maximum and minimum partitioning abnormal interval durations to obtain the multi-scale set.

[0041] In this embodiment, when extracting the maximum and minimum abnormal interval durations, the adjacent timestamp difference method is used to process each set of abnormal interval durations. First, the event sequences of the same abnormal type are arranged in ascending order of timestamps. The difference between the timestamps of adjacent events is calculated to form an abnormal interval duration sequence in the log. Extreme value extraction is performed on this sequence. The maximum value is taken as the maximum abnormal interval duration, and the minimum value is taken as the minimum abnormal interval duration. By performing the above process on all abnormal types one by one, multiple maximum abnormal interval durations and multiple minimum abnormal interval durations are accumulated.

[0042] Next, the union of the above-mentioned maximum and minimum anomaly interval durations is obtained. The sets of maximum and minimum anomaly interval durations from each anomaly type are merged, and deduplication and sorting are performed using set semantics. The output is a multi-scale set.

[0043] Step S400: Perform intra-group feature interaction based on the identified multi-scale feature group sequence to determine the identified interaction multi-scale feature sequence.

[0044] In this embodiment, when performing intra-group feature interaction based on the multi-scale feature group sequence of the identifier, the intra-group feature mean is first calculated, that is, the element-wise average of each feature vector in the multi-scale feature group sequence of the identifier is calculated to obtain the leading identifier multi-scale feature sequence. Then, based on the leading identifier multi-scale feature sequence, intra-group feature interaction is performed on the corresponding feature groups in the identifier multi-scale feature group sequence to generate the identifier intra-group interaction multi-scale feature group sequence. Finally, intra-group mean analysis is performed on the identifier intra-group interaction multi-scale feature group sequence, and the interacted features are aggregated and normalized within the group to obtain the identifier interaction multi-scale feature sequence.

[0045] Furthermore, in the method provided in the application embodiments, determining the identifier interaction multi-scale feature sequence by performing intra-group feature interaction based on the identifier multi-scale feature group sequence further includes:

[0046] Calculate the mean of the features within each group of the multi-scale feature group sequence of the identifier to obtain the leading identifier multi-scale feature sequence; based on the leading identifier multi-scale feature sequence, perform intra-group feature interaction on the corresponding identifier multi-scale feature groups in the multi-scale feature group sequence of the identifier to obtain the identifier intra-group interaction multi-scale feature group sequence; perform intra-group mean analysis on the identifier intra-group interaction multi-scale feature group sequence to obtain the identifier interaction multi-scale feature sequence.

[0047] In this embodiment, when calculating the mean value of the features within a group of the multi-scale feature group sequence, at the same scale and time position, all sub-feature vectors within the group are added element-wise according to their dimensions, and then divided by the number of sub-features to obtain the mean vector of the group. The mean vector is then normalized to ensure numerical stability. Afterwards, the mean vectors at all scales and time positions are arranged sequentially to form the leading multi-scale feature sequence.

[0048] Next, based on the leading indicator multi-scale feature sequence, intra-group feature interactions are performed on the corresponding indicator multi-scale feature groups in the indicator multi-scale feature group sequence. In this process, the similarity between the leading indicator multi-scale features and the corresponding sub-features of the same type in the feature group is first calculated, and the results are organized into a leading interaction matrix group sequence. Then, using the leading interaction matrix group sequence as a weight reference, convolution interaction operations are performed on the corresponding feature groups in the indicator multi-scale feature group sequence to obtain the intra-group interactive multi-scale feature group sequence.

[0049] Finally, within-group mean analysis was performed on the multi-scale feature sequence of the identifier group interaction. The feature set after each interaction was compressed into a single representation vector through mean pooling, and a normalization method was used to ensure the numerical consistency of each vector across different scales and time dimensions. Finally, the representation vectors at all times and scales were concatenated sequentially to obtain the identifier interaction multi-scale feature sequence.

[0050] Furthermore, in the method provided in the application embodiments, based on the leading identifier multi-scale feature sequence, intra-group feature interactions are performed on the corresponding identifier multi-scale feature groups in the identifier multi-scale feature group sequence to obtain an identifier intra-group interaction multi-scale feature group sequence, which further includes:

[0051] Calculate the similarity between the multi-scale features of the leading identifier and the corresponding sub-features of the multi-scale feature group in the identifier multi-scale feature group sequence, and construct the leading interaction matrix group sequence based on the calculation results; perform convolution interaction on the corresponding identifier multi-scale feature group in the identifier multi-scale feature group sequence based on the leading interaction matrix group sequence to obtain the interactive multi-scale feature group sequence within the identifier group.

[0052] In this embodiment, when calculating the similarity between the multi-scale features of the leading identifier and the sub-features of the same type within the corresponding multi-scale feature group, the multi-scale feature vectors of the leading identifier and the set of sub-feature vectors of the group are first extracted at the same time position and scale. After L2 normalization of each vector, cosine similarity is used to calculate the similarity value one by one, resulting in a similarity set. Then, Softmax normalization is applied to the similarity set, and the normalized weights are added sequentially to an initially empty matrix according to the index order of the sub-features within the group, obtaining the leading interaction matrix of that group. Finally, by traversing all time positions and scales, these are stacked sequentially to form a sequence of leading interaction matrix groups.

[0053] Next, convolutional interactions are performed on the corresponding multi-scale feature groups of the identifier based on the sequence of the leading interaction matrix. During this process, at each time point and scale, the normalized weights in the leading interaction matrix of the multi-scale feature group are treated as one-dimensional convolution kernels. One-dimensional convolution operations are then performed along the sub-feature vector sequence of the multi-scale feature group to obtain the post-interaction representation vector of the multi-scale feature group. All time points and scales are processed sequentially, and the post-interaction representation vectors of each multi-scale feature group are collected and concatenated in a predetermined order to output the sequence of interactive multi-scale feature groups within the identifier group.

[0054] Step S500: Perform dual memory iteration on the identifier interaction multi-scale feature sequence and the structured description text sequence to determine the target iterative memory.

[0055] In this embodiment, when performing dual-memory iteration on the identifier interaction multi-scale feature sequence and the structured description text sequence, the identifier interaction multi-scale features and structured description text are first extracted sequentially from both as input units, and an initial iterative memory is generated through the dual-memory iteration mechanism. Subsequently, new identifier interaction multi-scale features and structured description text are used to supplement and update the existing iterative memory, forming a stage iterative memory. After the entire sequence traversal is completed, the final identifier interaction multi-scale features and structured description text are added to the stage iterative memory, thereby obtaining the target iterative memory.

[0056] Furthermore, in the method provided in the application embodiments, the method of performing dual memory iteration on the identifier interaction multi-scale feature sequence and the structured description text sequence to determine the target iterative memory further includes:

[0057] Extract a first multi-scale feature of identifier interaction, a first structured description text, a second multi-scale feature of identifier interaction, and a second structured description text from the identifier interaction multi-scale feature sequence and the structured description text sequence; perform dual-memory iteration on the first multi-scale feature of identifier interaction, the first structured description text, the second multi-scale feature of identifier interaction, and the second structured description text to obtain a first iterative memory; use the second multi-scale feature of identifier interaction and the second structured description text to perform dual-memory iteration on the third multi-scale feature of identifier interaction and the third structured description text in the identifier interaction multi-scale feature sequence and the structured description text sequence, and supplement the first iterative memory according to the iteration result to obtain a second iterative memory, and so on, to obtain a stage iterative memory; add the last identifier interaction multi-scale feature and the last structured description text sequence in the identifier interaction multi-scale feature sequence and the structured description text sequence to the stage iterative memory to obtain the target iterative memory.

[0058] In this embodiment of the application, when extracting the first multi-scale feature of the identifier interaction, the first structured description text, the second multi-scale feature of the identifier interaction, and the second structured description text from the identifier interaction multi-scale feature sequence and the structured description text sequence, the first two identifier interaction multi-scale feature vectors of the identifier interaction multi-scale feature sequence are read in chronological order. At the same time, the first two natural language descriptions in the structured description text sequence are converted into vectorized representations through word segmentation and word vector mapping to obtain the first structured description text vector and the second structured description text vector.

[0059] Subsequently, during the dual-memory iteration of the first identifier interaction multi-scale feature, the first structured descriptive text, the second identifier interaction multi-scale feature, and the second structured descriptive text, the first identifier interaction multi-scale feature is first compared with the second identifier interaction multi-scale feature. The time-series differencing method is used to calculate the numerical changes between adjacent frames, extracting the magnitude and direction of change over time to obtain the first feature iteration trend information. Next, the vector representations of the first and second structured descriptive texts are semantically aligned. Cosine similarity is used to calculate the semantic closeness between the texts, and then differencing is combined to extract the semantic change patterns between adjacent texts, obtaining the first semantic iteration trend information. Finally, the first feature iteration trend information and the first semantic iteration trend information are concatenated, and normalization is used to ensure numerical stability. The result is then stored in the first iteration memory.

[0060] Next, during the dual-memory iteration using the second identifier interactive multi-scale features and the second structured descriptive text, along with the third identifier interactive multi-scale features and the third structured descriptive text in the sequence, the third identifier interactive multi-scale features are obtained by continuously reading the next element in the identifier interactive multi-scale feature sequence, and the third structured descriptive text is obtained by continuously reading the next element in the structured descriptive text sequence. For these two newly extracted inputs, the same time-series differencing method and semantic similarity calculation method are first used to extract new feature iteration trend information and semantic iteration trend information, which are then recursively updated with the first iteration memory. This recursive update replaces some outdated trend information and supplements it with new trend information, allowing the memory to continuously reflect new input features and the dynamic changes of the text while retaining existing results, thus outputting the second iteration memory.

[0061] During the traversal of the multi-scale feature sequence and structured descriptive text sequence, each iteration generates a new iterative memory, which is then supplemented and replaced. After most of the features and text in the sequence have been processed sequentially, stage iterative memories are gradually formed through multiple iterations. The stage iterative memories integrate feature iteration trend information and semantic iteration trend information at multiple time points, reflecting the dynamic evolution of the sequence as a whole.

[0062] After sequence traversal is complete, the last identifier interaction multi-scale feature and the last structured description text in the identifier interaction multi-scale feature sequence and structured description text sequence are directly added to the stage iterative memory. At this point, numerical differences are eliminated using a standardization method. The mean and standard deviation of each dimension in the stage iterative memory are calculated, and then all values ​​are subtracted from the mean of the corresponding dimension and divided by the standard deviation, so that the numerical distribution of each dimension is uniformly within the range of zero mean and unit variance. After standardization, the target iterative memory is obtained.

[0063] Furthermore, the method provided in the application embodiments, which performs dual-memory iteration on the first identifier interaction multi-scale feature, the first structured description text, the second identifier interaction multi-scale feature, and the second structured description text to obtain the first iterative memory, also includes:

[0064] Trend analysis is performed on the first identifier interaction multi-scale features to obtain first feature iteration trend information; semantic trend analysis is performed on the first structured description text and the second structured description text to obtain first semantic iteration trend information; the first feature iteration trend information and the first semantic iteration trend information are added to the first iteration memory.

[0065] In this embodiment, when performing trend analysis on the first and second identifier interaction multi-scale features, L2 normalization is first used to unify the numerical scale of the two identifier interaction multi-scale features to eliminate amplitude differences between different dimensions. Then, time series differencing is used to subtract the first identifier interaction multi-scale feature from the second identifier interaction multi-scale feature element by element to obtain feature change vectors at adjacent time points. To reduce noise interference, the feature change vector is smoothed using a mean filter, and then mapped to a zero-mean, unit-variance distribution using Z-score normalization to finally obtain the iterative trend information of the first feature.

[0066] Next, in performing semantic trend analysis on the first and second structured descriptive texts, the text is first segmented into word sequences using word segmentation, and each word is converted into a fixed-dimensional numerical representation using word vector mapping technology. Then, sentence vectors are obtained through mean calculation. Next, the cosine similarity method is used to calculate the closeness of the two text segments in the semantic space, and vector difference is used to obtain the changes in semantic direction. To ensure consistency of results across different dimensions, the semantic difference vectors are L2 normalized, and the cosine similarity is adjusted to a uniform range using interval linear mapping, ultimately generating the first semantic iterative trend information.

[0067] Finally, when adding the first feature iteration trend information and the first semantic iteration trend information into the first iteration memory, the two are first merged into a joint trend representation using a vector concatenation method. Then, a standardization method is used to adjust this joint trend representation to a distribution with zero mean and unit variance to ensure numerical stability in subsequent iterations. Finally, this joint trend representation is stored as the first iteration memory.

[0068] Step S600: Identify the target iterative memory and determine the image processing result of the monitoring window.

[0069] In this embodiment, when identifying the target iterative memory to determine the image processing result of the monitoring window, the numerical features in the target iterative memory are first used as input. A similarity calculation method is then used to compare these features with feature samples of known categories to determine their proximity in the feature space. Subsequently, a threshold determination method is used to filter the results. When the similarity is higher than a set threshold, the target iterative memory is assigned to the corresponding category. If the similarity is within a critical range, the nearest neighbor method is used to find the closest feature vector in the labeled sample set to improve the stability of the determination. Finally, based on the comparison and determination results, the image processing result of the monitoring window is output, reflecting the identification conclusion of the image frame in a specific state.

[0070] In summary, the embodiments of this application have at least the following technical effects:

[0071] This application uses the SDK driver interface of a target dynamic monitoring instrument to export image frames within a monitoring window, obtaining a sequence of monitoring image frames. It then iterates through the monitoring image frame sequence, scoring image quality according to preset scoring indicators, and identifies the frames based on the scoring results, obtaining an identified monitoring image frame sequence. Next, it iterates through the identified monitoring image frame sequence, performing semantic transformation analysis and multi-scale feature recognition to obtain a structured description text sequence and an identified multi-scale feature group sequence. Based on the identified multi-scale feature group sequence, it performs intra-group feature interaction to determine the identified interactive multi-scale feature sequence. Finally, it performs dual-memory iteration on the identified interactive multi-scale feature sequence and the structured description text sequence to determine the target iterative memory. Finally, it identifies the target iterative memory to determine the image processing result of the monitoring window. This invention solves the technical problems of low accuracy and poor reliability in dynamic image monitoring and recognition in existing technologies. By introducing image quality scoring, multi-scale feature interaction, and dual-memory iterative processing, it achieves the technical effect of improving the accuracy and stability of anomaly recognition.

[0072] Example 2, based on the same inventive concept as the image processing method for a dynamic monitoring instrument in the foregoing examples, such as... Figure 2 As shown, this application provides an image processing system for a dynamic monitoring instrument. The system and method embodiments in this application are based on the same inventive concept. The system includes:

[0073] Image frame export module 11 is used to export image frames within the monitoring window through the SDK driver interface of the target dynamic monitoring instrument to obtain a monitoring image frame sequence; scoring module 12 is used to traverse the monitoring image frame sequence to score image quality according to preset scoring indicators, and to mark the results to obtain a marked monitoring image frame sequence; analysis and recognition module 13 is used to traverse the marked monitoring image frame sequence to perform semantic transformation analysis and multi-scale feature recognition to obtain a structured description text sequence and a marked multi-scale feature group sequence; intra-group feature interaction module 14 is used to perform intra-group feature interaction based on the marked multi-scale feature group sequence to determine the marked interaction multi-scale feature sequence; dual memory iteration module 15 is used to perform dual memory iteration on the marked interaction multi-scale feature sequence and the structured description text sequence to determine the target iterative memory; recognition module 16 is used to recognize the target iterative memory to determine the image processing result of the monitoring window.

[0074] Furthermore, the system is also used to implement the following functions:

[0075] The preset scoring indicators include peak signal-to-noise ratio, sharpness, brightness, exposure, and contrast.

[0076] Furthermore, the system is also used to implement the following functions:

[0077] The semantic converter is invoked to perform semantic conversion analysis on the identification monitoring image frame sequence to obtain a structured description text sequence; a multi-scale set is obtained, and a multi-scale feature analyzer set is constructed; the multi-scale feature analyzer set is used to perform multi-scale feature recognition on the identification monitoring image frame sequence to obtain the identification multi-scale feature group sequence.

[0078] Furthermore, the system is also used to implement the following functions:

[0079] Obtain the historical image anomaly log set of the dynamic monitoring instrument; classify the historical image anomaly log set according to the anomaly type to obtain multiple partitioned historical image anomaly log sets; extract the anomaly interval duration within the logs of the multiple partitioned historical image anomaly log sets to obtain multiple partitioned anomaly interval duration sets; filter based on the multiple partitioned anomaly interval duration sets to obtain the multi-scale set.

[0080] Furthermore, the system is also used to implement the following functions:

[0081] Extract the maximum and minimum partitioning abnormal interval durations from the multiple partitioning abnormal interval duration sets to obtain multiple maximum and minimum partitioning abnormal interval durations; perform a union of the multiple maximum and minimum partitioning abnormal interval durations to obtain the multi-scale set.

[0082] Furthermore, the system is also used to implement the following functions:

[0083] Calculate the mean of the features within each group of the multi-scale feature group sequence of the identifier to obtain the leading identifier multi-scale feature sequence; based on the leading identifier multi-scale feature sequence, perform intra-group feature interaction on the corresponding identifier multi-scale feature groups in the multi-scale feature group sequence of the identifier to obtain the identifier intra-group interaction multi-scale feature group sequence; perform intra-group mean analysis on the identifier intra-group interaction multi-scale feature group sequence to obtain the identifier interaction multi-scale feature sequence.

[0084] Furthermore, the system is also used to implement the following functions:

[0085] Calculate the similarity between the multi-scale features of the leading identifier and the corresponding sub-features of the multi-scale feature group in the identifier multi-scale feature group sequence, and construct the leading interaction matrix group sequence based on the calculation results; perform convolution interaction on the corresponding identifier multi-scale feature group in the identifier multi-scale feature group sequence based on the leading interaction matrix group sequence to obtain the interactive multi-scale feature group sequence within the identifier group.

[0086] Furthermore, the system is also used to implement the following functions:

[0087] Extract a first multi-scale feature of identifier interaction, a first structured description text, a second multi-scale feature of identifier interaction, and a second structured description text from the identifier interaction multi-scale feature sequence and the structured description text sequence; perform dual-memory iteration on the first multi-scale feature of identifier interaction, the first structured description text, the second multi-scale feature of identifier interaction, and the second structured description text to obtain a first iterative memory; use the second multi-scale feature of identifier interaction and the second structured description text to perform dual-memory iteration on the third multi-scale feature of identifier interaction and the third structured description text in the identifier interaction multi-scale feature sequence and the structured description text sequence, and supplement the first iterative memory according to the iteration result to obtain a second iterative memory, and so on, to obtain a stage iterative memory; add the last identifier interaction multi-scale feature and the last structured description text sequence in the identifier interaction multi-scale feature sequence and the structured description text sequence to the stage iterative memory to obtain the target iterative memory.

[0088] Furthermore, the system is also used to implement the following functions:

[0089] Trend analysis is performed on the first identifier interaction multi-scale features to obtain first feature iteration trend information; semantic trend analysis is performed on the first structured description text and the second structured description text to obtain first semantic iteration trend information; the first feature iteration trend information and the first semantic iteration trend information are added to the first iteration memory.

[0090] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0091] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0092] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. An image processing method for a dynamic monitoring instrument, characterized in that, The method includes: The image frames within the monitoring window are exported through the SDK driver interface of the target dynamic monitoring instrument to obtain the monitoring image frame sequence. The monitored image frame sequence is traversed and the image quality is scored according to a preset scoring index. The image frame sequence is then identified based on the scoring results to obtain the identified monitored image frame sequence. The sequence of monitored image frames is traversed to perform semantic transformation analysis and multi-scale feature recognition, resulting in a structured description text sequence and a multi-scale feature group sequence of the identifier. Based on the identified multi-scale feature group sequence, perform intra-group feature interaction to determine the identified interaction multi-scale feature sequence; The target iterative memory is determined by performing dual memory iteration on the multi-scale feature sequence of the identifier interaction and the structured description text sequence. The target iterative memory is identified to determine the image processing result of the monitoring window; The step of performing dual-memory iteration on the multi-scale feature sequence of the identifier interaction and the structured description text sequence to determine the target iterative memory includes: Extract the first multi-scale feature of the identifier interaction, the first structured description text, the second multi-scale feature of the identifier interaction, and the second structured description text from the identifier interaction multi-scale feature sequence and the structured description text sequence; The first identifier interaction multi-scale feature, the first structured description text, the second identifier interaction multi-scale feature, and the second structured description text are subjected to dual memory iteration to obtain the first iterative memory; Using the second identifier interaction multi-scale feature and the second structured description text, a dual memory iteration is performed on the third identifier interaction multi-scale feature and the third structured description text in the identifier interaction multi-scale feature sequence and the structured description text sequence. The first iterative memory is supplemented according to the iteration result to obtain the second iterative memory. This process is repeated to obtain the stage iterative memory. The last identifier interaction multi-scale feature and the last structured description text sequence in the identifier interaction multi-scale feature sequence and the last structured description text sequence are respectively added to the stage iterative memory to obtain the target iterative memory.

2. The image processing method for a dynamic monitoring instrument as described in claim 1, characterized in that, The preset scoring indicators include peak signal-to-noise ratio, sharpness, brightness, exposure, and contrast.

3. The image processing method for a dynamic monitoring instrument as described in claim 1, characterized in that, The sequence of monitored image frames is traversed to perform semantic transformation analysis and multi-scale feature recognition, resulting in a structured descriptive text sequence and a multi-scale feature group sequence of the identifier, including: The semantic converter is invoked to perform semantic conversion analysis on the sequence of identified monitoring image frames to obtain a structured description text sequence; Obtain a multi-scale set and construct a multi-scale feature analyzer set; The multi-scale feature analyzer set is used to perform multi-scale feature recognition on the identification monitoring image frame sequence to obtain the identification multi-scale feature group sequence.

4. The image processing method for a dynamic monitoring instrument as described in claim 3, characterized in that, Obtain a multi-scale set and construct a multi-scale feature analyzer set, including: Obtain the historical image anomaly log set of the dynamic monitoring instrument; The historical image anomaly log set is divided into similar categories according to the anomaly type to obtain multiple divided historical image anomaly log sets. Extract the abnormal interval duration within the logs from the multiple sets of historical image anomaly logs to obtain multiple sets of anomaly interval durations. The multi-scale set is obtained by filtering based on the multiple sets of abnormal interval durations.

5. The image processing method for a dynamic monitoring instrument as described in claim 4, characterized in that, The multi-scale set is obtained by filtering based on the multiple sets of abnormal interval durations, including: Extract the maximum and minimum partitioning abnormal interval durations from the multiple partitioning abnormal interval duration sets respectively to obtain multiple maximum partitioning abnormal interval durations and multiple minimum partitioning abnormal interval durations; The multi-scale set is obtained by taking the union of the multiple maximum and minimum anomaly interval durations.

6. The image processing method for a dynamic monitoring instrument as described in claim 1, characterized in that, Based on the identified multi-scale feature group sequence, intra-group feature interaction is performed to determine the identified interaction multi-scale feature sequence, including: Calculate the mean of the features within each group of the multi-scale feature group sequence of the identifier to obtain the multi-scale feature sequence of the leading identifier; Based on the leading identifier multi-scale feature sequence, perform intra-group feature interaction on the corresponding identifier multi-scale feature groups in the identifier multi-scale feature group sequence to obtain the identifier intra-group interaction multi-scale feature group sequence. The interaction multi-scale feature sequence within the identifier group is analyzed by within-group mean analysis to obtain the identifier interaction multi-scale feature sequence.

7. The image processing method for a dynamic monitoring instrument as described in claim 6, characterized in that, Based on the leading identifier multi-scale feature sequence, intra-group feature interactions are performed on the corresponding identifier multi-scale feature groups in the identifier multi-scale feature group sequence to obtain the identifier intra-group interaction multi-scale feature group sequence, including: Calculate the similarity between the multi-scale features of the leading identifier and the corresponding sub-features of the multi-scale feature group in the identifier multi-scale feature group sequence, and construct the leading interaction matrix group sequence based on the calculation results; Based on the leading interaction matrix group sequence, the corresponding multi-scale feature groups in the multi-scale feature group sequence of the identifier are convolved and interacted to obtain the multi-scale feature group sequence of the identifier group interaction.

8. The image processing method for a dynamic monitoring instrument as described in claim 1, characterized in that, The first identifier interaction multi-scale feature, the first structured descriptive text, the second identifier interaction multi-scale feature, and the second structured descriptive text are subjected to dual-memory iteration to obtain the first iterative memory, including: Trend analysis is performed on the first identifier interaction multi-scale feature and the first identifier interaction multi-scale feature to obtain the first feature iteration trend information; Semantic trend analysis is performed on the first structured description text and the second structured description text to obtain first semantic iteration trend information; The first feature iteration trend information and the first semantic iteration trend information are added to the first iteration memory.

9. An image processing system for a dynamic monitoring instrument, characterized in that, The system is used to execute an image processing method for a dynamic monitoring instrument as described in any one of claims 1-8, the system comprising: The image frame export module is used to export image frames within the monitoring window through the SDK driver interface of the target dynamic monitoring instrument, thereby obtaining a monitoring image frame sequence. The scoring module is used to traverse the monitoring image frame sequence, score the image quality according to the preset scoring index, and mark the image according to the scoring results to obtain the marked monitoring image frame sequence. The analysis and recognition module is used to traverse the sequence of monitoring image frames for semantic transformation analysis and multi-scale feature recognition to obtain a structured description text sequence and a multi-scale feature group sequence of the identifier; The intra-group feature interaction module is used to perform intra-group feature interaction based on the identifier multi-scale feature group sequence to determine the identifier interaction multi-scale feature sequence. The dual-memory iteration module is used to perform dual-memory iteration on the identifier interaction multi-scale feature sequence and the structured description text sequence to determine the target iterative memory; The identification module is used to identify the target iterative memory and determine the image processing result of the monitoring window.

Images