Intensive care unit video image processing method based on image semantic segmentation

Abstract

The invention relates to the technical field of image segmentation, in particular to an intensive care unit video image processing method based on image semantic segmentation, which comprises the following steps of: acquiring a video image through monitoring camera shooting, dividing a semantic region, calculating a pixel displacement direction of each part, identifying an abnormal track, erasing interference, recombining a boundary contour, and comparing shape change. And constructing a trend and re-drawing a structure, evaluating stability in combination with directions, speeds and amplitudes, screening and correcting inconsistent labels, and outputting an index result. According to the method, track features are constructed by introducing pixel direction coding, a deviation region is identified and interference is eliminated in combination with a direction change trend, an edge structure is reconstructed by using a boundary connection sequence and curve fitting, contour division is optimized through deformation direction consistency, and labels are corrected by synthesizing direction change and boundary speed. Dynamic tracking and accurate labeling of the patient state are achieved, the abnormal action recognition efficiency and the label updating accuracy are improved, and the time sequence integrity and the space expression ability of image data are enhanced.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a method for processing video images in monitoring rooms based on semantic image segmentation. Background Technology

[0002] Image segmentation technology mainly involves dividing an image into multiple regions with specific semantic information, enabling computers to recognize and understand the boundaries and content of different objects in the image. Core aspects include pixel-level classification of targets in the image, contour extraction of regions, semantic information mapping, and label generation. It is widely used in medical image analysis, autonomous driving, video surveillance, intelligent security, and other fields. In the field of medical image processing, especially in the special scenario of neurosurgical intensive care units, image segmentation technology is of great value for extracting key physiological states of patients, monitoring sudden abnormal events, and ensuring timely medical intervention. Among them, the video image processing method for intensive care units based on image semantic segmentation refers to using convolutional neural networks to perform frame-by-frame pixel-level semantic segmentation of regions in the image frames when processing video images acquired in neurosurgical intensive care units. A training set is constructed through manual annotation, and a deep network with a fixed structure is used to segment regions such as the head, limbs, and body posture, extracting image feature information related to the patient's behavioral state. The processing often relies on static image frame input, offline labeled data training, and regularized feature extraction as processing methods to achieve segmentation and analysis of the monitoring target region in the video.

[0003] Existing technologies rely on static image frame input and offline annotation training, which makes it difficult to effectively track the motion trajectory of semantic regions in continuous dynamic monitoring scenarios. They lack the ability to perceive subtle pixel displacement and directional changes in adjacent frames in real time, resulting in lag in abnormal action recognition. When patients suddenly exhibit abnormal conditions, image misjudgment or label drift is likely to occur. At the same time, since the image segmentation process is highly dependent on regular feature extraction, the recognition accuracy decreases when faced with blurred boundaries or occlusion interference. Especially in neurosurgical monitoring scenarios, the instability of image semantic region segmentation can affect the accuracy of subsequent medical judgments and reduce the reliability of auxiliary diagnosis. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide a method for processing monitoring room video images based on image semantic segmentation. The technical solution is as follows:

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for processing intensive care unit video images based on image semantic segmentation, comprising the following steps:

[0006] S1: Acquire video images of the monitoring room through monitoring camera equipment, divide the image semantic regions, obtain pixel label maps of the patient's head, upper limbs, lower limbs and bed, calculate the pixel displacement of semantically consistent regions in adjacent frames, encode the change trajectory through angle transformation, and output the direction features of the image patches;

[0007] Video images from the monitoring room are acquired using monitoring cameras. A Swin Transformer or equivalent advanced segmentation network, combined with channel and spatial attention mechanisms, is used to segment semantic regions of the images, accurately obtaining pixel-level masks and labels for the patient's head, upper limbs, lower limbs, and bed. Based on this, high-precision optical flow estimation algorithms such as PWC-Net or RAFT are introduced to calculate the pixel-level displacement fields of semantically consistent regions in adjacent frames. By performing angle quantization transformation on the optical flow vectors and directional encoding on the changing trajectories, a refined patch orientation feature map is formed.

[0008] S2: Call the tile orientation features, analyze the changing trend of tile orientation trajectory, identify semantic tiles that deviate from the normal range, analyze the duration of deviation, filter semantic tiles that show abnormal motion features, and perform a balanced comparison of pixel orientation distribution, filter out disordered orientation areas and erase interference, and output clean image data.

[0009] S3: Based on the cleaned image data, analyze the pixel connection order of the erased boundary, fit the boundary curve between edge pixels, sort and reorganize the connection boundary according to the contour law of the closed area, and output the reconstructed boundary data.

[0010] S4: Call the reconstructed boundary data, compare it with the shape changes of the corresponding semantic tile boundary, construct the outline trend of the target area, evaluate the fusion degree through the consistency of the deformation direction of the boundary connection points, perform path segmentation and boundary redrawing on the tile structure, and output updated tile labels.

[0011] As a further aspect of the present invention, the tile orientation features include orientation change angle, orientation conversion frequency, and orientation distribution density; the cleaned image data includes abnormal tile identifiers, orientation cluttered area masks, and interference removal layers; the reconstructed boundary data includes edge connection order, curve fitting shape, and closed region topology; and the updated tile labels include structural segmentation paths, boundary reconstruction shapes, and label mapping rules.

[0012] As a further aspect of the present invention, step S1 specifically comprises:

[0013] S101: Acquire video images of the monitoring room through monitoring camera equipment, perform semantic region division based on each frame of image, subdivide the human body region in the image into head, upper limbs, lower limbs and bed body, generate corresponding category label images in pixels, construct pixel distribution information of semantic blocks, and obtain the distribution of semantic block label image data.

[0014] S102: Based on the distribution of the semantic block label image data, extract the edge pixel coordinates of semantically consistent regions in adjacent images, call the edge pixel position of the corresponding semantic block, compare the changes in horizontal and vertical coordinate values, calculate the displacement vector of the edge pixels in the corresponding region, and obtain the semantic region pixel offset displacement value.

[0015] S103: Based on the pixel offset displacement value of the semantic region, perform angle transformation on the pixel movement trajectory of the semantic region, call the coordinate change between the trajectory endpoint and the starting point, calculate the motion direction angle value, and encode the trajectory angle transformation results in multiple regions into a tile direction classification to generate tile direction features.

[0016] As a further aspect of the present invention, step S2 specifically includes:

[0017] S201: Call the tile orientation feature, calculate the orientation change amplitude value of the tile based on the orientation encoding sequence of semantically consistent tiles in the image frame, and count the orientation jump frequency per unit time. Compare the difference between the change frequency and the normal orientation change frequency range, mark the tile number that deviates from the normal range, and obtain the orientation offset interval value.

[0018] The normal direction change frequency range refers to the judgment benchmark range set by combining the number of image frames and the number of changes in the direction encoding of the blocks per unit time, based on the upper and lower bounds of the direction change frequency obtained statistically under the state of no abnormal motion.

[0019] S202: Based on the directional offset interval value, extract the directional offset time period of the marked blocks, record the offset frame number of each block and compare it with the directional offset duration reference value, filter the block numbers whose time is greater than the reference value, construct an abnormal motion block set, and generate a directional abnormal block number set.

[0020] The reference value for directional offset duration refers to the distribution of the number of consecutive frames when a semantic patch without directional clutter occurs in an image frame, and the upper quartile of the distribution is extracted as a threshold setting.

[0021] S203: Based on the set of directional anomaly patch numbers, extract the directional distribution values ​​of the corresponding region pixels, calculate the dispersion of the directional distribution within the patch, and filter regions where the directional dispersion is greater than the directional balance threshold. Erase interference by clearing the directional coding, mark the erased region and boundary information, reconstruct the directional coding distribution map of the image frame, and obtain the purified image data.

[0022] The orientation equalization threshold is set by statistically sorting the standard deviation values ​​of pixel orientation codes in semantic blocks that do not exhibit orientation clutter features, and selecting the standard deviation value corresponding to the 85th percentile in the sorting results.

[0023] As a further aspect of the present invention, the specific formula for calculating the discreteness of the directional distribution within the map patch is as follows:

[0024]

[0025] Calculate the directional dispersion value;

[0026] Among them, D u This represents the directional dispersion value of the u-th patch. α represents the directional angle distribution value of the v-th pixel in the u-th patch. u m represents the weighted average of all directional angle values ​​in the u-th tile. u μ represents the number of pixels contained in the u-th patch. u ν represents the mean gradient of the edge response of the u-th patch. u ζ represents the orientation variance factor of the boundary pixels in the u-th patch. u This represents the local contrast gain factor of the u-th patch, where v is the pixel index and u is the patch number index.

[0027] As a further aspect of the present invention, step S3 specifically comprises:

[0028] S301: Based on the cleaned image data, extract the pixel set of the erased boundary, scan according to the spatial distribution order of the pixels in the image plane, record the adjacency relationship of each group of edge pixels and mark the sequence number, and obtain the pixel connection sequence number value.

[0029] S302: Based on the pixel connection sequence number value and the coordinate difference between adjacent pixels, fit the continuous curve shape between multiple edge segments, extract the curvature change value and connection angle between edge connection segments, construct the curve segment sequence of each group of erased edges, and generate edge pixel fitting curve values.

[0030] S303: Call the edge pixel fitting curve value, identify the boundary contour direction features in the closed area, combine and sort the edge curve segments according to the priority sorting criterion of contour continuity, update the complete boundary contour structure of the corresponding area, and output the reconstructed boundary data.

[0031] The "contour continuity priority" refers to the practice of selecting the closest segment with the smallest angle change for connection when sorting boundary curve segments by comparing the Euclidean distance between the endpoints of adjacent segments and the angle between the connection direction.

[0032] As a further aspect of the present invention, step S4 specifically comprises:

[0033] S401: Call the reconstructed boundary data, extract the node set of the current block boundary according to the original boundary coordinates of the semantic block in the image frame, compare the position of the current node with the corresponding original boundary node, obtain the spatial offset vector of each node, and generate the boundary deformation direction change value by calculating the trend of the node displacement direction sequence.

[0034] S402: Based on the boundary deformation direction change value, identify the consistency of direction change between boundary connection points, calculate the direction continuity rate of the boundary point sequence in each block, compare the direction continuity rate value with the boundary connection reference value, filter the block positions that are lower than the boundary connection reference value, and obtain the boundary fusion consistency coefficient.

[0035] The boundary connection benchmark value is determined by statistically sorting the directional continuity rate of the boundary point sequence in the semantic block without directional clutter features, and selecting the value corresponding to the 80th percentile of all continuity rate samples as the set value.

[0036] S403: Based on the boundary fusion consistency coefficient, locate the tile region where the boundary connection direction changes abruptly, re-divide the internal boundary path, and re-label the tile boundary position according to the direction gradient of the boundary turning point, update the spatial range distribution label of the semantic tile in the image, and obtain the updated tile label.

[0037] As a further aspect of the present invention, the specific formula for calculating the directional continuity rate of the boundary point sequence within each tile is as follows:

[0038]

[0039] Calculate the directional continuity rate value;

[0040] Among them, R i θ represents the directional continuity of the sequence of boundary points within the i-th tile. i,j θ represents the tangent direction angle at the j-th boundary point in the i-th tile. i,j+1δ represents the tangent direction angle value of the (j+1)th boundary point in the i-th tile. i λ represents the maximum difference in the direction changes of all boundary points in the i-th tile. i ε represents the arithmetic mean of the directional changes of all boundary points in the i-th tile. i ρ represents the pixel density value per unit area within the boundary region of the i-th tile boundary point. i n represents the boundary texture complexity index in the region of the i-th tile boundary point. i This represents the total number of boundary points in the i-th tile, where i is the tile index number used to identify different tiles, and j is the sequence index number of the boundary points within the tile.

[0041] As a further aspect of the present invention, the method further includes:

[0042] S5: Call the updated tile label, analyze the stability of the tile semantic label by combining the direction change, boundary change speed and spatial movement of the tile within the corresponding time period, filter out tiles that do not meet the consistency conditions, perform label correction, and output the label index result;

[0043] The specific results of the tag index are the directional stability level, tag consistency score, and spatiotemporal offset index.

[0044] As a further aspect of the present invention, step S5 specifically includes:

[0045] S501: Call the updated tile label, scan the direction encoding sequence of each tile in the corresponding time period, count the number of direction switching times and the change angle fluctuation value per unit time, analyze according to the direction change frequency range and the angle difference threshold, and obtain the tile direction change fluctuation value.

[0046] The frequency range of direction change is defined as low, medium, and high frequency by counting the number of times the direction code of the map block switches per unit time.

[0047] The angle difference threshold is set according to the angle difference between adjacent direction codes, and is set as micro change, medium change and drastic change;

[0048] S502: Based on the tile orientation change fluctuation value, extract the number of boundary change pixels and the center point position offset of the tile in the corresponding image frame, calculate the boundary change speed and spatial movement amplitude per unit time, construct a tile consistency score including orientation change frequency, angle jump fluctuation, boundary speed and centroid displacement, and compare it with the consistency judgment benchmark to generate a tile consistency deviation value.

[0049] The consistency judgment benchmark extracts four indicators: direction change frequency, angle jump amplitude, boundary velocity, and centroid displacement. Combined with the statistical results of 1000 stable tile samples, it is set as a multi-dimensional reference interval, where the direction frequency is set to ≤3 times / second, the angle fluctuation is set to ≤15°, the boundary velocity is set to ≤60 pixels / second, and the displacement is set to ≤3 pixels / frame.

[0050] S503: Based on the block consistency deviation value, filter the block numbers whose deviation value exceeds the consistency judgment benchmark, replace the original labels, reassign the label values ​​to the corresponding semantic classification identifiers, and update the number correspondence of all blocks to obtain the label index result.

[0051] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0052] By introducing directional encoding within semantic regions to reveal pixel displacement trends, and combining trajectory direction changes to achieve dynamic identification of abnormal behaviors, the system effectively removes interference information through balanced processing of directional distribution. In boundary data recovery, edge connection order and curve fitting are introduced to improve structural integrity. When reconstructing the regional shape trend, consistency assessment of boundary deformation direction is introduced to improve the coherence of contour expression. Finally, through the label correction process, multi-dimensional features are integrated for semantic consistency judgment, enabling continuous modeling and accurate annotation of patient state details in images. This enhances the ability to discriminate abnormal actions in image sequences and the spatiotemporal continuity of monitoring images, improves the real-time adaptability and semantic interpretation depth of image recognition in neurosurgical monitoring scenarios, and ensures the accuracy and traceability of behavior analysis. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of the workflow of the present invention;

[0055] Figure 2 This is a flowchart of the semantic segmentation and directional feature extraction process of the present invention;

[0056] Figure 3 This is a flowchart of the abnormal direction detection and purification image data generation process of the present invention;

[0057] Figure 4 This is a flowchart of the boundary erasure and reconstruction process of the present invention;

[0058] Figure 5This is a flowchart of the boundary deformation analysis and label update process of this invention;

[0059] Figure 6 This is a flowchart of the label consistency assessment and correction process for this invention. Detailed Implementation

[0060] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0061] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0062] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0063] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0064] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0065] Please see Figure 1 This invention provides a technical solution: a method for processing video images in a monitoring room based on image semantic segmentation, comprising the following steps:

[0066] S1: Video images from the monitoring room are acquired using monitoring cameras. A Swin Transformer or equivalent advanced segmentation network, combined with channel and spatial attention mechanisms, is used to segment semantic regions of the images, accurately obtaining pixel-level masks and labels for the patient's head, upper limbs, lower limbs, and bed. Based on this, high-precision optical flow estimation algorithms such as PWC-Net or RAFT are introduced to calculate the pixel-level displacement fields of semantically consistent regions in adjacent frames. By performing angle quantization transformation on the optical flow vectors and encoding the changing trajectories, a refined patch orientation feature map is formed.

[0067] S2: The orientation features of the image tiles are retrieved and input into a pre-trained LSTM network to analyze the changing trends of the tile orientation trajectories over time. This network can learn and identify normal movement patterns (e.g., turning over, sleep micro-movements) and abnormal patterns deviating from the normal range (e.g., violent struggle, convulsions, signs of falling out of bed). The persistence of the deviation is analyzed by combining preset duration thresholds to filter out semantic image tiles exhibiting abnormal movement features. Simultaneously, a generative adversarial network (GAN) inpainting model is used to intelligently identify and erase cluttered areas caused by IV tubes, blankets, etc., outputting a highly cleaned image data stream.

[0068] S3: Based on the cleaned image data, an active contour model (or Snakes) or level-set method is used to analyze the pixel connection order at the erased boundaries. This algorithm, guided by energy minimization, automatically fits the optimal boundary curves between edge pixels. By introducing prior knowledge of the patient's body structure as a constraint, the connected boundaries are sorted and reorganized according to the topological rules of the closed region contours, outputting smooth reconstructed boundary data that conforms to physiological logic.

[0069] S4: Using the reconstructed boundary data, the semantic tiles and their connections are abstracted into a graph structure, where tiles are nodes and boundaries are edges. The shape changes of this graph are compared temporally with those of the corresponding semantic tile boundaries. A Graph Convolutional Network (GCN) is used to analyze the structural relationships and deformation propagation trends between tiles, evaluating the consistency and fusion of deformation directions at boundary connection points. Based on the GCN output, path segmentation is performed on tiles with strong relationships but structural fragmentation, or boundary redrawing is performed on tiles with blurred boundaries, outputting updated tile labels.

[0070] S5: Using the updated tile labels, a spatiotemporal attention model is constructed, incorporating orientation changes, boundary velocities, spatial displacements, and inter-tile relationships. This model dynamically and weights the analysis of various features of tiles within their time series and spatial neighborhood, comprehensively evaluating the stability of their semantic labels. This model filters out tiles with confidence levels below a threshold and those that do not meet the motion and morphological consistency criteria. Based on their spatiotemporal correlation with high-confidence neighboring tiles, a "majority vote" or "optimal propagation" correction of the labels is performed. The final output is a highly accurate and stable label index, providing a decision-making basis for subsequent medical diagnosis and alert systems.

[0071] Patch orientation features include orientation change angle, orientation conversion frequency, and orientation distribution density. Cleaning image data includes abnormal patch identifiers, orientation cluttered area masks, and interference removal layers. Reconstructing boundary data includes edge connection order, curve fitting morphology, and closed region topology. Updating patch labels includes structural segmentation paths, boundary reconstruction morphology, and label mapping rules. The label index results specifically include orientation stability level, label consistency score, and spatiotemporal offset index.

[0072] Please see Figure 2 The specific steps of S1 are as follows:

[0073] S101: Acquire video images of the monitoring room through monitoring camera equipment, perform semantic region division based on each frame of image, subdivide the human body region in the image into head, upper limbs, lower limbs and bed body, generate corresponding category label images in pixels, construct pixel distribution information of semantic blocks, and obtain the distribution of semantic block label image data.

[0074] Video images from the monitoring room are acquired using monitoring cameras, typically at a rate of 25 frames per second. For each frame, preprocessing is performed, including color equalization, noise filtering, and resolution normalization. For example, the image resolution is uniformly adjusted to 1920×1080, and Gaussian filtering is applied to remove outliers with grayscale fluctuations exceeding 10 within a 3×3 area. Semantic segmentation is then performed, involving detailed annotation of human body regions. This process involves dividing the human skeletal structure into regions such as the head, upper limbs, lower limbs, and bed frame. For instance, for a patient lying in bed, gradient distribution can be used to identify regions located at the top of the image. The image pixels occupied by each type of region—the head region, the upper limb region extending from the middle section, the lower limb region covering the bed, and the bed region with regular rectangular boundaries in the background—are encoded and represented in the form of a category label map. Each category label is assigned a unique numerical value, for example, 1 for the head, 2 for the upper limbs, 3 for the lower limbs, and 4 for the bed. The RGB value of each pixel is scanned by traversing the image and the Euclidean distance is compared with the average RGB value of the same type of region in the labeled samples. If the distance is less than a set threshold of 20, the pixel is classified into that label, and the number of pixels it occupies in the entire image is counted. For example, if the upper limb region occupies 85,000 pixels out of a total of 1920×1080 pixels in the image, a semantic block label image is generated and the pixel distribution information of this type of region is recorded, as detailed in the table below.

[0075] Table 1. Semantic Block Pixel Distribution Statistics

[0076] Region Type Tag value Number of pixels Pixel percentage (%) head 1 24,000 1.16 upper limbs 2 85,000 4.11 lower limbs 3 104,500 5.08 bed 4 250,000 12.06

[0077] As shown in Table 1, based on the above mapping method, the spatial proportion of each semantic region in the image can be intuitively obtained, thereby completing the extraction of the semantic block label image data distribution.

[0078] S102: Based on the distribution of semantic block label image data, extract the edge pixel coordinates of semantically consistent regions in adjacent images, call the edge pixel position of the corresponding semantic block, compare the changes in horizontal and vertical coordinate values, calculate the displacement vector of the edge pixels in the corresponding region, and obtain the pixel offset displacement value of the semantic region.

[0079] Based on the constructed semantic block label image data distribution, edge pixel coordinates of semantically consistent regions are extracted from consecutive frame images. Taking frames t and t+1 as examples, a set of pixels with semantic label 2 (upper limb region) is selected, and the pixel position coordinates on the contour boundary of this region are extracted. By performing pixel gradient change detection on the eight neighbors of each boundary pixel, the position of the boundary point is determined. For example, in frame t, the edge point position of a certain upper limb region is (110, 560), and in frame t+1, the closest matching point in the same label region is determined. The position is (115, 563). At this point, the changes in the horizontal and vertical coordinates are Δx = 5 and Δy = 3, respectively. The displacements of all edge points are iterated and solved to form a set of overall displacement vectors. The overall offset displacement value of the pixels in this region is obtained by summing the average values ​​of all Δx and Δy values ​​in this set. If the average Δx of 100 edge points in an upper limb region is 4.6 and the average Δy is 3.2, then the corresponding displacement vector is (4.6, 3.2). Further, the magnitude of the displacement is calculated using the modulus method, that is, by taking the square root of the sum of squares. When the displacement value exceeds the set reference value of 5, it is determined to be a valid offset. The reference value is set with reference to the maximum static fluctuation of the last 20 frames in the same area. By collecting data, it is found that the maximum pixel jitter amplitude of the upper limb region in the static state does not exceed 5 pixels. Therefore, the threshold is set to 5. This result shows that actual displacement behavior occurred in the upper limb region between adjacent frames. This process can be extended to pixel displacement evaluation of other regions such as the head and lower limbs to obtain the pixel offset displacement value of each semantic region.

[0080] S103: Based on the pixel offset displacement value of the semantic region, perform angle transformation on the pixel movement trajectory of the semantic region, call the coordinate change between the trajectory endpoint and the starting point, calculate the motion direction angle value, and encode the trajectory angle transformation results in multiple regions into a tile direction classification to generate tile direction features.

[0081] Based on the semantic region pixel offset displacement values ​​obtained in paragraph S102, an angle conversion operation is performed on the displacement vector of each semantic region. Taking the upper limb region as an example, if the coordinate change between the endpoint and the starting point is Δx = 4.6 and Δy = 3.2, then the angle value θ of its movement direction can be calculated using the arctangent function:

[0082] This angle value indicates that the upper limb region underwent pixel movement along a 34.9-degree direction between two image frames. Further, the same angle transformation operation was performed on multiple regions, calculating the motion angle values ​​for the head, lower limbs, and bed, for example, 5.2 degrees for the head region, 72.5 degrees for the lower limbs region, and 0.2 degrees for the bed region. Then, all the angle values ​​were normalized and mapped to the image block orientation classification. The orientation classification was set to 8 categories, dividing 0-360 degrees into 8 intervals, each interval being 45 degrees wide. For example, 0-45 degrees was mapped to orientation 1, 45-90 degrees to orientation 2, and so on. Combining the above region orientation angle values, the head was classified as orientation 1, the upper limbs as orientation 1, the lower limbs as orientation 2, and the bed as orientation 1. Each orientation category was then used to construct the image block orientation feature using a four-bit array, i.e., (1,1,2,1). This vector represents the composition of the motion orientation types of the four semantic blocks in the image frame, providing basic data support for subsequent behavior analysis.

[0083] Please see Figure 3 The specific steps of S2 are as follows:

[0084] S201: Call the tile orientation features, calculate the orientation change amplitude value of the tile based on the orientation encoding sequence of semantically consistent tiles in the image frame, and count the orientation jump frequency per unit time. Compare the difference between the change frequency and the normal orientation change frequency range, mark the tile number that deviates from the normal range, and obtain the orientation offset interval value.

[0085] To access tile orientation features, the tile orientation encoding sequence in each frame must first be extracted. For example, in a 10-second video at 25fps, there are 250 frames. The extracted tile orientation feature vectors (e.g., (1,2,2,1)) are concatenated into a complete sequence, and a change trajectory table is independently built for each tile dimension. Then, the difference in orientation encoding sequences is calculated for each tile. For each tile, the orientation encoding values ​​of its adjacent frames are iterated, and the change amplitude is calculated. For example, if a tile's orientation is 2 in frame t and 4 in frame t+1, the change amplitude is |4-2|=2. The average orientation change amplitude is obtained by summing the differences in orientation encoding values ​​of all adjacent frames and dividing by the total number of frames minus 1 within the time period. Furthermore, the orientation jump frequency per unit time is calculated for each tile. This frequency is the ratio of the number of frames with an orientation change amplitude greater than 1 to the total number of frames. For example, if 45 out of 250 frames have an orientation change amplitude greater than 1, the jump frequency is... Then compare this frequency value with the set normal direction change frequency range to determine whether it deviates from the range. The normal frequency range needs to be statistically obtained under the condition of no abnormal movement. Assuming that the frequency fluctuation range under the condition of no abnormality within 10 seconds is [0.05, 0.12], if the current jump frequency value 0.18 is greater than the upper limit 0.12, it is determined to be a direction abnormality. The block number needs to be marked and the degree of deviation needs to be statistically analyzed. The deviation range value is defined as the difference between the jump frequency and the nearest boundary value. In this example, it is 0.18-0.12=0.06. The calculation of the direction offset range value is completed. See the table below for details.

[0086] Table 2 Analysis of Block Direction Jumps

[0087]

[0088] As shown in Table 2, the directional jump frequencies of blocks 001 and 003 both deviated from the normal range and were marked as abnormal blocks. Their directional offset range values ​​were 0.06 and 0.02, respectively. This result shows that the jump frequency calculation and difference judgment process can accurately indicate the abnormal directional change area and provide a basic identifier for subsequent processing.

[0089] The normal direction change frequency range refers to the judgment benchmark range set by combining the number of image frames per unit time and the number of changes in the direction encoding of the blocks, based on the upper and lower limits of the direction change frequency obtained statistically under the condition of no abnormal motion.

[0090] S202: Based on the directional offset interval value, extract the directional offset time period of the marked blocks, record the offset frame number of each block and compare it with the directional offset duration benchmark value, filter the block numbers whose time is greater than the benchmark value, construct an abnormal motion block set, and generate a directional abnormal block number set;

[0091] Based on the directional offset interval value, continue tracing the directional encoding sequence within the marked abnormal tiles. Record the frame time range corresponding to each offset behavior, extract the start and end frame numbers of the offset occurrence, and calculate the number of frames the offset lasts. For example, tile number 001 experiences continuous directional encoding offsets between frames 105 and 140, consistent with the aforementioned frequency jump anomaly area. Therefore, the number of frames for this offset segment is 140 - 105 + 1 = 36 frames. Then, convert the time length using a frame rate of 25fps. The time interval is then measured in seconds. A baseline value for the directional offset duration is then obtained. This baseline value is the distribution data of the directional offset duration of tiles in areas without anomalies. The upper quartile is extracted to set this baseline. For example, the following sequence is extracted from the offset duration distribution of normal tiles: 0.8s, 0.6s, 1.1s, 0.9s, 1.4s, 1.0s, 0.7s, 0.5s. Sorted from smallest to largest, these are: 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.4. The upper quartile, i.e., the 6th value, is then extracted. 1.0s, the baseline value for the direction offset duration is set to 1.0s. If the offset duration of the aforementioned block 001 is 1.44s > 1.0s, then the block meets the abnormal block screening criteria and needs to be included in the set of abnormal direction block numbers to complete the construction of the abnormal block set. Finally, a set of block numbers that meet the offset duration exceeding the baseline, such as {001, 003}, can be generated for subsequent behavior discrimination processing. The result can be directly exported as a block abnormal index file or used as a Boolean array to build an index mask for subsequent filtering and identification operations.

[0092] The baseline value for directional offset duration refers to the distribution of the number of consecutive frames when a semantic patch without directional clutter occurs in an image frame, and the upper quartile of the distribution is extracted as a threshold setting.

[0093] S203: Based on the set of directional anomaly patch numbers, extract the directional distribution values ​​of pixels in the corresponding region, calculate the dispersion of directional distribution within the patch, and filter regions with directional dispersion greater than the directional equalization threshold. Erase interference by clearing directional coding, mark the erased region and boundary information, reconstruct the directional coding distribution map of the image frame, and obtain the cleaned image data.

[0094] The specific formula for calculating the dispersion of directional distribution within a tile is as follows:

[0095]

[0096] Calculate the directional dispersion value;

[0097] Among them, D u This represents the directional dispersion value of the u-th patch. α represents the directional angle distribution value of the v-th pixel in the u-th patch. um represents the weighted average of all directional angle values ​​in the u-th tile. u μ represents the number of pixels contained in the u-th patch. u ν represents the mean gradient of the edge response of the u-th patch. u ζ represents the orientation variance factor of the boundary pixels in the u-th patch. u This represents the local contrast gain factor of the u-th patch, where v is the pixel index and u is the patch number index.

[0098] formula:

[0099]

[0100] Detailed explanation of the formula and its calculation derivation:

[0101] The formula is used to calculate the directional dispersion value of the u-th patch in the image. The result is used to determine the degree of local perturbation of the directional features within the patch, and serves as the input for interference determination in the image denoising process.

[0102] Parameter meanings and settings:

[0103] The directional angle distribution value of the v-th pixel in patch u is obtained by extracting it under the gray-level gradient direction of the image using the Sobel edge operator and then performing inverse Chebyshev fitting. The unit is degrees, and the values ​​are set to 35, 37, 38, 42, 43, 40, 41, 44, and 45 respectively.

[0104] α u The weighted average of all directional angles in patch u is calculated using the following expression: The local gradient response magnitudes for each pixel's directional angle are set to 12, 15, 14, 16, 19, 13, 12, 20, and 17.

[0105] ζ u The local contrast gain factor for patch u is derived from the difference between the maximum and minimum direction angles in the patch, divided by 255, and is set to...

[0106] m u The number of pixels in patch u is set to 9 for a 3×3 patch.

[0107] μ u Let be the average gradient response magnitude of the non-flat pixels in patch u. The measured pixel gradient magnitude values ​​are set to 12, 15, 14, 16, 19, 13, 12, 20, and 17, and the calculation is as follows:

[0108] νu Let $\begin{pmatrix}$ be the standard deviation of the directional angle variation of boundary pixels in patch $u$. The mean and variance of the directional angle values ​​(35°, 37°, 38°, 41°, 45°) for the corner points and edges of the patch are calculated.

[0109] Substitute the parameters into the formula to calculate:

[0110]

[0111] 24.01·(1+0.0392)≈24.01×1.0392=24.95;

[0112]

[0113] Substitute into the formula to calculate the directional dispersion value:

[0114]

[0115] The result of 1.009 indicates that the deviation of the orientation angle within the patch is greater than the suppression benchmark composed of the number of patch pixels and the boundary stability factor. The orientation information exhibits large fluctuations in the local area. The patch can be identified as an orientation structure disturbance area. The orientation dispersion value enters the subsequent image encoding and clearing module to perform angle reconstruction and boundary correction operations.

[0116] The orientation equalization threshold is set by statistically sorting the standard deviation values ​​of pixel orientation codes in semantic blocks that do not exhibit orientation clutter features, and selecting the standard deviation value corresponding to the 85th percentile of the sorting results.

[0117] Please see Figure 4 The specific steps of S3 are as follows:

[0118] S301: Based on the cleaned image data, extract the pixel set of the erased boundary, scan according to the spatial distribution order of the pixels in the image plane, record the adjacency relationship of each group of edge pixels and mark the sequence number, and obtain the pixel connection sequence number value.

[0119] Based on the cleaned image data, it is necessary to first extract the boundaries of the erased areas in the image. This extraction process involves performing edge detection on areas with varying grayscale thresholds to obtain a set of boundary pixels. Grayscale changes can be determined using a fixed threshold; for example, a pixel is considered a boundary pixel when the grayscale difference between any two adjacent pixels exceeds a set value of 20. This threshold is set based on the amplitude of background texture noise in the cleaned image. Subsequently, the extracted boundary pixel set is sorted by coordinates. The sorting method starts from the top left corner of the image, proceeding along the x-coordinate from smallest to largest, and, when x-coordinates are the same, along the y-coordinate from smallest to largest to form a scanning path. Given an image pixel plane size of 512×512, if the set of edge point coordinates is {(10,5),(11,5),(12,5),(12,6),(12,7)}, then they are numbered 1, 2, 3, 4, 5 in sequence. After completing the spatial order numbering, each edge point is used as the center point to search its 8 neighboring pixels, and the numbers of all adjacent edge points with connection relationships are recorded. For example, if pixel (12,5) is adjacent to (12,6), then its adjacency relationship is recorded as 3→4. This completes the process of statistically analyzing and numbering the connection sequence of the entire set of edge pixels, and finally outputs the connection sequence number value of each edge pixel.

[0120] S302: Based on the pixel connection sequence number value and the coordinate difference between adjacent pixels, fit the continuous curve shape between multiple edge segments, extract the curvature change value and connection angle between edge connection segments, construct the curve segment sequence of each group of erased edges, and generate edge pixel fitting curve values.

[0121] Based on the obtained pixel connection sequence number, traverse all adjacent point pairs in the entire edge pixel sequence, extract the difference between their horizontal and vertical coordinates, and calculate the direction vector. Then, analyze the direction change process of each continuous edge segment using a sliding window. The direction change is achieved by calculating the angle between the vectors. Given a three-point sequence A(10,5), B(11,5), and C(12,6), the direction of segment AB is (1,0), the direction of segment BC is (1,1), and the angle θ between them satisfies: The curvature change value of this segment is 45 degrees. The included angles of all continuous segments are calculated and accumulated to finally generate complete edge curve segment curvature distribution data. The connection direction of adjacent points is further converted into an angle set, and then the curve shape is fitted by combining the coordinate difference. For example, if three points form a continuous polyline, spline curve segments can be generated by spline interpolation or continuous connection of points to line segments. Each segment records its start and end points, curvature value and turning direction to complete the edge curve fitting process and form the edge pixel fitting curve value. This fitting value includes the start and end point coordinates, curvature, turning angle and connection order of each edge line segment.

[0122] S303: Call the edge pixel fitting curve value, identify the boundary contour direction features in the closed region, combine and sort the edge curve segments according to the priority sorting criterion of contour continuity, update the complete boundary contour structure of the corresponding region, and output the reconstructed boundary data.

[0123] The obtained edge pixel fitting curve values ​​are used to identify the boundary contours in all closed regions. It is determined whether the endpoints of the curve segments form closed paths, and the curve segments are sorted and combined according to the principle of contour continuity priority. The sorting criterion is to prioritize connecting segments with the smallest Euclidean distance and the smallest angle change. For example, the Euclidean distance between the end point of segment A (20, 30) and the starting point of segment B (21, 31) is: Simultaneously, it determines whether the connection direction is smooth. If the direction of the first segment is (1,0) and the direction of the second segment is (1,1), and the included angle is 45 degrees, then the sorting score is set as distance weight + angle weight. For example, if the Euclidean distance accounts for 0.6 and the included angle accounts for 0.4, the total score is: 0.6×1.414+0.4×45=0.8484+18=18.8484. The segment with the smallest score is selected from all candidate connection segments as the next connection object. The entire segment set is traversed in this way for combination and sorting, and finally a continuous and uninterrupted contour boundary path is generated. The closed region structure information is updated and the reconstructed boundary data is output.

[0124] Table 3. Parameters for Reconstructing Edge Curves

[0125] Curve number Starting coordinates End point coordinates Curvature (degrees) Turning angle (degrees) Euclidean distance Sorting and rating A (10,5) (12,6) 45.0 45.0 2.236 20.141 B (12,6) (14,6) 0.0 0.0 2.0 1.2 C (14,6) (16,8) 45.0 45.0 2.828 20.897

[0126] As shown in Table 3, the lowest ranking score will be connected first to form a continuous boundary path. This scoring combination mode completes the priority ranking of contour continuity and is used to update the boundary contour structure of closed regions in the target image.

[0127] Contour continuity priority means that when sorting boundary curve segments, by comparing the Euclidean distance between the endpoints of adjacent segments and the angle between the connection direction, the segment with the closest distance and the smallest angle change is selected for connection.

[0128] Please see Figure 5 The specific steps of S4 are as follows:

[0129] S401: Call the reconstructed boundary data, extract the node set of the current tile boundary based on the original boundary coordinates of the semantic tile in the image frame, compare the position of the current node with the corresponding original boundary node, obtain the spatial offset vector of each node, and generate the boundary deformation direction change value by calculating the trend of the node displacement direction sequence.

[0130] To access reconstructed boundary data, the reconstructed boundary contour segments of the current semantic tile must first be extracted from the image frame. This constructs a set of boundary paths composed of continuous nodes, where each path is represented by the coordinates of several boundary nodes. For example, the current boundary node sequence for tile A is {(105,202),(106,202),(107,203),(108,205)}. Then, the original boundary nodes corresponding to the tile are extracted from the tile's historical records or the original image frame, for example, {(104,200),(105,201),(106,202),(107,203)}. Next, the coordinate difference between each current node coordinate and its corresponding node in the original boundary is calculated to generate a spatial offset vector. Specifically, let the current node be (107,203) and the original node be (106,202), with offset vectors Δx = 1 and Δy = 1. After performing the same processing on all nodes, a node displacement vector is constructed. The set is then used to form a vector direction sequence in node index order. The direction change trend of this sequence is further calculated. For example, the vector direction of node 1→2 is (1,0), 2→3 is (1,1), and 3→4 is (1,2). The angle between any two consecutive vectors is calculated. If the angle change value in the vector sequence gradually increases, it can be regarded as a curvature change in the boundary line shape. The boundary deformation direction change value is quantified by statistically analyzing the proportion of positive changes in the direction change trend. Assuming that there are 7 changes in the direction angle that increase out of 10 changes, the deformation direction change rate is 0.7. This value is compared with the baseline deformation rate. The baseline value is set with reference to the median and standard deviation range of the direction change rate of normal tiles in the static frame. For example, the change rate in the normal frame falls in the range of [0.2,0.6]. The current 0.7 exceeds the upper limit, indicating that the boundary of the tile has undergone a large deformation. Finally, the boundary deformation direction change value of the tile is output for subsequent tile spatial structure updates.

[0131] S402: Based on the change value of the direction of boundary deformation, identify the degree of consistency of the direction change between boundary connection points, calculate the direction continuity rate of the boundary point sequence in each block, compare the direction continuity rate value with the boundary connection reference value, filter the block positions that are lower than the boundary connection reference value, and obtain the boundary fusion consistency coefficient.

[0132] The specific formula for calculating the directional continuity rate of the boundary point sequence within each tile is as follows:

[0133]

[0134] Calculate the directional continuity rate value;

[0135] Among them, R i θ represents the directional continuity of the sequence of boundary points within the i-th tile. i,j θ represents the tangent direction angle at the j-th boundary point in the i-th tile.i,j+1 δ represents the tangent direction angle value of the (j+1)th boundary point in the i-th tile. i λ represents the maximum difference in the direction changes of all boundary points in the i-th tile. i ε represents the arithmetic mean of the directional changes of all boundary points in the i-th tile. i ρ represents the pixel density value per unit area within the boundary region of the i-th tile boundary point. i n represents the boundary texture complexity index in the region of the i-th tile boundary point. i This represents the total number of boundary points in the i-th tile, where i is the tile index number used to identify different tiles, and j is the sequence index number of the boundary points within the tile.

[0136] formula:

[0137]

[0138] Detailed explanation of the formula and its calculation derivation:

[0139] The formula is used to calculate the continuity of the direction change of boundary points within a tile, and the result is used to determine the degree of consistency of the tile boundary fusion.

[0140] Parameter meanings and settings:

[0141] θ i,j The tangent direction angle value of the j-th boundary point within the image patch, in degrees, is obtained by calculating the gradient direction angle from the image boundary extraction results. The set values ​​are 42.6 degrees, 47.1 degrees, 49.3 degrees, 46.7 degrees, 41.2 degrees, and 40.5 degrees.

[0142] δ i The maximum difference in the direction angle of the boundary points within the map block is calculated by taking the maximum value after calculating the difference in the direction angle of each group of adjacent boundary points. The value is set to 8.8 degrees.

[0143] λ i The arithmetic mean of the changes in the orientation angle of the map block boundary is obtained by summing the orientation angle differences and dividing by the number of groups, with a set value of 6.3 degrees.

[0144] ε i The density of boundary pixels per unit area within the tile is obtained by counting the number of boundary pixels within the tile and dividing by the tile's pixel area. The number of boundary pixels is set to 28, the tile size is set to 16×16 pixels, the total area is 256 pixels, and the density value is 28 divided by 256, with a set value of 0.1094.

[0145] ρ i The texture complexity index of the tile boundary is calculated by normalizing the standard deviation of the gray-level gradient in the edge region of the tile and set to 0.0761.

[0146] n i The total number of boundary points within the map block is set to 6;

[0147] Substitute the parameters into the formula to calculate:

[0148] The differences in direction angles for each group are calculated as follows:

[0149] θ i,2 -θ i,1 =47.1 - 42.6 = 4.5;

[0150] θ i,3 -θ i,2 =49.3-47.1=2.2;

[0151] θ i,4 -θ i,3 =46.7-49.3=-2.6;

[0152] θ i,5 -θ i,4 =41.2 - 46.7 = -5.5;

[0153] θ i,6 -θ i,5 =40.5-41.2=-0.7;

[0154] Calculate the absolute value expression:

[0155] |4.5+8.8-6.3+0.1094×0.0761|=|4.5+2.5+0.00833|=|7.00833|=7.00833;

[0156] |2.2+8.8-6.3+0.00833|=|4.7+0.00833|=4.70833;

[0157] |-2.6+8.8-6.3+0.00833|=|-0.1+0.00833|=|-0.09167|=0.09167;

[0158] |-5.5+8.8-6.3+0.00833|=|-3.0+0.00833|=|-2.99167|=2.99167;

[0159] Item 5: |-0.7+8.8-6.3+0.00833|=|1.8+0.00833|=1.80833;

[0160] Calculate the sum of the numerators:

[0161] ∑=7.00833+4.70833+0.09167+2.99167+1.80833=16.60833;

[0162] Substitute into the formula to calculate:

[0163]

[0164] The result of -2.321666 indicates a negative directional continuity rate, far below the acceptable continuity range. This result suggests that the directional changes of boundary points within a tile exhibit significant abrupt changes between adjacent points. This value indicates discontinuous and uneven boundary connectivity, and the tile should be marked as an inconsistent region in subsequent boundary fusion operations. This result can be further used to filter out tile regions lacking boundary connectivity, serving as a lower limit reference threshold for the boundary fusion consistency coefficient.

[0165] The boundary connectivity baseline value is determined by statistically sorting the directional continuity rates of boundary point sequences in semantic patches that do not exhibit directional clutter features, and selecting the value corresponding to the 80th percentile among all continuity rate samples as the set value.

[0166] S403: Based on the boundary fusion consistency coefficient, locate the tile regions where the boundary connection direction changes abruptly, re-divide the internal boundary paths, and re-label the tile boundary positions according to the directional gradient of the boundary inflection points. Update the spatial range distribution labels of semantic tiles in the image to obtain updated tile labels.

[0167] Based on the known deformation patch information, the boundary fusion consistency coefficient is called to check all boundary connection segments in the image to identify whether there is a sudden change in direction at the connection point. The judgment criterion is whether the angle between the directions of the two boundary segments before and after the connection point exceeds a set threshold. For example, if the direction before the connection segment is (1,1) and the direction after it is (-1,0), then the angle θ satisfies: If the directional change threshold is set to 60°, the connection segment is determined to have a change. Then, the internal boundary path of the patch is re-divided, the point sequence is broken down and the breakpoint is recombined at the turning point to generate a new set of boundary segments. Based on the new boundary segments, the directional angle of each segment is re-labeled according to the gradient direction to form a new patch boundary direction. The boundary node set of the new path is further mapped back to the original image space, and the spatial position of the pixels it surrounds is recalculated. The range label matrix of the semantic patch in the image is updated, and finally the complete patch label update result is obtained.

[0168] Table 4. Statistics on Patch Boundary Abrupt Changes and Label Updates

[0169]

[0170] As shown in Table 4, tiles A and C were redefined and their labels were updated due to the presence of abrupt directional changes and an angle exceeding 60 degrees. Tile B, however, retained its label because it did not meet the directional change condition. This result demonstrates that the boundary reconstruction and spatial label re-delineation process can effectively identify and address tile structural anomalies.

[0171] Please see Figure 6 The specific steps of S5 are as follows:

[0172] S501: Call the update tile label, scan the direction encoding sequence of each tile in the corresponding time period, count the number of direction switching times and the change angle fluctuation value per unit time, analyze according to the direction change frequency range and the angle difference threshold, and obtain the tile direction change fluctuation value.

[0173] After calling the updated tile labels, extract the complete orientation encoding sequence for each tile within its corresponding time period. Set the analysis time period to 2 seconds and the image frame rate to 25fps, then each tile requires analysis of 50 frames of data. The orientation encoding is recorded in four-bit vector form, such as (1,2,3,2). Compare the orientation encoding of each frame with the previous frame and count the number of orientation changes. If the orientation sequence of tile number T001 is [1,1,2,2,3,3,2,2,1,1], then the number of changes is 4, which is 2 times / second within 2 seconds. Next, analyze the angle difference corresponding to each orientation change. After converting the orientation encoding into actual angle values, calculate the absolute value of the angle difference. For example, if orientation 1 is 0° and orientation 2 is 45°, then the orientation difference is |45-0|=45°, and so on. If the above 4 changes are 45°, 45°, 45°, and 45° respectively, then the angle change fluctuation value is the average of the 4 changes, calculated as follows: Based on the set directional frequency range, 2 times / second is classified into the "medium frequency" range (defined as 1 to 3 times / second), and 45° is classified into the "drastic change" angle difference range (defined as >30°). If the frequency of the map block number T002 is 4 times / second and the average angle is 12°, then the frequency is "high frequency" and the angle is "slight change". The directional frequency and angle difference value labels of each map block are output to form a set of map block directional change fluctuation values.

[0174] The frequency range of direction change is defined as low, medium, and high frequency by counting the number of times the direction code of the map block switches per unit time.

[0175] The angle difference threshold is set according to the angle difference between adjacent direction codes, and is set as micro change, medium change and drastic change.

[0176] S502: Based on the tile orientation change fluctuation value, extract the number of boundary change pixels and the center point position offset of the tile in the corresponding image frame, calculate the boundary change speed and spatial movement amplitude per unit time, construct a tile consistency score including orientation change frequency, angle jump fluctuation, boundary speed and centroid displacement, and compare it with the consistency judgment benchmark to generate a tile consistency deviation value.

[0177] Based on the aforementioned set of directional change fluctuation values, the number of boundary change pixels and the center point offset are extracted for each patch within the current image frame set. The number of boundary change pixels is obtained by comparing the pixel count of the patch outline in the preceding and following frames. For example, if the number of boundary pixels in frame t is 2200 and in frame t+1 it is 2250, then the change is 50 pixels. The center point offset is calculated using the Euclidean distance between the centroid coordinates of the frames. Let the centroid of frame t be (256.2, 340.5) and in frame t+1 be (258.4, 339.7), then the offset is: The average of the total changes in all frames per second yields the boundary change rate and center displacement amplitude. For example, if the cumulative boundary change in 50 frames is 3100 pixels, the rate is 3100 / 2 = 1550 pixels / second, and the average center displacement is 2.5 pixels / frame. The obtained four parameters—direction frequency, angle fluctuation, boundary speed, and center displacement—are compared with the consistency judgment benchmark. The benchmark is frequency ≤ 3 times / second, angle fluctuation ≤ 15°, boundary speed ≤ 60 pixels / second, and displacement ≤ 3 pixels / frame. If the four values ​​of a certain patch are 4.2 times / second, 36°, 1550 pixels / second, and 2.5 pixels / frame, then the direction frequency, angle fluctuation, and boundary speed exceed the benchmark, which is recorded as the consistency deviation item number 3. A consistency scoring mechanism is constructed with a score of 0 to 1. For each item exceeding the benchmark, 0.25 is deducted. The final consistency score is 1 - 0.75 = 0.25, corresponding to a deviation value of 0.75. See the table below for specific data examples.

[0178] Table 5. Statistics on the Deviation of Patch Consistency Indicators

[0179]

[0180] As shown in Table 5, block T002 has a deviation value of 0.75 due to several indicators deviating from the consistency benchmark. This value will be used in the next step to determine whether the block label needs to be updated.

[0181] The consistency judgment benchmark extracts four indicators: direction change frequency, angle jump amplitude, boundary velocity, and centroid displacement. Combined with the statistical results of 1000 stable patch samples, it is set as a multi-dimensional reference interval, where the direction frequency is set to ≤3 times / second, the angle fluctuation is set to ≤15°, the boundary velocity is set to ≤60 pixels / second, and the displacement is set to ≤3 pixels / frame.

[0182] S503: Based on the generated tile consistency deviation value, all tiles are screened sequentially, and tile numbers exceeding the 0.25 threshold are extracted using the following formula: Calculate the consistency deviation index for each map tile, replace the original labels, reassign the label values ​​to the corresponding semantic classification identifiers, and update the number correspondence of all map tiles to obtain the label index results;

[0183] in, P represents the consistency deviation index of the i-th tile, where i represents the tile number. ik P represents the pixel value of the k-th pixel within the i-th tile, where k represents the pixel index within the tile. i C represents the average pixel value of all pixels in the i-th patch, n represents the total number of pixels in the i-th patch, and C represents the average pixel value of all pixels in the i-th patch. ij C represents the color feature value of the j-th feature point within the i-th tile, where j represents the index of the feature point within the tile. N w represents the average color feature value of all tiles adjacent to the i-th tile, m represents the total number of feature points in the i-th tile, and w represents the average color feature value of all tiles adjacent to the i-th tile. s Spatial location weight factor

[0184] Based on the generated tile consistency deviation value, all tiles are screened sequentially. Tile numbers with deviation values ​​exceeding the 0.25 threshold are extracted. For example, T002 and T003 both exceed this threshold. Their original semantic labels are replaced with new classification labels. For example, if the original label is "upper limb activity area", it is changed to "dynamic abnormal area". After updating the label values, the tile number-label correspondence table is reconstructed. At the same time, the area mapped by the corresponding tile number in the image annotation matrix is ​​updated synchronously. The new label annotation values ​​are written to the pixel-level label map, completing the semantic annotation replacement and number mapping reconstruction of the tiles. Finally, a complete label index result set is formed for subsequent image behavior analysis processes.

[0185] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0186] It should be understood that the term "and / or" in this article merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0187] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0188] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0189] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0190] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0191] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0192] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0194] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0195] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for processing video images in a monitoring room based on image semantic segmentation, characterized in that, The method includes: S1: Acquire video images of the monitoring room through monitoring camera equipment, divide the image semantic regions, obtain pixel label maps of the patient's head, upper limbs, lower limbs and bed, calculate the pixel displacement of semantically consistent regions in adjacent frames, encode the change trajectory through angle transformation, and output the direction features of the image patches; S2: Call the tile orientation features, analyze the changing trend of tile orientation trajectory, identify semantic tiles that deviate from the normal range, analyze the duration of deviation, filter semantic tiles that show abnormal motion features, and perform a balanced comparison of pixel orientation distribution, filter out disordered orientation areas and erase interference, and output clean image data. S3: Based on the cleaned image data, analyze the pixel connection order of the erased boundary, fit the boundary curve between edge pixels, sort and reorganize the connection boundary according to the contour law of the closed area, and output the reconstructed boundary data. S4: Call the reconstructed boundary data, compare it with the shape changes of the corresponding semantic tile boundary, construct the outline trend of the target area, evaluate the fusion degree through the consistency of the deformation direction of the boundary connection points, perform path segmentation and boundary redrawing on the tile structure, and output updated tile labels.

2. The method for processing intensive care unit video images based on image semantic segmentation according to claim 1, characterized in that, The tile orientation features include orientation change angle, orientation conversion frequency, and orientation distribution density. The cleaned image data includes abnormal tile identifiers, orientation cluttered area masks, and interference removal layers. The reconstructed boundary data includes edge connection order, curve fitting shape, and closed region topology. The updated tile labels include structural segmentation paths, boundary reconstruction shapes, and label mapping rules.

3. The method for processing intensive care unit video images based on image semantic segmentation according to claim 1, characterized in that, The specific steps of S1 are as follows: S101: Acquire video images of the monitoring room through monitoring camera equipment, perform semantic region division based on each frame of image, subdivide the human body region in the image into head, upper limbs, lower limbs and bed body, generate corresponding category label images in pixels, construct pixel distribution information of semantic blocks, and obtain the distribution of semantic block label image data. S102: Based on the distribution of the semantic block label image data, extract the edge pixel coordinates of semantically consistent regions in adjacent images, call the edge pixel position of the corresponding semantic block, compare the changes in horizontal and vertical coordinate values, calculate the displacement vector of the edge pixels in the corresponding region, and obtain the semantic region pixel offset displacement value. S103: Based on the pixel offset displacement value of the semantic region, perform angle transformation on the pixel movement trajectory of the semantic region, call the coordinate change between the trajectory endpoint and the starting point, calculate the motion direction angle value, and encode the trajectory angle transformation results in multiple regions into a tile direction classification to generate tile direction features.

4. The method for processing intensive care unit video images based on image semantic segmentation according to claim 3, characterized in that, The specific steps of S2 are as follows: S201: Call the tile orientation feature, calculate the orientation change amplitude value of the tile based on the orientation encoding sequence of semantically consistent tiles in the image frame, and count the orientation jump frequency per unit time. Compare the difference between the change frequency and the normal orientation change frequency range, mark the tile number that deviates from the normal range, and obtain the orientation offset interval value. The normal direction change frequency range refers to the judgment benchmark range set by combining the number of image frames and the number of changes in the direction encoding of the blocks per unit time, based on the upper and lower bounds of the direction change frequency obtained statistically under the state of no abnormal motion. S202: Based on the directional offset interval value, extract the directional offset time period of the marked blocks, record the offset frame number of each block and compare it with the directional offset duration reference value, filter the block numbers whose time is greater than the reference value, construct an abnormal motion block set, and generate a directional abnormal block number set. The reference value for directional offset duration refers to the distribution of the number of consecutive frames when a semantic patch without directional clutter occurs in an image frame, and the upper quartile of the distribution is extracted as a threshold setting. S203: Based on the set of directional anomaly patch numbers, extract the directional distribution values ​​of the corresponding region pixels, calculate the dispersion of the directional distribution within the patch, and filter regions where the directional dispersion is greater than the directional balance threshold. Erase interference by clearing the directional coding, mark the erased region and boundary information, reconstruct the directional coding distribution map of the image frame, and obtain the purified image data. The orientation equalization threshold is set by statistically sorting the standard deviation values ​​of pixel orientation codes in semantic blocks that do not exhibit orientation clutter features, and selecting the standard deviation value corresponding to the 85th percentile in the sorting results.

5. The method for processing intensive care unit video images based on image semantic segmentation according to claim 4, characterized in that, The specific formula for calculating the dispersion of directional distribution within the plot is as follows: Calculate the directional dispersion value; Among them, D u This represents the directional dispersion value of the u-th patch. α represents the directional angle distribution value of the v-th pixel in the u-th patch. u m represents the weighted average of all directional angle values ​​in the u-th tile. u μ represents the number of pixels contained in the u-th patch. u ν represents the mean gradient of the edge response of the u-th patch. u ζ represents the orientation variance factor of the boundary pixels in the u-th patch. u This represents the local contrast gain factor of the u-th patch, where v is the pixel index and u is the patch number index.

6. The method for processing intensive care unit video images based on image semantic segmentation according to claim 4, characterized in that, The specific steps for S3 are as follows: S301: Based on the cleaned image data, extract the pixel set of the erased boundary, scan according to the spatial distribution order of the pixels in the image plane, record the adjacency relationship of each group of edge pixels and mark the sequence number, and obtain the pixel connection sequence number value. S302: Based on the pixel connection sequence number value and the coordinate difference between adjacent pixels, fit the continuous curve shape between multiple edge segments, extract the curvature change value and connection angle between edge connection segments, construct the curve segment sequence of each group of erased edges, and generate edge pixel fitting curve values. S303: Call the edge pixel fitting curve value, identify the boundary contour direction features in the closed area, combine and sort the edge curve segments according to the priority sorting criterion of contour continuity, update the complete boundary contour structure of the corresponding area, and output the reconstructed boundary data. The "contour continuity priority" refers to the practice of selecting the closest segment with the smallest angle change for connection when sorting boundary curve segments by comparing the Euclidean distance between the endpoints of adjacent segments and the angle between the connection direction.

7. The method for processing intensive care unit video images based on image semantic segmentation according to claim 6, characterized in that, The specific steps for S4 are as follows: S401: Call the reconstructed boundary data, extract the node set of the current block boundary according to the original boundary coordinates of the semantic block in the image frame, compare the position of the current node with the corresponding original boundary node, obtain the spatial offset vector of each node, and generate the boundary deformation direction change value by calculating the trend of the node displacement direction sequence. S402: Based on the boundary deformation direction change value, identify the consistency of direction change between boundary connection points, calculate the direction continuity rate of the boundary point sequence in each block, compare the direction continuity rate value with the boundary connection reference value, filter the block positions that are lower than the boundary connection reference value, and obtain the boundary fusion consistency coefficient. The boundary connection benchmark value is determined by statistically sorting the directional continuity rate of the boundary point sequence in the semantic block without directional clutter features, and selecting the value corresponding to the 80th percentile of all continuity rate samples as the set value. S403: Based on the boundary fusion consistency coefficient, locate the tile region where the boundary connection direction changes abruptly, re-divide the internal boundary path, and re-label the tile boundary position according to the direction gradient of the boundary turning point, update the spatial range distribution label of the semantic tile in the image, and obtain the updated tile label.

8. The method for processing intensive care unit video images based on image semantic segmentation according to claim 7, characterized in that, The specific formula for calculating the directional continuity rate of the boundary point sequence within each tile is as follows: Calculate the directional continuity rate value; Among them, R i θ represents the directional continuity of the sequence of boundary points within the i-th tile. i,j θ represents the tangent direction angle at the j-th boundary point in the i-th tile. i,j+1 δ represents the tangent direction angle value of the (j+1)th boundary point in the i-th tile. i λ represents the maximum difference in the direction changes of all boundary points in the i-th tile. i ε represents the arithmetic mean of the directional changes of all boundary points in the i-th tile. i ρ represents the pixel density value per unit area within the boundary region of the i-th tile boundary point. i n represents the boundary texture complexity index in the region of the i-th tile boundary point. i This represents the total number of boundary points in the i-th tile, where i is the tile index number used to identify different tiles, and j is the sequence index number of the boundary points within the tile.

9. The method for processing intensive care unit video images based on image semantic segmentation according to claim 1, characterized in that, The method further includes: S5: Call the updated tile label, analyze the stability of the tile semantic label by combining the direction change, boundary change speed and spatial movement of the tile within the corresponding time period, filter out tiles that do not meet the consistency conditions, perform label correction, and output the label index result; The specific results of the tag index are the directional stability level, tag consistency score, and spatiotemporal offset index.

10. The method for processing intensive care unit video images based on image semantic segmentation according to claim 9, characterized in that, The specific steps of S5 are as follows: S501: Call the updated tile label, scan the direction encoding sequence of each tile in the corresponding time period, count the number of direction switching times and the change angle fluctuation value per unit time, analyze according to the direction change frequency range and the angle difference threshold, and obtain the tile direction change fluctuation value. The frequency range of direction change is defined as low, medium, and high frequency by counting the number of times the direction code of the map block switches per unit time. The angle difference threshold is set according to the angle difference between adjacent direction codes, and is set as micro change, medium change and drastic change; S502: Based on the tile orientation change fluctuation value, extract the number of boundary change pixels and the center point position offset of the tile in the corresponding image frame, calculate the boundary change speed and spatial movement amplitude per unit time, construct a tile consistency score including orientation change frequency, angle jump fluctuation, boundary speed and centroid displacement, and compare it with the consistency judgment benchmark to generate a tile consistency deviation value. The consistency judgment benchmark extracts four indicators: direction change frequency, angle jump amplitude, boundary velocity, and centroid displacement. Combined with the statistical results of 1000 stable tile samples, it is set as a multi-dimensional reference interval, where the direction frequency is set to ≤3 times / second, the angle fluctuation is set to ≤15°, the boundary velocity is set to ≤60 pixels / second, and the displacement is set to ≤3 pixels / frame. S503: Based on the block consistency deviation value, filter the block numbers whose deviation value exceeds the consistency judgment benchmark, replace the original labels, reassign the label values ​​to the corresponding semantic classification identifiers, and update the number correspondence of all blocks to obtain the label index result.

Images