Campus video processing method and system based on intelligent image recognition

Abstract

This invention relates to the field of intelligent video surveillance technology, specifically disclosing a campus video processing method and system based on intelligent image recognition. The method involves acquiring indoor video streams on campus and extracting a set of visible posture anchor points; obtaining the geometric boundaries of fixed obstacles as environmental constraints; inputting the posture anchor points and environmental constraints into an implicit distribution generation process; obtaining multiple sets of lower limb complete posture sequences through sampling and segment-by-segment extension; evaluating the temporal coherence of candidate sequences in consecutive frames; selecting the optimal results and splicing them to form a complete human posture temporal chain; extracting the angle change temporal sequence and matching it with a pre-stored abnormal pattern fluctuation envelope; and outputting an early warning based on the matching probability. This invention can reasonably restore lower limb postures in environments obstructed by desks, improving the reliability of identifying abnormal behaviors such as falls and slips.

Description

Technical Field

[0001] This invention relates to the field of intelligent video surveillance technology, specifically to a campus video processing method and system based on intelligent image recognition. Background Technology

[0002] With the development of artificial intelligence and computer vision technologies, image recognition-based campus video surveillance is gradually shifting from manual patrols to automated analysis. Existing technologies commonly employ campus video processing methods including: using object detection algorithms to identify the locations of students, teachers, and other personnel; utilizing facial recognition for identity authentication and attendance tracking; and extracting key points of the human skeleton through pose estimation techniques and combining them with temporal networks to identify and warn of abnormal behaviors such as falls, fights, and gatherings. Among these, skeletal key point-based behavior recognition methods (such as OpenPose, HRNet combined with LSTM or temporal convolutional networks) have been tested and applied in scenarios such as classrooms and corridors.

[0003] The existing technology has the following shortcomings:

[0004] In classroom settings, students' lower limbs (such as legs, knees, and feet) are often obscured by desks, making it difficult for existing video surveillance systems to capture the full body posture (especially the lower body). Therefore, when students exhibit abnormal behaviors such as falling or slipping off their chairs, the system easily misses these actions because it cannot see the complete movement, failing to issue timely warnings. Summary of the Invention

[0005] The purpose of this invention is to provide a campus video processing method and system based on intelligent image recognition to solve the problems mentioned above.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A campus video processing method based on intelligent image recognition includes the following steps:

[0008] S1: Collect continuous video streams of indoor scenes on campus, extract the spatial coordinates and confidence scores of unobstructed parts of the human body frame by frame, and form a set of visible pose anchor points.

[0009] S2, acquire the geometric boundary data of fixed obstacles in the scene, as environmental attitude constraints;

[0010] S3 inputs the set of visible pose anchor points and environmental pose constraints into the implicit distribution generation process. By sampling the poses of the occluded parts multiple times, several sets of lower limb completion pose sequences are obtained in the current frame.

[0011] S4. The temporal coherence of the lower limb completion posture sequence corresponding to multiple consecutive frames is evaluated. Based on the smoothness of joint displacement between adjacent frames and the degree of satisfaction of environmental posture constraints, the optimal completion result is selected from the candidate sequences of each frame and spliced ​​to form a complete human posture temporal chain.

[0012] S5 inputs the complete human posture time sequence chain into the behavior discrimination network, calculates its matching probability with various abnormal motion patterns, and outputs an early warning signal and saves the associated video clips when the probability exceeds a preset threshold.

[0013] As a further aspect of the present invention: S1 specifically includes:

[0014] Perform a difference operation on two adjacent frames in a continuous video stream to obtain pixel-level motion heatmaps;

[0015] The gradient magnitude and direction of each pixel are calculated within the motion hot zone, and sliding window matching is performed according to the predefined geometric template of human body parts to output multiple candidate joint positions.

[0016] Based on the frequency of occurrence and spatial consistency of candidate joints within the time window, jitter outliers are eliminated, and visible attitude anchors with high confidence are retained.

[0017] As a further aspect of the present invention: obtaining pixel-level motion thermal areas specifically includes:

[0018] The current frame and the previous frame are each divided into local windows of equal size. The difference in the total brightness of all pixels in each window is calculated to obtain the window-level motion intensity.

[0019] Based on the difference in motion intensity between each window and its eight neighboring windows, the binarization truncation threshold within the window is adaptively adjusted.

[0020] For pixel locations exceeding the truncation threshold, neighborhood connectivity is marked, and the pixels within the connected region are assigned grayscale contrast weights with the center pixel, outputting pixel-level motion hotspots.

[0021] As a further aspect of the present invention: S2 specifically includes:

[0022] Perform temporal median calculation on multiple consecutive video frames to eliminate moving target interference and output a static background image;

[0023] Detect pixel value jumps along horizontal scan lines on a static background image, record the jump positions, and connect them as candidate edge segments;

[0024] Candidate edge segments are filtered based on the length of the line connecting their two endpoints projected onto the ground. Segments with a length exceeding a preset threshold are retained, and the coordinate set of these segments is used as an environmental attitude constraint.

[0025] As a further aspect of the present invention: the output static background image specifically includes:

[0026] Accumulate the continuous brightness value sequence of each pixel position on the time axis, and calculate the sum of the absolute values ​​of the brightness differences between adjacent pixels in the sequence as the fluctuation feature value;

[0027] Based on the preset range of the fluctuation feature value, the number of participating frames corresponding to each pixel position is determined; the greater the fluctuation, the more participating frames.

[0028] Within a continuous frame window corresponding to each pixel position, the brightness values ​​are sorted by size and the brightness value at the middle position is selected. The middle brightness values ​​of all pixel positions constitute a static background image.

[0029] As a further aspect of the present invention: S3 specifically includes:

[0030] Connect the head and shoulder key points of the visible posture anchor point set to form the upper limb reference baseline, and calculate the spatial orientation angle of the upper limb reference baseline.

[0031] Based on the boundary position of the front edge of the desk in the environmental posture constraints, the distributable area of ​​the hip key points is determined, and multiple candidate hip positions are randomly generated within the distribution area.

[0032] Each candidate hip position is extended segment by segment from the end of the upper limb reference baseline according to the proportion of human joint length, and the coordinates of the knee and ankle joints are determined in turn. Each set of extension results corresponds to a set of lower limb completion posture sequences.

[0033] As a further aspect of the present invention: the calculation of the spatial orientation angle of the upper limb reference baseline specifically includes:

[0034] The centroid coordinates of the head region are extracted from the visible posture anchor point set as head anchor points, and the centroid coordinates of the left and right acromion regions are extracted as shoulder anchor points.

[0035] Connect the geometric midpoints of the head anchor point and the shoulder anchor points, and use the projection direction of the corresponding line in the vertical plane as the direction vector of the upper limb reference baseline.

[0036] Calculate the deviation arc between the direction vector and the preset normal to the horizontal ground plane of the classroom, and use the deviation arc as the spatial orientation angle of the upper limb reference baseline.

[0037] As a further aspect of the present invention: S4 specifically includes:

[0038] For each candidate lower limb complete posture sequence, calculate and sum the spatial movement distance of all relevant nodes between two adjacent frames to obtain the total displacement of the lower limb complete posture sequence.

[0039] For the ankle joint in the lower limb completed posture sequence, calculate the minimum interval between it and the ground constraint boundary. At the same time, calculate the penetration depth between the knee joint and the lower edge constraint boundary of the table. Add the minimum interval and the penetration depth to obtain the constraint violation value.

[0040] The reciprocal of the total displacement is weighted and combined with the constraint violation value, and the candidate sequence with the smallest weighted result is selected as the optimal completion result for the current frame.

[0041] As a further aspect of the present invention: S5 specifically includes:

[0042] Extract the spatial angles between the lines connecting adjacent joints of the torso and limbs from the complete human posture time sequence to form an angle change time sequence.

[0043] For each abnormal motion pattern, the upper and lower allowable boundaries of its angle change time sequence are pre-stored to form a fluctuation envelope.

[0044] Calculate the percentage of the cumulative duration of the actual angle change time series falling within the fluctuation envelope, and output the percentage of the cumulative duration as the matching probability.

[0045] A campus video processing system based on intelligent image recognition includes:

[0046] The visible pose extraction module collects continuous video streams of indoor scenes on campus, extracts the spatial coordinates and confidence scores of the unobstructed parts of the human body frame by frame, and forms a set of visible pose anchor points.

[0047] The environmental constraint construction module acquires the geometric boundary data of fixed obstacles in the scene, which serves as environmental attitude constraint conditions.

[0048] The lower limb pose completion module inputs the set of visible pose anchor points and environmental pose constraints into the implicit distribution generation process. By sampling the poses of occluded parts multiple times, it obtains several sets of lower limb pose completion sequences for the current frame.

[0049] The temporal optimization and splicing module evaluates the temporal coherence of the lower limb completed posture sequence corresponding to multiple consecutive frames. Based on the smoothness of joint displacement between adjacent frames and the degree of satisfaction of environmental posture constraints, it selects the optimal completion result from the candidate sequences of each frame and splices them to form a complete human posture temporal chain.

[0050] The behavior warning output module inputs the complete human posture time sequence chain into the behavior discrimination network, calculates its matching probability with various abnormal movement patterns, and outputs a warning signal and saves the associated video clip when the probability exceeds a preset threshold.

[0051] The beneficial effects of this invention are:

[0052] (1) This invention extracts visible upper limb posture anchor points and combines them with the geometric boundaries of fixed obstacles such as desks and the ground as constraints. It generates multiple sets of lower limb completion candidate sequences by implicit sampling and segment-by-segment extension, and then selects the best sequence for splicing after temporal coherence evaluation. This can reasonably restore the lower limb movement trajectory obscured by the desktop in a classroom with high occlusion. Compared with existing methods that rely solely on visible skeletons, this invention effectively solves the problem of missed detection of abnormal behaviors such as falls and slips caused by missing lower limb key points, and improves the reliability of behavior recognition in complex classroom scenarios.

[0053] (2) This invention uses a time sequence of angle changes to match the pre-stored fluctuation envelope band frame by frame and joint by joint, and uses the cumulative duration ratio as the matching probability output. This eliminates the need to build a complex deep classification network and reduces the dependence on large-scale labeled samples. At the same time, by selecting the best from multiple candidate sequences through a weighted combination of total displacement and constraint violation value, it effectively suppresses unreasonable results such as jitter and penetration caused by single-frame completion, so that the warning output has good stability and interpretability, and is easy for campus security personnel to understand and verify. Attached Figure Description

[0054] The invention will now be further described with reference to the accompanying drawings.

[0055] Figure 1 This is a flowchart of the method of the present invention;

[0056] Figure 2 This is a system block diagram of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please see Figure 1 As shown, this invention is a campus video processing method based on intelligent image recognition, comprising the following steps:

[0059] S1: Collect continuous video streams of indoor scenes on campus, extract the spatial coordinates and confidence scores of unobstructed parts of the human body frame by frame, and form a set of visible pose anchor points.

[0060] S2, acquire the geometric boundary data of fixed obstacles in the scene, as environmental attitude constraints;

[0061] S3 inputs the set of visible pose anchor points and environmental pose constraints into the implicit distribution generation process. By sampling the poses of the occluded parts multiple times, several sets of lower limb completion pose sequences are obtained in the current frame.

[0062] S4. The temporal coherence of the lower limb completion posture sequence corresponding to multiple consecutive frames is evaluated. Based on the smoothness of joint displacement between adjacent frames and the degree of satisfaction of environmental posture constraints, the optimal completion result is selected from the candidate sequences of each frame and spliced ​​to form a complete human posture temporal chain.

[0063] S5 inputs the complete human posture time sequence chain into the behavior discrimination network, calculates its matching probability with various abnormal motion patterns, and outputs an early warning signal and saves the associated video clips when the probability exceeds a preset threshold.

[0064] In S1, continuous video streams of indoor campus scenes are acquired, and the spatial coordinates and confidence scores of unobstructed parts of the human body are extracted frame by frame to form a set of visible pose anchor points, specifically including:

[0065] First, a visible light video stream of the indoor classroom scene is continuously captured at a rate of 25 frames per second using fixed network cameras deployed on the front or upper side walls of the classroom. The captured video stream only covers the student seating area and aisle, does not involve areas sensitive to personal privacy, and is used within the scope permitted by the school's security management regulations. After median filtering and noise reduction on each captured raw image frame, the following steps are performed to obtain a set of visible pose anchor points.

[0066] Obtaining pixel-level motion heatmaps: For two consecutive frames (the current frame and the previous frame), horizontally divide them into equal-sized local windows, each 8 pixels wide and 8 pixels high. Calculate the sum of the brightness values ​​of all pixels within each window. Subtract the sum of the brightness values ​​of the corresponding window in the previous frame from the sum of the brightness values ​​of the current frame window, and take the absolute value to obtain the motion intensity of that window. Then, for each window, obtain the motion intensity values ​​of its eight neighboring windows, calculate the average of the motion intensities of these nine windows, and multiply this average by a preset coefficient of 0.8 as the adaptive binarization truncation threshold for that window. Next, within each window, compare the absolute value of the difference between the original brightness value of each pixel and the brightness value of the corresponding pixel in the previous frame. If this absolute value exceeds the aforementioned truncation threshold, the pixel is marked as a candidate moving pixel. Perform eight-neighbor connected component analysis on all candidate moving pixels, marking the set of pixels that are connected to each other as a connected region. For each connected region, the absolute value of the difference between the gray value of each pixel in the region and the gray value of the center pixel of the region is calculated. The ratio obtained by dividing the absolute value by the maximum gray value difference in the region is used as the gray-level contrast weight of the pixel. This weight value is then assigned to the corresponding pixel, and finally a pixel-level motion heat map is output, where the value of each pixel represents the heat index of its motion intensity and local contrast.

[0067] Output multiple candidate joint point locations: Within the pixel-level motion heatmap, calculate the gradient magnitude and gradient direction for each pixel. Specifically, for a pixel location, subtract the brightness value of the left pixel from the brightness value of the pixel to its right as the horizontal gradient, and subtract the brightness value of the upper pixel from the brightness value of the pixel below as the vertical gradient. The square root of the sum of the squares of the horizontal and vertical gradients is the gradient magnitude, and the gradient direction is the vertical gradient divided by the arctangent of the horizontal gradient. Then, predefine geometric templates corresponding to human head, shoulder, elbow, wrist, etc., each template consisting of a set of edge segments with specific relative positions and angles. Slide the template pixel by pixel on the motion heatmap image, calculating the consistency score between the gradient direction of all pixels within the template coverage area and the template direction each time. When the consistency score exceeds a preset threshold of 0.65, record the center position of the template as a candidate joint point.

[0068] Retaining high-confidence visible pose anchors: For each candidate joint, count its occurrence frequency in the most recent 15 consecutive frames. If a candidate joint appears less than 10 times in 15 frames, it is identified as a jitter outlier and removed. For the retained candidate joints, calculate the average of their spatial coordinates within the time window, and use this average as the final joint position. Then, calculate whether the Euclidean distance between joints in the same frame conforms to the normal proportions of the human body (e.g., the ratio of shoulder width to head length is between 1.5 and 2.5), and further remove joints that do not conform to the proportion constraints. All remaining joints and their corresponding confidence scores (i.e., the product of consistency score and frequency of occurrence) together constitute the set of visible pose anchors.

[0069] In S2, the geometric boundary data of fixed obstacles in the scene are acquired as environmental pose constraints, specifically including:

[0070] By deploying the same fixed network camera in the classroom, while continuously acquiring video streams, the following steps are performed to obtain geometric boundary data of fixed obstacles in the scene (such as desks, chair backs, windowsills, etc.), which serve as environmental attitude constraints for subsequent steps.

[0071] Output static background image: For 30 consecutively acquired video frames, record the brightness value sequence of each pixel location on the time axis. First, calculate the absolute value of the brightness difference between two adjacent frames in the sequence, and sum the absolute values ​​of all adjacent differences to obtain the fluctuation characteristic value of that pixel location. Based on three pre-divided intervals: fluctuation characteristic values ​​less than 50 are classified as low fluctuation intervals, values ​​between 50 and 150 as medium fluctuation intervals, and values ​​greater than 150 as high fluctuation intervals. For pixels in the low fluctuation interval, the most recent 10 frames are used for calculation; for pixels in the medium fluctuation interval, the most recent 20 frames are used; and for pixels in the high fluctuation interval, all 30 frames are used. Then, within the consecutive frame window corresponding to each pixel, sort the brightness values ​​of all frames in the window from smallest to largest, and select the brightness value located in the middle position after sorting (i.e., the (frame number + 1) / 2th position) as the background brightness of that pixel. The above operation is performed sequentially on all pixel positions, and the resulting set of background brightness constitutes a static background image. Moving targets (such as walking students) contribute more of the median value of the frames due to their large brightness fluctuations, and are thus effectively eliminated.

[0072] Obtaining candidate edge segments: On a static background image, scan horizontally line by line starting from the top. For each line, compare the brightness values ​​of two adjacent pixels. If the absolute value of the brightness difference is greater than 20, record that position as a transition point. Record all transition points in a line consecutively, considering the interval between any two adjacent transition points as an edge segment, and record the coordinates of the midpoint of that segment. Then, connect the midpoints of adjacent lines that are horizontally less than 5 pixels apart and vertically consecutive to form a line segment. For each line segment, calculate its length (i.e., the number of midpoints it contains). If the length exceeds 10 pixels, retain it as a candidate edge segment.

[0073] Environmental pose constraints are obtained through screening: For each candidate edge segment, the image coordinates of its two endpoints are acquired. Based on the camera calibration parameters (a ground projection matrix obtained beforehand using a checkerboard calibration method), the image coordinates of each endpoint are converted into projected coordinates on the ground. The straight-line distance between the two projected coordinates is calculated. If this distance is greater than 30 centimeters (a threshold preset based on the actual width of the desk), the segment is retained; otherwise, it is discarded. The set of endpoint coordinates of all retained segments is sorted from top to bottom and left to right according to their position in the image, forming the environmental pose constraints. These constraints specifically characterize the projected boundaries of fixed obstacles such as the front edge of the desk and the edge of the seat on the ground, and are used to subsequently determine whether the key points of the human lower limbs exceed the reasonable range of movement.

[0074] In S3, the set of visible pose anchor points and environmental pose constraints are input into the implicit distribution generation process. Through multiple candidate sampling of the pose of occluded parts, several sets of lower limb completion pose sequences are obtained for the current frame, specifically including:

[0075] To construct an upper limb reference baseline and calculate its spatial orientation angle: First, extract all pixel coordinates of the head region from the set of visible posture anchor points, and calculate the arithmetic mean of these coordinates as the head anchor point. Extract the pixel coordinates of the left and right acromion regions respectively, and calculate their centroid coordinates to obtain the left and right shoulder anchor points. Then, calculate the geometric midpoint of the left and right shoulder anchor points, i.e., half the sum of the x-coordinates of the left and right shoulder anchor points as the midpoint x-coordinate, and half the sum of the y-coordinates of the left and right shoulder anchor points as the midpoint y-coordinate. Connect the head anchor point to this geometric midpoint with a straight line segment, and take the projection direction of this straight line segment in a vertical plane perpendicular to the image plane as the direction vector of the upper limb reference baseline. Pre-calibrate the normal direction of the classroom's horizontal ground plane (i.e., a unit vector perpendicular to the ground upwards). Calculate the angle in radians between the direction vector and the ground plane normal; this radian value is the spatial orientation angle of the upper limb reference baseline. Specifically, the following formula is used for calculation: ;

[0076] in, Indicates spatial orientation angle, in radians; The direction vector of the upper limb reference baseline; This is the preset normal vector to the horizontal ground plane of the classroom; This represents the dot product operation of two vectors, which is the sum of the products of their corresponding components; and These represent the magnitudes of the two vectors, i.e., the square root of the sum of the squares of their components. When the included angle in radians is greater than 0.5 radians, it indicates that the student's torso is leaning significantly forward or backward.

[0077] Determine the potential hip distribution area and generate multiple candidate hip locations: Extract the coordinates of the boundary line segment of the desk's front edge from the environmental posture constraints. Take the average of the ground projection coordinates of the two endpoints of this line segment as the center point of the desk's front edge. Using this center point as a reference, extend 15 cm to the left and right, 10 cm forward (towards the seat), and 5 cm backward to form a rectangular area as the potential distribution area for hip key points. Within this area, generate 20 candidate points using a uniform random sampling method. The x-coordinate of each candidate point is randomly selected between the left and right boundaries of the rectangle, and the y-coordinate (depth direction) is randomly selected between the front and rear boundaries. Each candidate point corresponds to a candidate hip location.

[0078] The coordinates of the knee and ankle joints are determined segment by segment: For each candidate hip position, the starting point is first the end point of the upper limb reference baseline (i.e., the geometric midpoint of the shoulder anchor points). The distance from the starting point to the candidate hip position is recorded as the trunk length. The knee and ankle joints are determined sequentially according to the standard human joint length ratio (thigh length is 0.55 times the trunk length, and lower leg length is 0.9 times the thigh length). The specific process is as follows: Starting from the candidate hip position, extend the thigh length vertically downwards to obtain the candidate knee joint point; then extend the lower leg length vertically downwards from the candidate knee joint point to obtain the candidate ankle joint point. Then, the extension direction is fine-tuned according to the spatial orientation angle of the current frame: if the spatial orientation angle is greater than 0.3 radians, the coordinates of the knee and ankle joints are shifted forward in the body direction by an offset equal to the corresponding length multiplied by the tangent of the angle. Finally, a complete set of lower limb coordinates is obtained: hip, knee, and ankle. Repeat the above operation for 20 candidate hip positions to obtain 20 sets of lower limb completion posture sequences. Each set of sequences contains the two-dimensional image coordinates of the three key points of hip, knee and ankle in the current frame.

[0079] Output several sets of lower limb completed pose sequences: Arrange the calculated hip, knee, and ankle coordinates of each set in sequence and merge them with the upper limb keypoints in the existing set of visible pose anchor points to form a complete set of human pose hypotheses for the current frame. All 20 sets of hypotheses together constitute several sets of lower limb completed pose sequences for the current frame, which are used in subsequent temporal evaluation steps. During the generation process, if a candidate hip position causes the knee or ankle joint to extend beyond the image boundary or significantly penetrate the desk plane (i.e., the coordinates are within the desk boundary line and the depth value is less than the depth of the desk's front edge), the sequence is marked as invalid and discarded, while valid sequences are retained for further processing.

[0080] In S4, the temporal coherence of the lower limb completed pose sequences corresponding to multiple consecutive frames is evaluated. Based on the smoothness of joint displacements between adjacent frames and the degree to which environmental pose constraints are met, the optimal completion result is selected from the candidate sequences of each frame and spliced ​​together to form a complete human pose temporal chain, specifically including:

[0081] Calculate the total displacement of each candidate lower limb completed pose sequence: For each candidate lower limb completed pose sequence in the current frame, compare it with the selected optimal completion result in the previous frame. Extract the image coordinates of the hip, knee, and ankle joints in the candidate sequence of the current frame, and the image coordinates of the corresponding three joints in the optimal result of the previous frame. For each joint, calculate the absolute value of the difference between the horizontal coordinates of the current frame and the previous frame, and add the absolute value of the difference between the vertical coordinates to obtain the single-frame movement distance of that joint. Add the single-frame movement distances of the hip, knee, and ankle joints to obtain the single-frame total displacement of the current candidate sequence relative to the previous frame. Then, sum the single-frame total displacements of the candidate sequence in the current frame with those of the three consecutive frames before it (if any), and the sum is the total displacement of the candidate lower limb completed pose sequence. The smaller the total displacement, the smoother the motion of the sequence on the time axis.

[0082] Calculate the constraint violation value for each candidate lower limb completed pose sequence: For a candidate lower limb completed pose sequence in the current frame, first obtain the ankle joint coordinates. Based on the pre-defined ground boundary line (composed of ground projection line segments extracted from the static background image) in the environmental pose constraints, calculate the vertical distance from the ankle joint coordinates to the nearest ground boundary line. This distance is the minimum interval between the ankle joint and the ground constraint boundary. If the ankle joint is within the allowable range of the ground boundary line, the minimum interval is positive; if it exceeds the allowable range, the minimum interval is negative. Then, obtain the knee joint coordinates in the candidate sequence. Based on the boundary line segment of the lower edge of the desk in the environmental pose constraints (obtained through the aforementioned edge detection), calculate the shortest vertical distance from the knee joint coordinates to this boundary line. If the knee joint is inside the lower edge constraint boundary of the desk (i.e., penetrating the desk), the absolute value of this distance is recorded as the penetration depth; if it does not penetrate, the penetration depth is zero. Finally, add the minimum interval (or its absolute value if negative) to the penetration depth to obtain the constraint violation value of the candidate sequence. The smaller the constraint violation value, the more the sequence satisfies the environmental attitude constraints.

[0083] Weighted combination and selection of the optimal completion result: For each candidate lower limb completed pose sequence, calculate the reciprocal of its total displacement. Specifically: divide the number 1 by the previously obtained total displacement (if the total displacement is zero, the reciprocal is taken as a preset large number, such as 10000). Multiply the reciprocal of the total displacement by the first weighting coefficient 0.4 to obtain the first product; multiply the constraint violation value by the second weighting coefficient 0.6 to obtain the second product; add the first product and the second product to obtain the weighted result. Calculate the weighted result for all candidate sequences in the current frame, and select the candidate sequence with the smallest weighted result as the optimal completion result for that frame. Concatenate the optimal completion results obtained from multiple consecutive frames in chronological order, using the output of the previous frame as the input reference for the next frame, ultimately forming a continuous and complete human pose temporal chain. Each moment in this temporal chain contains the coordinates of three lower limb key points: hip, knee, and ankle, which are merged with the visible upper limb key points for subsequent behavior recognition.

[0084] In S5, the complete human pose temporal sequence is input into the behavior discrimination network, and its matching probability with various abnormal motion patterns is calculated. When the probability exceeds a preset threshold, an early warning signal is output and the associated video segment is saved. Specifically, this includes:

[0085] Extracting the temporal sequence of angle changes: From the complete human pose temporal sequence, for each frame, determine the connecting lines between adjacent joints corresponding to the torso, left thigh, left calf, right thigh, and right calf. Specifically, the torso connection is taken from the line connecting the geometric midpoint of the shoulder anchor points to the hip keypoint; the left thigh connection is taken from the line connecting the hip keypoint to the left knee keypoint; the left calf connection is taken from the line connecting the left knee keypoint to the left ankle keypoint; the right thigh connection is taken from the line connecting the hip keypoint to the right knee keypoint; and the right calf connection is taken from the line connecting the right knee keypoint to the right ankle keypoint. For each connection, calculate its angle with the vertical direction (i.e., the vertically upward direction of the image), with the angle ranging from 0 to 180 degrees. Arrange the angles of all five connections within the same frame in a fixed order to obtain a five-dimensional angle vector. Repeat this operation for multiple consecutive frames, concatenating the angle vectors of all frames in chronological order to form the temporal sequence of angle changes.

[0086] Forming the fluctuation envelope of abnormal motion patterns: For each abnormal motion pattern to be identified (e.g., falling, fighting, squatting, climbing), the upper and lower allowable boundaries of its angle change time sequence are pre-stored in the following way: Collect 50 labeled video clips of this pattern, each clip being 2 seconds long (corresponding to 50 frames). For each sample, extract its angle change time sequence using the method in step one. For each frame number (from 1 to 50) and each joint connection index (1 to 5), count the maximum and minimum values ​​of the 50 samples at the angle between that connection in that frame, which are used as the upper and lower allowable boundaries, respectively. The set of upper boundaries for all frames and all connections is called the upper allowable boundary curve, and the set of lower boundaries is called the lower allowable boundary curve. The area between the two is the fluctuation envelope of this abnormal motion pattern.

[0087] Calculate the matching probability and output an early warning: For the current actual angle change time sequence to be identified (the length is the same as the pre-stored sequence length, both being 50 frames), determine whether the actual included angle value falls within the fluctuation envelope corresponding to the pattern by connecting each joint frame by frame. Specifically, for the included angle of the first connection in the first frame, if its value is greater than or equal to the lower allowable boundary and less than or equal to the upper allowable boundary, it is recorded as a hit; otherwise, it is recorded as a miss. Perform the above operation on all frames (50 frames in total) and all connections (5 lines in total), a total of 250 judgment positions, and count the number of hits. Divide the number of hits by 250 to obtain the cumulative duration percentage, which is the matching probability of the current actual posture sequence with the abnormal motion pattern. A threshold is preset (e.g., 0.75). When the matching probability exceeds the threshold, an early warning signal is output. At the same time, a 10-second video segment (5 seconds before and 5 seconds after the current moment) is extracted from the original video stream, saved in compressed format to a designated storage location for security personnel to review. If the matching probability does not exceed the threshold, no warning will be issued, and subsequent video frames will continue to be processed.

[0088] Please see Figure 2 As shown, the campus video processing system based on intelligent image recognition includes:

[0089] The visible pose extraction module collects continuous video streams of indoor scenes on campus, extracts the spatial coordinates and confidence scores of the unobstructed parts of the human body frame by frame, and forms a set of visible pose anchor points.

[0090] The environmental constraint construction module acquires the geometric boundary data of fixed obstacles in the scene, which serves as environmental attitude constraint conditions.

[0091] The lower limb pose completion module inputs the set of visible pose anchor points and environmental pose constraints into the implicit distribution generation process. By sampling the poses of occluded parts multiple times, it obtains several sets of lower limb pose completion sequences for the current frame.

[0092] The temporal optimization and splicing module evaluates the temporal coherence of the lower limb completed posture sequence corresponding to multiple consecutive frames. Based on the smoothness of joint displacement between adjacent frames and the degree of satisfaction of environmental posture constraints, it selects the optimal completion result from the candidate sequences of each frame and splices them to form a complete human posture temporal chain.

[0093] The behavior warning output module inputs the complete human posture time sequence chain into the behavior discrimination network, calculates its matching probability with various abnormal movement patterns, and outputs a warning signal and saves the associated video clip when the probability exceeds a preset threshold.

[0094] The working principle of this invention is as follows: By acquiring continuous video streams of indoor campus scenes, and through motion heatmap analysis, gradient template matching, and temporal consistency filtering, the spatial coordinates and confidence scores of unobstructed parts of the human body are extracted frame by frame to form a set of visible posture anchor points. Simultaneously, geometric boundary data of fixed obstacles in the scene are obtained as environmental posture constraints through temporal median calculation, edge detection, and ground projection filtering. The set of visible posture anchor points and environmental posture constraints are input into an implicit distribution generation process. This involves constructing an upper limb reference baseline and calculating the spatial orientation angle, randomly sampling candidate hip positions within the hip distribution area, and then extending the range segment by segment according to the proportion of human joint length. Fine-tuning is performed to obtain several sets of lower limb completion posture sequences for the current frame. Then, the temporal coherence of the lower limb completion posture sequences for multiple consecutive frames is evaluated, and the total displacement and constraint violation value of each candidate sequence are calculated. The optimal completion result of each frame is selected by weighted combination and spliced ​​to form a complete human posture temporal chain. Finally, the spatial angle between the lines connecting the torso and the adjacent joints of each limb is extracted from the complete human posture temporal chain to form an angle change temporal sequence. This sequence is matched frame by frame and joint by joint with the pre-stored fluctuation envelope of various abnormal motion patterns. The cumulative duration ratio is calculated as the matching probability. When the probability exceeds a preset threshold, an early warning signal is output and the associated video segment is saved.

[0095] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A campus video processing method based on intelligent image recognition, characterized in that, Includes the following steps: S1: Collect continuous video streams of indoor scenes on campus, extract the spatial coordinates and confidence scores of unobstructed parts of the human body frame by frame, and form a set of visible pose anchor points. S2, acquire the geometric boundary data of fixed obstacles in the scene, as environmental attitude constraints; S3 inputs the set of visible pose anchor points and environmental pose constraints into the implicit distribution generation process. By sampling the poses of the occluded parts multiple times, several sets of lower limb completion pose sequences are obtained in the current frame. S4. The temporal coherence of the lower limb completion posture sequence corresponding to multiple consecutive frames is evaluated. Based on the smoothness of joint displacement between adjacent frames and the degree of satisfaction of environmental posture constraints, the optimal completion result is selected from the candidate sequences of each frame and spliced ​​to form a complete human posture temporal chain. S5 inputs the complete human posture time sequence chain into the behavior discrimination network, calculates its matching probability with various abnormal motion patterns, and outputs an early warning signal and saves the associated video clips when the probability exceeds a preset threshold.

2. The campus video processing method based on intelligent image recognition according to claim 1, characterized in that, S1 specifically includes: Perform a difference operation on two adjacent frames in a continuous video stream to obtain pixel-level motion heatmaps; The gradient magnitude and direction of each pixel are calculated within the motion hot zone, and sliding window matching is performed according to the predefined geometric template of human body parts to output multiple candidate joint positions. Based on the frequency of occurrence and spatial consistency of candidate joints within the time window, jitter outliers are eliminated, and visible attitude anchors with high confidence are retained.

3. The campus video processing method based on intelligent image recognition according to claim 2, characterized in that, The acquisition of pixel-level motion heat zones specifically includes: The current frame and the previous frame are each divided into local windows of equal size. The difference in the total brightness of all pixels in each window is calculated to obtain the window-level motion intensity. Based on the difference in motion intensity between each window and its eight neighboring windows, the binarization truncation threshold within the window is adaptively adjusted. For pixel locations exceeding the truncation threshold, neighborhood connectivity is marked, and the pixels within the connected region are assigned grayscale contrast weights with the center pixel, outputting pixel-level motion hotspots.

4. The campus video processing method based on intelligent image recognition according to claim 1, characterized in that, S2 specifically includes: Perform temporal median calculation on multiple consecutive video frames to eliminate moving target interference and output a static background image; Detect pixel value jumps along horizontal scan lines on a static background image, record the jump positions, and connect them as candidate edge segments; Candidate edge segments are filtered based on the length of the line connecting their two endpoints projected onto the ground. Segments with a length exceeding a preset threshold are retained, and the coordinate set of these segments is used as an environmental attitude constraint.

5. The campus video processing method based on intelligent image recognition according to claim 4, characterized in that, The output static background image specifically includes: Accumulate the continuous brightness value sequence of each pixel position on the time axis, and calculate the sum of the absolute values ​​of the brightness differences between adjacent pixels in the sequence as the fluctuation feature value; Based on the preset range of the fluctuation feature value, the number of participating frames corresponding to each pixel position is determined; the greater the fluctuation, the more participating frames. Within a continuous frame window corresponding to each pixel position, the brightness values ​​are sorted by size and the brightness value at the middle position is selected. The middle brightness values ​​of all pixel positions constitute a static background image.

6. The campus video processing method based on intelligent image recognition according to claim 1, characterized in that, S3 specifically includes: Connect the head and shoulder key points of the visible posture anchor point set to form the upper limb reference baseline, and calculate the spatial orientation angle of the upper limb reference baseline. Based on the boundary position of the front edge of the desk in the environmental posture constraints, the distributable area of ​​the hip key points is determined, and multiple candidate hip positions are randomly generated within the distribution area. Each candidate hip position is extended segment by segment from the end of the upper limb reference baseline according to the proportion of human joint length, and the coordinates of the knee and ankle joints are determined in turn. Each set of extension results corresponds to a set of lower limb completion posture sequences.

7. The campus video processing method based on intelligent image recognition according to claim 6, characterized in that, The calculation of the spatial orientation angle of the upper limb reference baseline specifically includes: The centroid coordinates of the head region are extracted from the visible posture anchor point set as head anchor points, and the centroid coordinates of the left and right acromion regions are extracted as shoulder anchor points. Connect the geometric midpoints of the head anchor point and the shoulder anchor points, and use the projection direction of the corresponding line in the vertical plane as the direction vector of the upper limb reference baseline. Calculate the deviation arc between the direction vector and the preset normal to the horizontal ground plane of the classroom, and use the deviation arc as the spatial orientation angle of the upper limb reference baseline.

8. The campus video processing method based on intelligent image recognition according to claim 1, characterized in that, S4 specifically includes: For each candidate lower limb complete posture sequence, calculate and sum the spatial movement distance of all relevant nodes between two adjacent frames to obtain the total displacement of the lower limb complete posture sequence. For the ankle joint in the lower limb completed posture sequence, calculate the minimum interval between it and the ground constraint boundary. At the same time, calculate the penetration depth between the knee joint and the lower edge constraint boundary of the table. Add the minimum interval and the penetration depth to obtain the constraint violation value. The reciprocal of the total displacement is weighted and combined with the constraint violation value, and the candidate sequence with the smallest weighted result is selected as the optimal completion result for the current frame.

9. The campus video processing method based on intelligent image recognition according to claim 1, characterized in that, S5 specifically includes: Extract the spatial angles between the lines connecting adjacent joints of the torso and limbs from the complete human posture time sequence to form an angle change time sequence. For each abnormal motion pattern, the upper and lower allowable boundaries of its angle change time sequence are pre-stored to form a fluctuation envelope. Calculate the percentage of the cumulative duration of the actual angle change time series falling within the fluctuation envelope, and output the percentage of the cumulative duration as the matching probability.

10. A campus video processing system based on intelligent image recognition, characterized in that, The campus video processing method based on intelligent image recognition as described in any one of claims 1-9 includes: The visible pose extraction module collects continuous video streams of indoor scenes on campus, extracts the spatial coordinates and confidence scores of the unobstructed parts of the human body frame by frame, and forms a set of visible pose anchor points. The environmental constraint construction module acquires the geometric boundary data of fixed obstacles in the scene, which serves as environmental attitude constraint conditions. The lower limb pose completion module inputs the set of visible pose anchor points and environmental pose constraints into the implicit distribution generation process. By sampling the poses of occluded parts multiple times, it obtains several sets of lower limb pose completion sequences for the current frame. The temporal optimization and splicing module evaluates the temporal coherence of the lower limb completed posture sequence corresponding to multiple consecutive frames. Based on the smoothness of joint displacement between adjacent frames and the degree of satisfaction of environmental posture constraints, it selects the optimal completion result from the candidate sequences of each frame and splices them to form a complete human posture temporal chain. The behavior warning output module inputs the complete human posture time sequence chain into the behavior discrimination network, calculates its matching probability with various abnormal movement patterns, and outputs a warning signal and saves the associated video clip when the probability exceeds a preset threshold.

Images