Video image-based system for recognizing stereotyped behaviors of children with autism spectrum disorder

Abstract

This invention relates to the field of image recognition technology, specifically to a system for recognizing stereotyped behaviors in children with autism spectrum disorder based on video images. The system filters target pixels from a pool of pixels; determines a motion intensity index based on the trajectory point sequence of the target pixels; converts the trajectory point sequence of the target pixels into a motion vector sequence; determines directional similarity based on the directional similarity between vectors; clusters the vectors based on directional similarity and identifies target clusters; determines a motion regularity index based on the number of vectors and directional similarity within the target cluster; and determines a saliency index in conjunction with the motion intensity index; clusters the target pixels based on the saliency index and sets regions of interest; determines enhancement weights based on the target pixel density and saliency index within the region; and determines the total enhancement requirement value in conjunction with the grayscale and gradient features of the pixels within the region; and maps the total enhancement requirement value back to gamma correction parameters, thereby improving the accuracy of recognition.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a system for recognizing stereotyped behaviors in children with autism spectrum disorder based on video images. Background Technology

[0002] Video-based stereotyped behavior recognition for childhood autism spectrum disorder (ASD) is an important tool for observing and analyzing behavior using computer vision technology. Its core lies in detecting and identifying repetitive, ritualistic stereotyped movements exhibited by children from videos collected in natural settings (such as homes and classrooms), such as repeated hand clapping, body swaying, and spinning in place.

[0003] Currently, existing automated video image recognition technologies typically apply behavior recognition models directly to the captured video. However, in real-world natural scenes, lighting conditions are complex and varied (e.g., dimly lit corners indoors), and children with ASD are often sensitive to light, potentially choosing to be in environments with alternating light and dark or poor lighting. This often results in captured video images that are generally or locally too dark, or have blurred details. Furthermore, existing video enhancement techniques (such as global histogram equalization) are mostly "blind enhancements," meaning they process all regions uniformly, failing to specifically improve the quality of motion regions crucial for behavior recognition. This leads to a significant decrease in the performance of subsequent pose estimation-based behavior recognition models, resulting in missed detections or false positives. Summary of the Invention

[0004] To address the technical problem that acquired video images often suffer from overall or partial darkness and blurred details, leading to missed detections or misjudgments in subsequent stereotyped behavior recognition, this invention provides a video image-based stereotyped behavior recognition system for children with autism spectrum disorder. The specific technical solution adopted is as follows: This invention proposes a video image-based system for recognizing stereotyped behaviors in children with autism spectrum disorder. The system includes: The acquisition module is used to acquire video sequences of the target object's behavior in natural scenes; and to extract reference time periods centered on each image frame. The first determining module is used to select candidate pixels from the pixels for each image frame; obtain the motion trajectory point sequence by optical flow tracing within its reference time period; filter out candidate pixels that do not meet the preset conditions to determine the target pixel; and determine the motion intensity index based on the trajectory point sequence of the target pixel. The second determining module is used to convert the trajectory point sequence of target pixels into a motion vector sequence; determine the directional similarity based on the directional similarity between vectors; cluster the vectors based on the directional similarity and identify the target clusters; determine the motion regularity index based on the number of vectors and directional similarity within the target clusters; and determine the significance index in combination with the motion intensity index. The correction module is used to cluster target pixels based on saliency index, and set the region corresponding to each cluster as the region of interest; for each region of interest, the enhancement weight is determined according to the target pixel density and saliency index in the region; and the total enhancement requirement value is determined by combining the pixel gray level and gradient features in the region; and the total enhancement requirement value is mapped inversely to the gamma correction parameters of the image frame. The recognition module is used to enhance video sequences based on gamma correction parameters to obtain enhanced video sequences; and to identify repetitive motion patterns from the enhanced video sequences.

[0005] Furthermore, the reference time period includes multiple consecutive image frames centered on each image frame; The extraction of reference time periods centered on each image frame includes: For each image frame in the behavioral video sequence, a predetermined number of image frames are extracted from both sides of the image frame, with the image frame as the center, to form the target image frame sequence. If the number of image frames before or after an image frame does not meet the preset number, the image frames on the side where the number of image frames meets the preset number are extracted, with the image frame as the center, to form the target image frame sequence; wherein, the reference time period is used to provide the target image frame sequence for each pixel in the image frame.

[0006] Further, the step of filtering candidate pixels from the pixels includes: For each pixel in each image frame, take the pixel as the center, obtain the gray values ​​of all pixels within a local neighborhood window of a preset size; calculate the variance of the gray values ​​of all pixels within the local neighborhood window, and use it as the spatial feature change value of the pixel. Based on the target image frame sequence corresponding to the pixel, the gray value of the pixel coordinate in each frame of the target image frame sequence is obtained to form a gray value sequence; the variance of the gray value sequence is calculated as the temporal feature change value of the pixel. The spatial feature change values ​​and temporal feature change values ​​are normalized respectively; and the arithmetic mean of the normalized spatial feature change values ​​and the normalized temporal feature change values ​​is calculated as the comprehensive significance. Sort all pixels in the image frame from highest to lowest based on their overall saliency value; select the top-ranked pixels by a predetermined percentage as candidate pixels.

[0007] Furthermore, each data point in the motion trajectory point sequence records the two-dimensional coordinates of the pixel in any image frame within the reference time period, and all data points in the motion trajectory point sequence are arranged in chronological order according to their corresponding times; the target pixel determination process includes: Calculate the total number of data points contained in the motion trajectory point sequence; If the total number of data points is less than the first preset threshold, the motion trajectory duration of the corresponding candidate pixel is determined to be insufficient, and it is recorded as the first candidate pixel. Calculate the distance between two data points that are adjacent in time sequence in the motion trajectory point sequence; calculate the arithmetic mean of the distances between all adjacent data point pairs in the motion trajectory point sequence as the average distance; If the average distance is less than the second preset threshold, the spatial motion significance of the corresponding candidate pixel is determined to be insufficient, and it is recorded as the second candidate pixel. Remove the first and second candidate pixels from all candidate pixels in the image frame to obtain the target pixel.

[0008] Furthermore, the process of determining the exercise intensity index includes: For each target pixel, a point set is constructed on the image plane based on all two-dimensional coordinates in the motion trajectory point sequence of the target pixel; the area of ​​the smallest convex polygon that completely contains the point set is calculated as the spatial coverage. Calculate the straight-line distance between each pair of adjacent data points according to the time order of the data points in the motion trajectory point sequence; calculate the arithmetic mean of the straight-line distances between all adjacent point pairs as the average displacement amplitude; Divide the spatial coverage by the size of the image frame containing the target pixel to obtain the normalized spatial coverage; divide the average displacement amplitude by the diagonal length of the image frame containing the target pixel to obtain the normalized average displacement amplitude. The initial intensity value is obtained by adding the normalized spatial coverage measurement to the normalized average displacement amplitude. The maximum and minimum values ​​are determined from the initial intensity values ​​of all target pixels in the image frame; based on the maximum and minimum values, the initial intensity values ​​of the target pixels are linearly scaled to the range of 0 to 1 to obtain the motion intensity index of the target pixels.

[0009] Furthermore, the motion vector sequence consists of a set of displacement vectors arranged in chronological order; the target cluster identification process includes: Calculate the cosine similarity between any two displacement vectors in the motion vector sequence, and take the absolute value of the cosine similarity as the directional similarity between any two displacement vectors. Subtracting the directional similarity from the positive integer 1 yields the dissimilarity distance between any two displacement vectors. Based on dissimilarity distance, cluster analysis is performed on all motion vectors in the motion vector sequence to divide them into at least one cluster. The cluster with the most displacement vectors among all clusters is selected as the target cluster.

[0010] Furthermore, the process of determining the significance index includes: The arithmetic mean of the directional similarity between all displacement vector pairs within the target cluster is normalized to obtain the first sub-index. The ratio of the number of displacement vectors contained in the target cluster to the total number of displacement vectors in the motion vector sequence is normalized to obtain the second sub-index. Calculate the arithmetic mean of the first sub-indicator and the second sub-indicator as an indicator of motion regularity; The arithmetic mean of the motion regularity index and the motion intensity index is calculated and used as the significance index of the target pixel.

[0011] Furthermore, the clustering of target pixels based on saliency indicators, and setting the region corresponding to each cluster as the region of interest, includes: For each image frame in the behavioral video sequence, based on the two-dimensional coordinates of each target pixel in the image frame, a spatial clustering algorithm is used to aggregate them into different clusters. The saliency index of the target pixel is used to adjust the distance metric in the clustering process. For each cluster, based on the coordinate distribution of all target pixels contained in it, a continuous image region covering all target pixels contained in the cluster is determined and marked as a region of interest.

[0012] Furthermore, the enhancement weight determination process includes: Count the total number of target pixels contained in the region of interest; calculate the pixel area occupied by the region of interest in the image; divide the total number of target pixels by the pixel area of ​​the region to obtain the original density value; Calculate the arithmetic mean of the significance indexes corresponding to all target pixels within the region of interest, and use it as the first statistical value; find the maximum value among all significance indexes within the region of interest, and use it as the second statistical value; average the first statistical value and the second statistical value to obtain the original significance value. Based on all regions of interest in the image frame, their original density values ​​are collected to form a first set, and their original saliency values ​​are collected to form a second set. For each region of interest, the original density value of the region of interest is linearly scaled based on the maximum and minimum values ​​in the first set to obtain a normalized density value. Based on the maximum and minimum values ​​in the second set, the original significance values ​​of the region of interest are linearly scaled to obtain normalized significance values; The enhanced weights are obtained by averaging the normalized density values ​​and the normalized significance values.

[0013] Furthermore, the process for determining the total enhancement requirement value includes: Calculate the average grayscale value of all pixels within each region of interest; classify regions with an average grayscale value less than a preset global brightness threshold as low-brightness regions. The arithmetic mean of the gradient magnitudes at all pixels within each region of interest is used as the original sharpness of each region of interest; the original sharpness of each region of interest is subtracted from the positive integer 1 to obtain the blur factor. For low-brightness areas, the first enhancement requirement value is obtained by using a preset first enhancement requirement value calculation function based on the difference between the positive integer 1 and the normalized grayscale average value, the enhancement weight, and the normalized blur factor. Calculate the sum of the first enhancement demand values, which is the total enhancement demand value.

[0014] The present invention has the following beneficial effects: This invention does not perform global uniform processing on the video. Instead, it uses a first determination module and a second determination module to deeply analyze motion features at the pixel level to determine "salience indicators" that characterize the probability of specific repetitive actions. Then, the correction module uses the "salience indicators" as guidance to delineate "areas of interest" that require focused processing. Subsequently, it corrects only the brightness and contrast of these areas, effectively avoiding over-processing of irrelevant areas such as the background and concentrating resources on improving the quality of key information. The correction module back-maps the "total enhancement requirement value" obtained based on content analysis to "gamma correction parameters," enabling the system to dynamically generate optimal enhancement strategies for each frame of the image. For example, it can selectively brighten overly dark areas, thereby restoring motion details lost due to lighting issues without excessively introducing noise. This significantly improves the quality of the "enhanced video sequence" received by the subsequent recognition module, making the posture and trajectory information of key actions clearer and more complete, thus improving the accuracy and reliability of the final "repetitive action pattern" recognition. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic diagram of a video image-based system for recognizing stereotyped behaviors in children with autism spectrum disorder, provided as an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the process of determining exercise intensity indicators according to an embodiment of the present invention; Figure 3This is a schematic diagram illustrating the process of determining the total enhancement requirement value according to an embodiment of the present invention. Detailed Implementation

[0017] The following description, in conjunction with the accompanying drawings, details a specific solution for a video image-based system for recognizing stereotyped behaviors in children with autism spectrum disorder, provided by this invention.

[0018] Please see Figure 1 The diagram illustrates a structural schematic of a video image-based stereotype recognition system for childhood autism spectrum disorder, according to an embodiment of the present invention. The system includes: The acquisition module 101 is used to acquire video sequences of the target object's behavior in natural scenes; and to extract reference time periods centered on each image frame.

[0019] The target object refers specifically to the individual whose behavioral patterns need to be analyzed and identified.

[0020] Natural scenes refer to the actual physical environment in which the target object engages in daily activities, without any artificial lighting or staging. Examples include the living room and bedroom in a home, and the classroom and activity room in a school.

[0021] In this context, it can be understood that behavioral video sequences are data composed of two-dimensional image frames that are continuously captured and digitized by image acquisition devices (such as surveillance cameras) and arranged in chronological order.

[0022] It should be noted that the reference time period includes multiple consecutive image frames centered on each image frame.

[0023] In this embodiment, for each image frame in the behavioral video sequence, a preset number of image frames are extracted from both sides of the image frame before and after the image frame, forming a target image frame sequence; if the number of image frames before or after the image frame does not meet the preset number, image frames from the side where the number of image frames meets the preset number are extracted, forming a target image frame sequence; wherein, the reference time period is used to provide a target image frame sequence for each pixel in the image frame.

[0024] It should be noted that the specific value of the preset number can be determined according to the actual situation, and this embodiment does not impose a specific limitation. For example, the preset number can be 100 frames.

[0025] For example, for most image frames in a video sequence, if there are consecutive frames before and after the current image frame that meet the preset number (let's say M), then the current image frame is taken as the center frame, and M consecutive frames are extracted from before it and M consecutive frames are extracted from after it. These (M+1+M) frames together form the target image frame sequence of the center frame. If the number of image frames before or after the current image frame does not meet the preset number, for example, if the number of image frames before the current image frame is less than the preset number, then the current image frame is still taken as the center frame, and M consecutive frames are extracted from after it. These (1+M) frames together form the target image frame sequence of the center frame.

[0026] It can be understood that the reference time period defines the time window range used to analyze each image frame in the video sequence; the target image frame sequence is physical data, which is an ordered set of all image frames belonging to the reference time period that are actually extracted from the video sequence according to the above image frame extraction rules.

[0027] The first determining module 102 is used to select candidate pixels from the pixels for each image frame; obtain the motion trajectory point sequence by optical flow tracing within its reference time period; filter out candidate pixels that do not meet the preset conditions to determine the target pixel; and determine the motion intensity index based on the trajectory point sequence of the target pixel.

[0028] It should be noted that each image frame has undergone preprocessing, such as histogram equalization.

[0029] It is important to understand that since natural scene videos contain hundreds of thousands to millions of pixels, most of which are static backgrounds or irrelevant textures, performing complex motion trajectory tracking and analysis on all pixels would generate huge computational overhead and introduce a lot of noise interference. Therefore, candidate pixels are selected from all pixels contained in each image frame to quickly locate the positions of pixels that have significant texture in space and show obvious grayscale changes in time. These positions are more likely to correspond to key parts of moving objects (such as joints and contours).

[0030] In this embodiment, for each pixel in each image frame, the grayscale values ​​of all pixels within a local neighborhood window of a preset size are obtained, centered on the pixel; the variance of the grayscale values ​​of all pixels within the local neighborhood window is calculated as the spatial feature change value of the pixel; based on the target image frame sequence corresponding to the pixel, the grayscale values ​​of the pixel coordinates in each frame of the target image frame sequence are obtained to form a grayscale value sequence; the variance of the grayscale value sequence is calculated as the temporal feature change value of the pixel; the spatial feature change value and the temporal feature change value are normalized respectively; and the arithmetic mean of the normalized spatial feature change value and the normalized temporal feature change value is calculated as the comprehensive significance; all pixels in the image frame are sorted from high to low according to their comprehensive significance value; and the pixels with the highest ranking (a preset proportion) are determined as candidate pixels.

[0031] It should be noted that the specific value of the preset size is usually determined based on a balance between experience and computational efficiency, and this embodiment does not impose a specific limitation. For example, 11×11 pixels is a commonly used balanced size in computer vision.

[0032] It should be noted that the method for calculating variance is a publicly available technique, and will not be described in detail in this embodiment.

[0033] Spatial feature variation value measures the dispersion of grayscale value distribution within a local region centered on the currently analyzed pixel. A larger spatial feature variation value in the local neighborhood centered on a given pixel indicates a greater difference in pixel grayscale within that neighborhood, suggesting the presence of obvious edges, corners, or rich textures. This reflects the likelihood that the pixel is located near object boundaries, facial contours, finger joints, or other areas with prominent features.

[0034] The pixel coordinates refer to an absolute, fixed position in the image, specified by the (x, y) coordinates of the pixel in the image coordinate system.

[0035] For example, suppose a pixel has coordinates (100, 200) in frame i, and the target image frame sequence contains frames i-2, i-1, i, i+1, and i+2 (a total of 5 target image frame sequences). Then, in frame i-2, the pixel grayscale value at coordinates (100, 200) is read and denoted as g1; in frame i-1, the pixel grayscale value at coordinates (100, 200) is read and denoted as g2; in frame i, the pixel grayscale value at coordinates (100, 200) is read and denoted as g3; in frame i+1, the pixel grayscale value at coordinates (100, 200) is read and denoted as g4; and in frame i+2, the pixel grayscale value at coordinates (100, 200) is read and denoted as g5. Finally, a grayscale value sequence [g1, g2, g3, g4, g5] is formed.

[0036] Temporal feature variation values ​​measure the degree of fluctuation in grayscale value at the same fixed pixel coordinates over time. Specifically, the larger the temporal feature variation value of a pixel, the more drastic and repeated the brightness changes at that pixel coordinates over time. This strongly suggests that a moving object has repeatedly passed by or covered the pixel (for example, a waving hand repeatedly sweeps across the pixel, causing the pixel brightness to switch between the hand and the background).

[0037] It should be noted that the spatial and temporal feature change values ​​can be normalized using a minimum-maximum normalization method, which is a technique well-known to those skilled in the art. Specifically, for spatial feature change values, the maximum and minimum values ​​of all pixels in the current image frame are calculated, and the spatial feature change value of each pixel is mapped to the interval [0, 1]. For temporal feature change values, the same method is used to map the maximum and minimum values ​​of the temporal feature change values ​​of all pixels in the current image frame to the interval [0, 1]. If the maximum value equals the minimum value, the normalization result is uniformly set to 0.

[0038] It's important to understand that a smaller normalized spatial feature change value for a pixel indicates that the pixel is more likely to be located in a flat texture region, lacking unique spatial features for stable tracking. Conversely, a smaller normalized temporal feature change value for a pixel indicates that the pixel's brightness is more stable within the reference time period, suggesting it's more likely to be in a static background without significant motion. Therefore, if one parameter is small, even if the other is large, the arithmetic mean will be lowered, resulting in low overall saliency. This means the system will filter out pixels with texture but located in a static background, and pixels that are moving but have blurred textures. Only when both the normalized spatial and temporal feature change values ​​for a pixel are large will the overall saliency be high, indicating that the pixel is both a spatially significant feature point (such as an edge) and is indeed undergoing significant motion.

[0039] It should be noted that the specific value of the preset ratio can be determined based on industry experience, and this embodiment does not impose a specific limitation. For example, a typical ratio value is 15%.

[0040] It should be noted that the specific process of tracking based on optical flow is a well-known technique in the art, and will not be described in detail in this embodiment.

[0041] For example, for each selected candidate pixel, its coordinates in the current image frame are used as the starting point for tracking. Within the reference time period, forward tracking and backward tracking are performed simultaneously, starting from the current frame: Forward tracking refers to calculating the position coordinates of the candidate pixel in subsequent consecutive frames frame by frame, based on optical flow, starting from the current frame, until the end frame of the reference time period is reached or the tracking point exceeds the image boundary; Backward tracking refers to calculating the position coordinates of the candidate pixel in previous consecutive frames frame by frame, starting from the current frame, based on optical flow, until the start frame of the reference time period is reached or the tracking point exceeds the image boundary; All position coordinates obtained from forward and backward tracking are arranged in chronological order to form a sequence of motion trajectory points of the candidate pixel within the reference time period.

[0042] It should be noted that each data point in the motion trajectory point sequence records the two-dimensional coordinates of the pixel in any image frame within the reference time period, and all data points in the motion trajectory point sequence are arranged in chronological order according to their corresponding times.

[0043] In this embodiment, the total number of data points in the motion trajectory point sequence is calculated; if the total number of data points is less than a first preset threshold, the motion trajectory time persistence of the corresponding candidate pixel is determined to be insufficient, and it is recorded as the first candidate pixel; the distance between two data points that are adjacent in time order in the motion trajectory point sequence is calculated; the arithmetic mean of the distances between all adjacent data point pairs in the motion trajectory point sequence is calculated as the average distance; if the average distance is less than a second preset threshold, the motion trajectory spatial motion significance of the corresponding candidate pixel is determined to be insufficient, and it is recorded as the second candidate pixel; the first candidate pixel and the second candidate pixel are deleted from all candidate pixels in the image frame to obtain the target pixel.

[0044] It should be noted that the first preset threshold is designed to filter out transient noise. The specific value of the first preset threshold is usually set based on the video frame rate and experience to ensure that effective and meaningful actions should last for a certain duration. For example, if it is believed that a repetitive action cycle should last at least about 0.3 seconds, then the minimum number of frames is calculated based on the frame rate (e.g., 30fps × 0.3 seconds = 9 frames, rounded up to 10 frames), so the first preset threshold is set to 10 frames.

[0045] It should be noted that calculating the distance between two data points that are adjacent in time sequence in the motion trajectory point sequence is equivalent to calculating their Euclidean distance on the two-dimensional plane of the image. The specific calculation process of the Euclidean distance will not be described in this embodiment.

[0046] Average distance quantifies the average pixel length that a pixel moves per frame within a reference time period, directly reflecting its motion amplitude. The larger the average distance of a pixel, the faster its movement speed or the greater its motion amplitude, indicating a violent and large-scale movement.

[0047] It should be noted that the second preset threshold is intended to filter out minor movements and can be used as a percentage of the image size to ensure this; this embodiment does not impose a specific limitation. For example, setting it to 1% of the image diagonal length would result in approximately 22 pixels for a 1080p image (approximately 2200 pixels diagonally).

[0048] Optionally, to further improve the analysis accuracy, after obtaining the target pixels through the above-mentioned screening based on trajectory length and average distance, a secondary screening step based on motion vector sequence can be included. This step can be a preferred embodiment of the method: for each target pixel that passes the initial screening, its motion trajectory point sequence is converted into a motion vector sequence; the amplitude of each vector in the motion vector sequence is calculated, and a very small amplitude threshold is set (for example, corresponding to a displacement of less than 1 pixel in the image). Vectors with amplitudes less than the amplitude threshold are marked as zero vectors and removed from the motion vector sequence; the number of vectors remaining in the motion vector sequence after removing zero vectors is counted; the target pixels with a remaining number of vectors not less than a third preset threshold are retained to form the final set of target pixels for subsequent motion regularity analysis.

[0049] It should be noted that the conversion method for the motion trajectory point sequence to the motion vector sequence is as follows: according to the time order, subtract the coordinates of the previous position point from the coordinates of the next position point in the motion trajectory point sequence to obtain the displacement vector from one moment to the next moment. All displacement vectors are arranged in time order to form the motion vector sequence.

[0050] It is understandable that setting an amplitude threshold aims to filter out data that is not meaningful for analysis, resulting from sensor noise or extremely minor jitter.

[0051] It should be noted that the specific value of the third preset threshold can be determined based on actual needs to avoid insufficient data for analysis; this embodiment does not impose a specific limitation. For example, the typical range of the third preset threshold is 5 to 10.

[0052] It's important to understand that the visual salience of an action depends not only on whether it is moving, but also on the amplitude and range of its movement. Furthermore, a fast, wide-ranging action (such as a forceful wave) is more easily perceived and recognized than a slow, small-amplitude action (such as a slight finger tremor), and is often associated with more typical stereotyped behaviors. Therefore, motion intensity metrics can be used to quantify and evaluate the visual intensity of the action represented by each target pixel from two dimensions: motion speed and spatial coverage.

[0053] The process of determining exercise intensity index is as follows Figure 2 As shown, it includes: S101-1: For each target pixel, construct a point set on the image plane based on all two-dimensional coordinates in the motion trajectory point sequence of the target pixel; calculate the area of ​​the smallest convex polygon that completely contains the point set, as the spatial coverage.

[0054] It should be noted that calculating the area of ​​the smallest convex polygon is a publicly available technique, which will not be described in detail in this embodiment. For example, firstly, an algorithm (such as Graham Scan) is used to find all the vertices that form the boundary of the convex hull in the point set, and they are arranged in clockwise or counterclockwise order; then, the shoelace formula is applied, and the coordinates of these vertices are substituted into a specific summation formula to calculate the area of ​​the smallest convex polygon (in square pixels).

[0055] Spatial coverage measures the size of the spatial range occupied by the motion trajectory of a target pixel on the image plane. The larger the spatial coverage of a target pixel, the wider the spatial distribution of its motion trajectory and the greater the amplitude of the motion, such as waving a hand or walking.

[0056] S101-2: Calculate the straight-line distance between each pair of adjacent data points according to the time order of the data points in the motion trajectory point sequence; calculate the arithmetic mean of the straight-line distances between all adjacent point pairs as the average displacement amplitude.

[0057] In this context, it can be understood that calculating the straight-line distance between each pair of adjacent data points is equivalent to calculating their Euclidean distance (in pixels) on the two-dimensional image plane.

[0058] The average displacement amplitude quantifies the average step size of a target pixel's motion within a unit of time (i.e., frame interval). A larger average displacement amplitude indicates a faster and more rapid movement of the target pixel.

[0059] S101-3: Divide the spatial coverage by the size of the image frame containing the target pixel to obtain the normalized spatial coverage; divide the average displacement amplitude by the diagonal length of the image frame containing the target pixel to obtain the normalized average displacement amplitude.

[0060] It is understandable that the total pixel area of ​​the image frame (i.e., the size of the image frame) is also in square pixels, so the normalized spatial coverage is a dimensionless value.

[0061] It is understandable that the unit of the diagonal length of the image frame is also pixels, so the normalized average displacement amplitude is a dimensionless value.

[0062] It should be noted that the dimensions (width and height) of the image frame are positive integers (e.g., 1920×1080) preset by the camera sensor and acquisition parameters, representing the total number of pixels contained in one frame. The diagonal length of the image frame is a positive real number calculated using the Pythagorean theorem.

[0063] S101-4: Add the normalized spatial coverage measurement to the normalized average displacement amplitude to obtain the initial intensity value.

[0064] It's important to understand that a smaller normalized spatial coverage of a target pixel indicates a more limited area occupied by its motion trajectory within the image, resulting in a smaller amplitude of motion. Even if the movement is rapid (i.e., a large average displacement after normalization), the overall visual intensity reflected by that target pixel is limited. Conversely, a smaller average displacement after normalization indicates a slower movement speed. Even if the movement range is wide (i.e., a large normalized spatial coverage), the action reflected by that target pixel appears sluggish, with weak visual impact. Therefore, only when both parameters are large will the sum of the two result in a high initial intensity value, reflecting a strong motion that is both widespread and rapidly moving.

[0065] S101-5: Determine the maximum and minimum values ​​from the initial intensity values ​​of all target pixels in the image frame; based on the maximum and minimum values, linearly scale the initial intensity values ​​of the target pixels to the range of 0 to 1 to obtain the motion intensity index of the target pixels.

[0066] It should be noted that the specific process of linear scaling using the maximum value minus the minimum value is a common technique, and will not be described in detail in this embodiment.

[0067] The second determining module 103 is used to convert the trajectory point sequence of the target pixel into a motion vector sequence; determine the directional similarity based on the directional similarity between vectors; cluster the vectors based on the directional similarity and identify the target cluster; determine the motion regularity index based on the number of vectors and directional similarity within the target cluster; and determine the significance index in combination with the motion intensity index.

[0068] It should be noted that the motion vector sequence consists of a set of displacement vectors arranged in chronological order.

[0069] It's important to understand that a target pixel's motion vector sequence may contain multiple transient or mixed motion patterns. To identify the single pattern that best represents the target pixel's primary, repetitive motion for quantitative analysis, clustering methods can be used to group vectors with similar directions. The cluster containing the most vectors is then identified as the target cluster. This target cluster represents the most frequent and dominant motion direction pattern of the target pixel within the reference time period, providing data support for subsequent calculations of the "motion regularity index."

[0070] In this embodiment, the cosine similarity between any two displacement vectors in the motion vector sequence is calculated, and the absolute value of the cosine similarity is taken as the directional similarity between any two displacement vectors. The directional similarity is subtracted from the positive integer 1 to obtain the dissimilarity distance between any two displacement vectors. Based on the dissimilarity distance, all motion vectors in the motion vector sequence are subjected to cluster analysis and divided into at least one cluster. The cluster with the most displacement vectors in all clusters is taken as the target cluster.

[0071] It should be noted that the specific method for calculating cosine similarity is a common technique, and will not be elaborated upon in this embodiment.

[0072] Directional similarity measures how close two displacement vectors are in direction, and is a value between 0 and 1. The greater the directional similarity (closer to 1) between any two displacement vectors in a motion vector sequence, the more consistent or regularly opposite the directions of the two displacement vectors are. For example, the two displacement vectors may both point to the right (same direction), or one may point to the right and the other to the left (opposite directions but regular).

[0073] Since dissimilarity distance is a complementary quantity of directional similarity, and also ranges between 0 and 1, if the dissimilarity distance between two displacement vectors is smaller (closer to 0), it means that the directions of the two displacement vectors are more similar (in the same direction or in opposite directions). The closer the two displacement vectors are in the directional feature space, the more likely they will be classified into the same cluster.

[0074] It should be noted that density-based clustering algorithms can be used to cluster all motion vectors in the motion vector sequence. Common techniques are not detailed in this embodiment. For example, the specific clustering process is as follows: In the feature space composed of all motion vectors, vectors whose dissimilarity distance is less than a neighborhood radius (e.g., 0.3) are considered to be adjacent to each other in the directional feature space, and are gradually connected into density-connected regions. All vectors within this region are then classified into a cluster.

[0075] It's important to understand that for an action to be identified as a noteworthy "repetitive behavior," it must simultaneously meet two key conditions: it must be visually sufficiently obvious (i.e., a high motion intensity index) and pattern sufficiently regular (the action exhibits strong repetitiveness). Therefore, motion intensity can be used to quantify the intensity of an action, and motion regularity can be used to quantify the repetitive pattern of the action. Combining these two metrics generates the final significance index. Since the significance index is derived from a comprehensive analysis of both "motion intensity" and "motion regularity," only those areas of action that are both intense and repetitive will be assigned high significance, i.e., high significance indexes.

[0076] In this embodiment, the arithmetic mean of the directional similarity between all displacement vector pairs within the target cluster is normalized to obtain the first sub-index; the ratio of the number of displacement vectors contained in the target cluster to the total number of displacement vectors in the motion vector sequence is normalized to obtain the second sub-index; the arithmetic mean of the first sub-index and the second sub-index is calculated as the motion regularity index; the arithmetic mean of the motion regularity index and the motion intensity index is calculated as the saliency index of the target pixel.

[0077] It should be noted that the first sub-index can be obtained by normalizing the arithmetic mean of the directional similarity between all displacement vector pairs using the minimum-maximum normalization method, or by obtaining the second sub-index using the minimum-maximum normalization method. For example, taking the normalization process to obtain the first sub-index as an example, firstly, the maximum and minimum values ​​are found from the arithmetic mean of the directional similarity of all target pixels in the image frame; then, based on the maximum and minimum values, the arithmetic mean of the directional similarity between all displacement vector pairs within the target cluster can be linearly mapped to the interval [0, 1] to obtain the first sub-index.

[0078] It is important to understand that if the first sub-index of a target pixel's target cluster is larger, it indicates that the direction of the dominant motion pattern of that target pixel is more consistent. The motion vectors are either almost in the same direction or more regularly in opposite directions, and the motion pattern is purer. If the second sub-index is larger, it indicates that the motion of that target pixel is more concentrated in one dominant pattern, with less motion in other directions, and the motion behavior pattern is more singular and focused. When both the first and second sub-indexes are large, the motion regularity index will be high, which represents a highly consistent and frequently repeated strong regular motion.

[0079] Specifically, the higher the saliency index of a target pixel (closer to 1), the higher the intensity and regularity of the motion corresponding to that pixel. For example, it could be a rapid, large-amplitude, and highly repetitive clapping or shaking motion. Such motions are visually very obvious and the pattern is highly predictable, making them the focus of subsequent identification and enhancement.

[0080] The correction module 104 is used to cluster target pixels based on saliency index, and set the region corresponding to each cluster as the region of interest; for each region of interest, the enhancement weight is determined according to the target pixel density and saliency index in the region; and the total enhancement requirement value is determined by combining the gray level and gradient features of the pixels in the region; and the total enhancement requirement value is mapped inversely to the gamma correction parameters of the image frame.

[0081] It's important to understand that a single frame of an image may contain a large number of discretely distributed highly saliency pixels (i.e., high saliency indices). However, these pixels may belong to different parts of the same body part of the target object. Processing them individually would not only be computationally redundant but also fail to create semantically meaningful augmented regions. Therefore, spatial clustering of target pixels can be performed based on saliency indices. Points that are geographically close and have high saliency can be grouped into different clusters, and each cluster can be mapped to a continuous region in the image, designated as a "region of interest." This allows subsequent augmentation processing to allocate augmentation resources on a coherent regional basis, thereby achieving synergistic optimization of important action parts.

[0082] In this embodiment, for each image frame in the behavioral video sequence, based on the two-dimensional coordinates of each target pixel in the image frame, they are aggregated into different clusters by a spatial clustering algorithm. The saliency index of the target pixel is used to adjust the distance metric in the clustering process. For each cluster, based on the coordinate distribution of all target pixels contained in it, a continuous image region covering all target pixels contained in the cluster is determined and marked as a region of interest.

[0083] It should be noted that density-based spatial clustering algorithms can be used to aggregate target pixels in an image frame into different clusters.

[0084] For example, when performing spatial clustering, the clustering process can be adjusted by incorporating the saliency index of the target pixels into the distance calculation. Specifically, when determining whether two target pixels should be clustered into one class, not only is the Euclidean spatial distance between the two target pixels calculated, but the average value of the saliency index of the two target pixels is also used as a weighting factor. Following the core rule that "the higher the average value of the saliency index of the two target pixels, the smaller the distance between the two target pixels will be calculated", this makes it easier for highly saliency pixels to be clustered into the same cluster even if the spatial distance is slightly far.

[0085] It should be noted that, based on the coordinate distribution of all target pixels within a cluster, the continuous image region covering them can be determined by calculating the minimum convex hull of these target pixels. Specifically, all target pixels within a cluster are considered as a set of points on a two-dimensional plane. Computational geometry algorithms (such as the Graham scan method) are used to find the minimum convex polygon that completely contains this set of points. The internal region enclosed by this minimum convex polygon is defined as the corresponding continuous image region, i.e., the region of interest.

[0086] It's important to understand that different regions of interest (ROIs) have varying degrees of importance for the final action recognition. Treating all ROIs equally will result in insufficient enhancement of critical regions rather than over-enhancing non-critical regions. Therefore, enhancement weights for each ROI can quantify its processing priority in subsequent image recognition. The core basis for these enhancement weights is the combined level of the distribution density and saliency of target pixels within the ROI, aiming to direct enhancement resources towards the ROIs with the densest motion features and the most significant action patterns.

[0087] In this embodiment, the total number of target pixels contained within the region of interest is counted; the pixel area occupied by the region of interest in the image is calculated; the total number of target pixels is divided by the pixel area of ​​the region to obtain the original density value; the arithmetic mean of the saliency indicators corresponding to all target pixels within the region of interest is calculated to obtain the first statistical value; the maximum value among all saliency indicators within the region of interest is found to obtain the second statistical value; the first statistical value and the second statistical value are averaged to obtain the original saliency value; based on all regions of interest in the image frame, their original density values ​​are collected to form a first set, and their original saliency values ​​are collected to form a second set; for each region of interest, based on the maximum and minimum values ​​in the first set, the original density value of the region of interest is linearly scaled to obtain a normalized density value; based on the maximum and minimum values ​​in the second set, the original saliency value of the region of interest is linearly scaled to obtain a normalized saliency value; the normalized density value and the normalized saliency value are averaged to obtain the enhancement weight.

[0088] It should be noted that the total number of pixels contained in the area of ​​interest is used as the pixel area of ​​the area.

[0089] The raw density value is used to characterize the distribution density of target pixels within a region of interest. A higher raw density value for a region of interest indicates a higher density of high-quality motion feature points per unit area, richer motion information, and a greater likelihood that it represents a core region of an action.

[0090] It is important to understand that if the first statistical value of a region of interest is larger, it means that the region of interest is generally composed of highly significant pixels, and the overall action pattern represented by the region of interest is more important. If the second statistical value is larger, it means that there is at least one extremely prominent highly significant pixel in the region of interest, and the action pattern represented by the region of interest is more critical. Therefore, only when both the first and second statistical values ​​are large will the original significance value be high, indicating that the region of interest is both generally important and has prominent highly significant pixels.

[0091] It should be noted that the original density value and the original significance value can be linearly scaled using the min-max normalization method, which is a common technique.

[0092] Enhancement weights measure the priority or resource allocation intensity that a region of interest should receive in subsequent image recognition processing. A larger enhancement weight (closer to 1) indicates that the region of interest possesses both higher target pixel density and higher motion saliency, meaning that the region of interest is more likely to be a core part of an action and more likely to involve important, repetitive action patterns.

[0093] The process for determining the total enhanced demand value is as follows: Figure 3 As shown, it includes: S102-1: Calculate the average grayscale value of all pixels within each region of interest; classify regions with an average grayscale value less than a preset global brightness threshold as low-brightness regions.

[0094] It should be noted that the preset global brightness threshold is used to distinguish low-brightness areas, and its specific value can be the median of the grayscale values ​​of all pixels in the entire frame. For example, 128 (for an 8-bit grayscale image, the range is 0-255).

[0095] S102-2: Take the arithmetic mean of the gradient magnitudes at all pixels in each region of interest as the original sharpness of each region of interest; subtract the original sharpness of each region of interest from the positive integer 1 to obtain the blur factor.

[0096] Original sharpness measures the degree of gradient change at pixels within a region of interest. A higher original sharpness for a region of interest indicates sharper edges, richer texture details, higher image quality, and better detail preservation.

[0097] The blur factor is a complementary measure to the original sharpness, quantifying the degree of blur in a region of interest. A higher blur factor for a region of interest indicates greater blurriness, more severe detail loss, and a greater need for enhancement (e.g., brightening shadows to restore detail).

[0098] S102-3: For low-brightness areas, based on the difference between the positive integer 1 and the normalized average gray level, the enhancement weight, and the normalized blur factor, the first enhancement requirement value is obtained through a preset first enhancement requirement value calculation function.

[0099] Since the smaller the difference between a positive integer 1 and the normalized grayscale average value of a region of interest, the less dark the region of interest actually is. Even if the region of interest is important and blurry (i.e., high enhancement weight and high normalized blur factor), the underexposure problem is not serious, and the need for brightening is lower. If the enhancement weight of a region of interest is smaller, the less important the region of interest is. Even if the region of interest is dark and blurry (i.e., low normalized grayscale average value and high normalized blur factor), it does not need to be prioritized. If the normalized blur factor of a region of interest is smaller, the region of interest is still clearer and has not lost details due to being too dark, and the need for brightening is weaker. Therefore, the calculated first enhancement requirement value is high only when the difference between the positive integer 1 and the normalized grayscale average value of a certain region of interest, the enhancement weight, and the normalized blur factor are all large. This means that the more severely underexposed, crucial to the analysis of action behavior patterns, and whose details are blurred due to underexposure of the region of interest, the more necessary it is to enhance the contrast and restore the details of that region of interest. Assuming that U represents the total number of low-brightness regions, and there are u low-brightness regions, the first enhancement requirement value can be represented by the following preset first enhancement requirement value calculation function: in, This represents the first enhancement requirement value for the u-th low-brightness region; This represents the normalized average grayscale value of the u-th low-brightness region; This represents the enhancement weight of the u-th low-brightness region; This represents the blur factor for normalizing the u-th low-brightness region.

[0100] It should be noted that the normalized grayscale average can be obtained by normalizing the grayscale average relative to the grayscale range of the current frame. Specifically: First, obtain the grayscale values ​​of all pixels in the image frame and determine the maximum grayscale value (usually 255) and the minimum grayscale value (usually 0); then, for the grayscale average of each region of interest, subtract the minimum grayscale value from the average grayscale average, and then divide by (maximum grayscale value - minimum grayscale value), thereby mapping the grayscale average of each region of interest to the [0, 1] interval to obtain the normalized grayscale average.

[0101] It should be noted that the normalization of the blur factor is achieved through cross-region global normalization. Specifically: First, the blur factor of all regions of interest in the image frame is calculated, and these blur factors are set together. The maximum and minimum values ​​in this set are then found. Then, for each region of interest, the minimum value is subtracted from its blur factor, and then divided by (maximum value - minimum value), thereby linearly mapping the blur factors of all regions of interest to the interval [0, 1], and obtaining the normalized blur factor.

[0102] S102-4: Calculate the sum of the first enhancement demand values ​​as the total enhancement demand value.

[0103] It should be noted that the specific process of inverse mapping is a common technique, aiming to ensure that the larger the total enhancement requirement, the smaller the resulting gamma value should be. Specifically: a gamma value adjustment range is preset (using [...]). , ] indicates that, among them Less than 1, =1), and collect the total enhancement demand value of all image frames in the video sequence; then, map the total enhancement demand value of each frame to [ using the inverse linear mapping formula] , Within the specified interval, gamma correction parameters for each frame are obtained. The frame with the highest total enhancement requirement corresponds to the lowest gamma value. The frame with the smallest total enhancement requirement corresponds to =1 (basic no correction).

[0104] It should be noted that the inverse linear mapping formula is: Gamma correction parameter = 1 - (1 - ... )× , This represents the minimum gamma value within the gamma value adjustment range; This indicates the maximum gamma value within the gamma value adjustment range.

[0105] It should be noted that the specific value of the gamma value adjustment range can be determined through industry experience or by visual quality assessment of typical video sequences. The range should be selected to effectively improve the visual details of image frames without introducing obvious distortion in most scenarios. This embodiment does not impose specific limitations. For example, the typical range of gamma value adjustment is [0.4, 1].

[0106] The recognition module 105 is used to enhance the video sequence based on gamma correction parameters to obtain an enhanced video sequence; and to identify repetitive motion patterns from the enhanced video sequence.

[0107] It should be noted that the process of enhancing the image frame based on the gamma correction parameter (i.e., the adjusted gamma value) is a well-known technique in the art, and will not be described in detail in this embodiment. Specifically: for each frame of the video sequence, the gray value of each pixel (or each color channel if it is a color image) is subjected to gamma transformation. The transformation formula is: output gray value = 255 × (input gray value ÷ 255)^gamma correction parameter. The transformation process is a pixel-by-pixel non-linear operation: if the gamma correction parameter is less than 1, the contrast of the dark area will be stretched (making it brighter and the details clearer); if the gamma correction parameter is equal to 1, the output gray value is equal to the input gray value, with no change.

[0108] It can be understood that by performing gamma transformation on all image frames in the video sequence using the above method, and then combining all the processed image frames in their original order, an enhanced video sequence with optimized visual effects and more prominent details is obtained.

[0109] It should be noted that the identification of repetitive action patterns from enhanced video sequences mainly utilizes mature computer vision and deep learning processes; the specific identification process will not be described in detail in this embodiment.

[0110] For example, first, the human body region is located in each enhanced frame using an object detection model (such as YOLO); then, a pose estimation algorithm (such as OpenPose) is applied to the human body region to extract skeletal key points, forming a temporal pose sequence; finally, the enhanced video sequence is input into a temporal behavior recognition model (such as a network combining 3D-CNN, LSTM, or Transformer). After training, the model can analyze the spatiotemporal features of the action and determine whether the action belongs to a repetitive action pattern (such as clapping, shaking, etc.), and output the corresponding behavior category.

Claims

1. A video image-based system for recognizing stereotyped behaviors in children with autism spectrum disorder, characterized in that, The system includes: The acquisition module is used to acquire video sequences of the target object's behavior in natural scenes; and to extract reference time periods centered on each image frame. The first determining module is used to select candidate pixels from the pixels for each image frame; obtain the motion trajectory point sequence by optical flow tracing within its reference time period; filter out candidate pixels that do not meet the preset conditions to determine the target pixel; and determine the motion intensity index based on the trajectory point sequence of the target pixel. The second determining module is used to convert the trajectory point sequence of target pixels into a motion vector sequence; determine the directional similarity based on the directional similarity between vectors; cluster the vectors based on the directional similarity and identify the target clusters; determine the motion regularity index based on the number of vectors and directional similarity within the target clusters; and determine the significance index in combination with the motion intensity index. The correction module is used to cluster target pixels based on saliency index, and set the region corresponding to each cluster as the region of interest; for each region of interest, the enhancement weight is determined according to the target pixel density and saliency index in the region; and the total enhancement requirement value is determined by combining the pixel gray level and gradient features in the region; and the total enhancement requirement value is mapped inversely to the gamma correction parameters of the image frame. The recognition module is used to enhance video sequences based on gamma correction parameters to obtain enhanced video sequences; and to identify repetitive motion patterns from the enhanced video sequences.

2. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 1, characterized in that, The reference time period includes multiple consecutive image frames centered on each image frame; The extraction of reference time periods centered on each image frame includes: For each image frame in the behavioral video sequence, a predetermined number of image frames are extracted from both sides of the image frame, with the image frame as the center, to form the target image frame sequence. If the number of image frames before or after an image frame does not meet the preset number, the image frames on the side where the number of image frames meets the preset number are extracted, with the image frame as the center, to form the target image frame sequence; wherein, the reference time period is used to provide the target image frame sequence for each pixel in the image frame.

3. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 2, characterized in that, The step of selecting candidate pixels from the pixels includes: For each pixel in each image frame, take the pixel as the center, obtain the gray values ​​of all pixels within a local neighborhood window of a preset size; calculate the variance of the gray values ​​of all pixels within the local neighborhood window, and use it as the spatial feature change value of the pixel. Based on the target image frame sequence corresponding to the pixel, the gray value of the pixel coordinate in each frame of the target image frame sequence is obtained to form a gray value sequence; the variance of the gray value sequence is calculated as the temporal feature change value of the pixel. The spatial feature change values ​​and temporal feature change values ​​are normalized respectively; and the arithmetic mean of the normalized spatial feature change values ​​and the normalized temporal feature change values ​​is calculated as the comprehensive significance. Sort all pixels in the image frame from highest to lowest based on their overall saliency value; select the top-ranked pixels by a predetermined percentage as candidate pixels.

4. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 1, characterized in that, Each data point in the motion trajectory point sequence records the two-dimensional coordinates of the pixel in any image frame within the reference time period. All data points in the motion trajectory point sequence are arranged in chronological order according to their corresponding times. The target pixel determination process includes: Calculate the total number of data points contained in the motion trajectory point sequence; If the total number of data points is less than the first preset threshold, the motion trajectory duration of the corresponding candidate pixel is determined to be insufficient, and it is recorded as the first candidate pixel. Calculate the distance between two data points that are adjacent in time sequence in the motion trajectory point sequence; calculate the arithmetic mean of the distances between all adjacent data point pairs in the motion trajectory point sequence as the average distance; If the average distance is less than the second preset threshold, the spatial motion significance of the corresponding candidate pixel is determined to be insufficient, and it is recorded as the second candidate pixel. Remove the first and second candidate pixels from all candidate pixels in the image frame to obtain the target pixel.

5. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 4, characterized in that, The process of determining the exercise intensity index includes: For each target pixel, a point set is constructed on the image plane based on all two-dimensional coordinates in the motion trajectory point sequence of the target pixel; the area of ​​the smallest convex polygon that completely contains the point set is calculated as the spatial coverage. Calculate the straight-line distance between each pair of adjacent data points according to the time order of the data points in the motion trajectory point sequence; calculate the arithmetic mean of the straight-line distances between all adjacent point pairs as the average displacement amplitude; Divide the spatial coverage by the size of the image frame containing the target pixel to obtain the normalized spatial coverage; divide the average displacement amplitude by the diagonal length of the image frame containing the target pixel to obtain the normalized average displacement amplitude. The initial intensity value is obtained by adding the normalized spatial coverage measurement to the normalized average displacement amplitude. The maximum and minimum values ​​are determined from the initial intensity values ​​of all target pixels in the image frame; based on the maximum and minimum values, the initial intensity values ​​of the target pixels are linearly scaled to the range of 0 to 1 to obtain the motion intensity index of the target pixels.

6. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 1, characterized in that, The motion vector sequence consists of a set of displacement vectors arranged in chronological order; the target cluster identification process includes: Calculate the cosine similarity between any two displacement vectors in the motion vector sequence, and take the absolute value of the cosine similarity as the directional similarity between any two displacement vectors. Subtracting the directional similarity from the positive integer 1 yields the dissimilarity distance between any two displacement vectors. Based on dissimilarity distance, cluster analysis is performed on all motion vectors in the motion vector sequence to divide them into at least one cluster. The cluster with the most displacement vectors among all clusters is selected as the target cluster.

7. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 6, characterized in that, The process of determining the significance index includes: The arithmetic mean of the directional similarity between all displacement vector pairs within the target cluster is normalized to obtain the first sub-index. The ratio of the number of displacement vectors contained in the target cluster to the total number of displacement vectors in the motion vector sequence is normalized to obtain the second sub-index. Calculate the arithmetic mean of the first sub-indicator and the second sub-indicator as an indicator of motion regularity; The arithmetic mean of the motion regularity index and the motion intensity index is calculated and used as the significance index of the target pixel.

8. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 1, characterized in that, The clustering of target pixels based on saliency indicators, and setting the region corresponding to each cluster as the region of interest, includes: For each image frame in the behavioral video sequence, based on the two-dimensional coordinates of each target pixel in the image frame, a spatial clustering algorithm is used to aggregate them into different clusters. The saliency index of the target pixel is used to adjust the distance metric in the clustering process. For each cluster, based on the coordinate distribution of all target pixels contained in it, a continuous image region covering all target pixels contained in the cluster is determined and marked as a region of interest.

9. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 8, characterized in that, The enhancement weight determination process includes: Count the total number of target pixels contained in the region of interest; calculate the pixel area occupied by the region of interest in the image; divide the total number of target pixels by the pixel area of ​​the region to obtain the original density value; Calculate the arithmetic mean of the significance indexes corresponding to all target pixels within the region of interest, and use it as the first statistical value; find the maximum value among all significance indexes within the region of interest, and use it as the second statistical value; average the first statistical value and the second statistical value to obtain the original significance value. Based on all regions of interest in the image frame, their original density values ​​are collected to form a first set, and their original saliency values ​​are collected to form a second set. For each region of interest, the original density value of the region of interest is linearly scaled based on the maximum and minimum values ​​in the first set to obtain a normalized density value. Based on the maximum and minimum values ​​in the second set, the original significance values ​​of the region of interest are linearly scaled to obtain normalized significance values; The enhanced weights are obtained by averaging the normalized density values ​​and the normalized significance values.

10. The video image-based stereotyped behavior recognition system for children with autism spectrum disorder according to claim 9, characterized in that, The process for determining the total enhancement requirement includes: Calculate the average grayscale value of all pixels within each region of interest; classify regions with an average grayscale value less than a preset global brightness threshold as low-brightness regions. The arithmetic mean of the gradient magnitudes at all pixels within each region of interest is used as the original sharpness of each region of interest; the original sharpness of each region of interest is subtracted from the positive integer 1 to obtain the blur factor. For low-brightness areas, the first enhancement requirement value is obtained by using a preset first enhancement requirement value calculation function based on the difference between the positive integer 1 and the normalized grayscale average value, the enhancement weight, and the normalized blur factor. Calculate the sum of the first enhancement demand values, which is the total enhancement demand value.

Images