Children hyperactivity behavior early warning monitoring method based on video analysis and storage medium

Abstract

The invention discloses a video analysis-based early warning and monitoring method for children hyperactivity behavior and a storage medium, and the method comprises the steps: firstly, guaranteeing the comprehensive monitoring of children behaviors through employing a multi-channel video collection device, and providing a sufficient continuous data source for subsequent analysis; furthermore, the quality of monitoring data is ensured through the use of image noise reduction and segmentation technologies, and accurate motion features are effectively extracted through the combination of an optical flow algorithm and a background subtraction algorithm; further, the movement behaviors of the children are comprehensively analyzed by calculating behavior parameters such as movement amplitude, movement direction change and continuous movement time, and potential behavior abnormity is revealed; the multi-dimensional behavior feature vector obtained through behavior parameter analysis is compared with the standard template, the behavior type of the child is automatically recognized, early-stage symptoms of abnormal behaviors such as hyperactivity and the like are found in time, and a basis for early diagnosis and intervention is provided for behavior problems such as hyperactivity and the like of the child.

Description

Technical Field

[0001] The present invention relates to the field of video analysis, and in particular to a method and storage medium for early warning monitoring of children's attention deficit hyperactivity disorder (ADHD) behavior based on video analysis. Background Art

[0002] Attention Deficit Hyperactivity Disorder (ADHD) is a common neurodevelopmental disorder characterized by behavioral features such as inattention, hyperactivity, and difficulty controlling impulses. Symptoms of ADHD typically appear in early childhood and may persist into adulthood.

[0003] Early identification of ADHD in children is crucial, as timely intervention can effectively improve children's symptoms and reduce potential future academic and social problems. With technological advancements, video surveillance and data analysis have become effective tools for early intervention for children with ADHD. For example, continuous video surveillance can collect behavioral data from children. Combined with image processing techniques (such as image noise reduction, segmentation, and motion feature extraction), children's activity status and behavior patterns can be analyzed, helping to identify early signs of ADHD. Automated behavioral analysis systems can monitor children's attention, activity intensity, and impulsive behavior in real time, providing educators and parents with timely feedback and enabling early intervention.

[0004] However, in the process of using video surveillance analysis combined with image processing technology, simple motion detection algorithms or continuous recognition of children's motion vectors are difficult to cope with complex and dynamic scene changes. In addition, for the sudden changes in children's behavior (such as suddenly standing up), the existing image processing technology cannot accurately analyze the mutation points on the motion vector caused by the suddenness of children's behavior. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and storage medium for early warning monitoring of children's ADHD behavior based on video analysis, which solves the above-mentioned technical problems pointed out in the prior art.

[0006] The present invention provides a method for early warning monitoring of children's ADHD behavior based on video analysis, comprising the following steps:

[0007] Acquire continuous surveillance video data from a preset surveillance area through a multi-channel video acquisition device;

[0008] Preprocessing the surveillance video data to obtain a target area image;

[0009] The motion feature extraction based on the optical flow algorithm and background subtraction algorithm is used on the segmented target motion area to obtain the motion vector information between consecutive frames;

[0010] Calculating behavior parameters based on motion vector information between consecutive frames; the behavior parameters include motion amplitude, motion direction change, and duration of motion;

[0011] Normalizing the behavioral parameters and assembling them into a multi-dimensional behavioral feature vector;

[0012] The multi-dimensional behavior feature vector is compared with a pre-established standard behavior template to form classification information of the child's behavior in the target video.

[0013] Preferably, the segmented target motion region is subjected to motion feature extraction based on an optical flow algorithm and a background subtraction algorithm to obtain motion vector information between consecutive frames, including the following steps:

[0014] Performing image enhancement and brightness normalization processing on the target motion area to obtain an enhanced image;

[0015] Processing the enhanced image based on the continuous frames through a background subtraction algorithm to obtain a background subtracted image;

[0016] The enhanced images based on the continuous frames are processed by multi-scale optical flow to estimate and obtain initial motion vector information;

[0017] The target motion vector is obtained by analyzing the background subtracted image in combination with the initial motion vector information through regional correlation correction processing.

[0018] Preferably, the analyzing and obtaining the target motion vector based on the background subtracted image in combination with the initial motion vector information through regional correlation correction processing includes the following steps:

[0019] Preliminarily combining the background subtraction image with the initial motion vector information to obtain second motion vector information corresponding to each pixel position in the background subtraction image;

[0020] A comprehensive outlier value obtained based on the second motion vector information of the t-th frame through local space analysis, continuous frame timing analysis, and mutation analysis of the current frame;

[0021] Screening based on the comprehensive abnormal value to obtain abnormal second motion vector information;

[0022] The abnormal second motion vector information is smoothed to obtain a target motion vector.

[0023] Preferably, the comprehensive abnormal value obtained based on the second motion vector information of the t-th frame through local space analysis, continuous frame timing analysis and mutation analysis of the current frame includes the following steps:

[0024] Obtaining a local area image of each pixel point in the background subtraction image through a neighborhood window of a fixed size from the background subtraction image;

[0025] A spatial outlier is obtained by calculating the second motion vector information of the j-th pixel position of the i-th local area image of the t-th frame and the second motion vector information of the neighboring point of the j-th pixel position of the i-th local area image of the t-th frame;

[0026] A temporal smoothing coefficient is obtained by calculating according to the second motion vector information of the j-th pixel position of the i-th local area image of the t-th frame and the second motion vector information of the j-th pixel position of the i-th local area image of the t-1-th frame;

[0027] The local mutation evaluation value is obtained by analyzing the second motion vector information of the local area of ​​the continuous frames in combination with the global dynamic analysis;

[0028] A comprehensive outlier value is obtained by calculation based on the spatial outlier, the temporal smoothing coefficient, and the local mutation evaluation value.

[0029] The composite outlier The calculation method is:

[0030] = + γ;

[0031] Where, is a spatial outlier; is the time domain smoothing coefficient; is the mutation assessment value; 、 and γ are weight coefficients;

[0032] is the second motion vector information of the j-th pixel position in the i-th local area of ​​the t-th frame; is the mean of the second motion vector information of the S neighborhoods of the j-th pixel position in the i-th local area of ​​the t-th frame; is the standard deviation of the second motion vector information of the S neighborhoods of the j-th pixel position in the i-th local area in the t-th frame; is a smoothing factor used to prevent the denominator from being zero; is the second motion vector information of the j-th pixel position in the i-th local area of ​​the t-1-th frame; is the standard deviation of the second motion vector information of the t-th frame.

[0033] Preferably, the step of analyzing the local region second motion vector information of consecutive frames in combination with global dynamic analysis to obtain the local mutation evaluation value includes the following steps:

[0034] Calculating all second motion vector information of each local area of ​​the current frame to obtain a global motion vector information parameter;

[0035] The global motion vector information parameters include the global motion direction mean and the global motion amplitude standard deviation;

[0036] Extracting the motion direction angle corresponding to the second motion vector information at each pixel position in the current frame and N consecutive frames before the current frame, constructing a direction field matrix based on the motion direction angle; performing principal component extraction on the direction field matrix to obtain a global direction angle;

[0037] Each element in the direction field matrix represents the motion direction angle of the j-th pixel position in the t-th frame;

[0038] Based on the global motion vector information parameter and the global direction angle and in combination with the second motion vector information of each pixel point posture, a Mahalanobis distance obtained by projection analysis is used to perform screening and marking to obtain a local projection abnormality marking matrix A;

[0039] Each element in the local projection abnormality mark matrix A is processed by temporal continuity detection and spatial consistency detection to generate comprehensive mark information;

[0040] The mutation intensity is calculated based on the comprehensive abnormal marker information.

[0041] The method of screening and marking the local projection abnormality marker matrix A by using the Mahalanobis distance obtained by projection analysis based on the global motion vector information parameter and the global direction angle and in combination with the second motion vector information of each pixel point posture includes the following steps:

[0042] A global projection matrix P is constructed based on the global motion vector information parameters and the global direction angle and in combination with the second motion vector information of each pixel position; global motion vector information V'' corresponding to the second motion vector information of each pixel position is calculated based on the global projection matrix P; the Mahalanobis distance is calculated based on the global motion vector information V'' to obtain the difference between each global motion vector information V'' and the global motion vector parameter information; and a local projection anomaly marker matrix A is constructed after a plurality of global motion vector information V'' is obtained after screening based on the difference;

[0043] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0044] From the analysis of the above-mentioned video analysis-based early warning monitoring method and storage medium for children's ADHD behavior provided by the present invention, it can be seen that in specific applications, a multi-channel video acquisition device is first used to ensure comprehensive monitoring of children's behavior, providing sufficient continuous data sources for subsequent analysis; further, the quality of monitoring data is ensured by the use of image noise reduction and segmentation technology, and the combination of optical flow algorithm and background subtraction algorithm effectively extracts accurate motion features; further, by calculating behavioral parameters such as motion amplitude, motion direction change and continuous motion time, the child's movement behavior is comprehensively analyzed to reveal potential behavioral abnormalities; by comparing the multidimensional behavioral feature vector obtained by behavioral parameter analysis with the standard template, the child's behavior type is automatically identified, and the early signs of abnormal behaviors such as ADHD are discovered in a timely manner, providing a basis for early diagnosis and intervention of children's behavioral problems such as ADHD. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the main process of a method for early warning monitoring of children's ADHD behavior based on video analysis;

[0046] Figure 2 A color schematic diagram of background subtraction simulation in a method for early warning monitoring of ADHD behavior in children based on video analysis;

[0047] Figure 3 A schematic diagram of background subtraction simulation in a method for early warning monitoring of ADHD behavior in children based on video analysis;

[0048] Figure 4 A schematic diagram of the simulation of initial motion vector information obtained through multi-scale optical flow processing in a method for early warning monitoring of ADHD behavior in children based on video analysis;

[0049] Figure 5 A schematic diagram of a simulation of local abnormalities in some motion vectors in a method for early warning monitoring of ADHD behavior in children based on video analysis;

[0050] Figure 6 A schematic diagram of the correlation between multiple operating steps in a method for early warning monitoring of ADHD behavior in children based on video analysis;

[0051] Figure 7 The present invention is a schematic diagram of the operation of obtaining a local mutation evaluation value by combining the second motion vector information of the local area of ​​continuous frames with global dynamic analysis in a method for early warning monitoring of children's ADHD behavior based on video analysis. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] The present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0054] Example 1

[0055] like Figure 1 As shown, the first embodiment of the present invention provides a method for early warning monitoring of children's ADHD behavior based on video analysis, comprising the following steps:

[0056] S10: Acquire continuous surveillance video data from a preset surveillance area through a multi-channel video acquisition device;

[0057] It should be noted that the above embodiment of the present application first arranges multiple video acquisition devices to ensure that a predetermined monitoring area is covered, and collects continuous behavioral video data of the target child, thereby providing a sufficient data source for subsequent analysis of child ADHD;

[0058] S20: Preprocessing the surveillance video data to obtain a target area image;

[0059] The preprocessing includes converting the video into a frame sequence to obtain a frame image; performing noise reduction processing on the frame image to obtain a noise-reduced image; and performing image segmentation processing on the noise-reduced image to segment the target motion area of ​​the target child;

[0060] It should be noted that the above-mentioned embodiment of the present application first converts the collected continuous video into a frame sequence, and performs noise reduction processing on the frame image to remove noise in the image for subsequent analysis. Then, through image segmentation technology, the motion area where the target child is located is extracted; by converting the video into a frame sequence, each frame image can be extracted and analyzed; through noise reduction processing, the noise that may appear during the video acquisition process is removed, so that the image quality is improved; further, through image segmentation, the target area, that is, the area where the child is located, is accurately identified, laying the foundation for subsequent child behavior analysis.

[0061] S30: extracting motion features based on an optical flow algorithm and a background subtraction algorithm from the segmented target motion region to obtain motion vector information between consecutive frames;

[0062] It should be noted that the above-mentioned embodiments of the present application extract motion features in the target motion area by adopting optical flow algorithm and background subtraction algorithm, wherein the optical flow algorithm can track the motion changes of the target in the video and obtain the motion trajectory of the target; the background subtraction algorithm can remove the static background and only extract the moving object, further improving the accuracy of the motion area; the above-mentioned optical flow algorithm and background subtraction algorithm are used to accurately capture the motion trajectory and dynamic changes of the child. The combination of optical flow algorithm and background subtraction algorithm makes the motion information clearer and more stable, further providing high-quality data for subsequent behavioral analysis.

[0063] S40: Calculating behavior parameters based on motion vector information between consecutive frames; the behavior parameters include motion amplitude, motion direction change, and duration of motion;

[0064] It should be noted that the above-mentioned embodiments of the present application use motion vector information to calculate the target's behavioral parameters, such as the amplitude of the movement, the change in the direction of movement, and the duration of the movement; the motion amplitude reflects the intensity of the child's movement, the change in the direction of movement can reflect the regularity of the child's movement, and the duration of the movement reveals the continuity of the movement; by calculating the above-mentioned behavioral parameters, the child's behavioral characteristics are analyzed. For example, the motion amplitude can help determine whether the child is engaged in high-intensity activities; the change in the direction of movement can help determine whether the child has sudden changes in movement; the duration of the movement can help determine whether the child has continuous activities, etc.

[0065] S50: normalizing the behavior parameters and assembling them into a multi-dimensional behavior feature vector;

[0066] It should be noted that the above-mentioned embodiment of the present application normalizes all calculated behavioral parameters to ensure that the values ​​of different parameters are in the same scale range, and then combines all normalized parameters to assemble them into a multi-dimensional behavioral feature vector; through normalization, the analysis deviation caused by the scale differences of different parameters is avoided, and the multi-dimensional behavioral feature vector converts the motion data into a numerical representation, which is convenient for subsequent classification and comparison operations.

[0067] S60: Compare the multi-dimensional behavior feature vector with a pre-established standard behavior template to form classification information of the child's behavior in the target video.

[0068] It should be noted that the above-mentioned embodiment of the present application compares the obtained multi-dimensional behavioral feature vector with a pre-set standard behavioral template, and classifies the behavior of the child in the target video according to the characteristic pattern of the behavioral template, so as to determine whether the child has early signs of ADHD, thereby further intervening in the child's ADHD signs in advance.

[0069] The standard behavior template in the above-mentioned embodiment of the present application refers to a pre-established behavior pattern database, which contains characteristic data of different types of children's behaviors. These templates are obtained through analysis of a large amount of sample data and can represent the typical behavioral performance of normal children or children with characteristics such as ADHD; the standard behavior template usually contains a series of multidimensional feature vectors, representing the movement patterns of specific behaviors (such as quiet, active, overactive, etc.), including parameters such as movement amplitude, direction change, and continuous movement time.

[0070] The above-mentioned embodiment of the present application first uses a multi-channel video acquisition device to ensure comprehensive monitoring of children's behavior, providing sufficient continuous data sources for subsequent analysis; further, the quality of monitoring data is ensured by using image noise reduction and segmentation technology, and the combination of optical flow algorithm and background subtraction algorithm effectively extracts accurate motion features; further, by calculating behavioral parameters such as motion amplitude, motion direction change and continuous motion time, the child's movement behavior is comprehensively analyzed to reveal potential behavioral abnormalities; by comparing the multidimensional behavioral feature vector obtained by behavioral parameter analysis with the standard template, the child's behavior type is automatically identified, and the early signs of abnormal behaviors such as ADHD are discovered in a timely manner, providing a basis for early diagnosis and intervention for children's behavioral problems such as ADHD.

[0071] Specifically, in step S30, motion feature extraction based on the optical flow algorithm and background subtraction algorithm is used on the segmented target motion area to obtain motion vector information between consecutive frames, including the following steps:

[0072] Step S31: performing image enhancement and brightness normalization processing on the target motion area to obtain an enhanced image;

[0073] It should be noted that the image enhancement of the above-mentioned embodiment of the present application refers to the processing of the target motion area through grayscale, noise suppression and histogram equalization. First, through grayscale processing, the influence of color information is eliminated to improve subsequent processing efficiency. Then, through noise suppression processing (i.e., median filtering or mean filtering processing), the interference of random noise on subsequent analysis is reduced. Furthermore, through histogram mean and processing, the overall brightness and contrast of the image are balanced to alleviate the problem of local brightness inconsistency caused by lighting changes or ambient light interference. Finally, the image is divided into multiple local areas, and each local area is brightness normalized to ensure that each area can perform subsequent motion detection at a unified scale, and finally, an enhanced image is obtained.

[0074] Step S32: processing the enhanced image based on the continuous frames by a background subtraction algorithm to obtain a background subtracted image;

[0075] It should be noted that if Figure 2 、 Figure 3As shown, the above-mentioned embodiment of the present application constructs an initial background model by using a time averaging processing method on the enhanced images of continuous frames to capture static background information. Then, during the background subtraction algorithm processing of each frame, the background model is adjusted according to the local brightness mean, and abnormal pixel changes caused by sudden changes in illumination (i.e., sudden changes in illumination caused by shadows or reflections) are identified and eliminated by setting a threshold. By introducing an adaptive update method based on time weighting, the background model adapts to the natural changes in ambient illumination and reduces errors caused by the illumination effects caused by the movement of non-targets (i.e., non-child targets, such as flying insects) in the background. Furthermore, background subtraction is performed on the enhanced image of the current frame and the initial background model according to the pixel grayscale difference to generate a preliminary binary image of the motion area. Subsequently, the binary image of the motion area is filtered using morphological processing to remove isolated noise points caused by noise, so that the edges of the motion area are kept coherent and clear when the target child moves in the subsequent analysis process. The above-mentioned background subtraction algorithm is common knowledge to those skilled in the art and will not be described in detail in this application.

[0076] Step S33: performing multi-scale optical flow processing based on the enhanced images of consecutive frames to estimate and obtain initial motion vector information;

[0077] It should be noted that if Figure 4 As shown, the above embodiment of the present application first establishes a multi-scale pyramid model (i.e., multi-scale optical flow processing) for the enhanced image, and uses a multi-resolution processing method to capture motion information at different scales to reduce errors caused by motion blur or slight jitter; then, on each layer of the pyramid, the dense optical flow between consecutive frames is calculated by a local window method to obtain a preliminary motion vector distribution; the above multi-scale optical flow processing method is common knowledge to those skilled in the art and will not be elaborated in this application.

[0078] Step S34: Based on the background subtraction image and the initial motion vector information, a target motion vector (ie, the motion vector information between the above consecutive frames) is obtained through regional correlation correction processing.

[0079] It should be noted that the above embodiment of the present application firstly performs image enhancement and brightness normalization on the target motion area to ensure the stability and consistency of the image, thereby providing high-quality image data for subsequent motion detection, background subtraction, and optical flow processing;

[0080] Furthermore, when steps S32-S33 are executed, the static background is effectively removed from the image through the background subtraction algorithm, and the target motion area is accurately extracted, providing a clear motion target for subsequent motion analysis; then, through the adaptive update of the background model and the elimination of illumination mutations, the stability and accuracy of the system under different lighting conditions are guaranteed, and the accuracy of motion area recognition is improved; multi-scale optical flow processing is used to capture motion information of different scales in a multi-level resolution manner, especially in the case of fast or small motion, to reduce errors caused by blur or jitter, and through dense optical flow calculation, the motion vector information of each pixel is obtained, thereby improving the accuracy of motion estimation; further, by combining the background subtracted image and the initial motion vector information, regional correlation correction processing is used to further analyze and correct the motion vector information, and by refining the motion vector, the accuracy of the motion vector is improved, motion errors caused by background interference or non-target objects are avoided, and accurate matching of the motion vector with the target area is ensured, making subsequent behavior analysis and motion feature extraction more reliable.

[0081] In the above embodiment of the present application, the target motion vector is obtained by analyzing the background subtraction image in combination with the initial motion vector information through regional correlation correction processing in order to further optimize the accuracy of the motion vector. However, in the specific implementation process, the research technicians found that if Figure 5 As shown, Figure 5 At the circle pointed by the arrow, due to the regional boundary or local noise, some motion vectors show local anomalies (i.e., isolated outliers or mutation points), resulting in discontinuous distribution of motion vector data in the region. Specifically, due to lighting or sudden movement of the child, background subtraction and motion vector information estimation cannot be accurately captured, or local noise points are generated, making the image information unstable. For example, if the motion vector in a certain area shows a sudden change in direction or a jump in amplitude, this will result in obvious faults or discontinuous areas in the motion vector map. This discontinuity will affect the stability of motion analysis and, in turn, the accuracy of subsequent tasks such as target tracking and behavior analysis. Therefore, in the process of obtaining target vector information, it is also necessary to optimize and adjust the mutation points in the analysis process to obtain more accurate target vector information that is more convenient for subsequent analysis of children's behavior. For details, see steps S341 to S344.

[0082] See also Figure 6 as well as Figure 7 In step S34, based on the background subtraction image and the initial motion vector information, a target motion vector is obtained by performing regional correlation correction processing, which includes the following steps:

[0083] Step S341: Preliminarily combining the background subtracted image with the initial motion vector information to obtain second motion vector information corresponding to each pixel position in the background subtracted image;

[0084] It should be noted that in the above-mentioned embodiment of the present application, when using an algorithm such as optical flow to calculate motion information, a motion vector is usually calculated for each pixel point (or each defined area / block) in the image. The motion vector describes the displacement information (such as amplitude and direction) of the pixel point (or area) in continuous frames. Based on this, by performing a preliminary combination of the background subtraction image and the initial motion vector information based on the pixel position, the second motion vector information of each pixel point in the background subtraction image is obtained. In addition, it should be noted that since the above-mentioned embodiment of the present application is performed on the enhanced image of continuous frames, the initial motion vector information obtained is the initial motion vector information corresponding to the continuous frame condition, that is, the initial motion vector information corresponding to the enhanced image of multiple frames. In other words, the initial motion vector information changes continuously and is different in each frame. It can also be said that the above-mentioned initial motion vector information is the initial motion vector information corresponding to each pixel point position in multiple frames (that is, the initial motion vector information of each pixel point in the enhanced image of continuous frames); after the above-mentioned preliminary combination operation, the second motion vector information obtained is also the second motion vector information of each pixel point in multiple frames, that is, the second motion vector information of each pixel point in the multiple frames based on the background subtraction image.

[0085] Step S342: obtaining a comprehensive abnormal value based on the second motion vector information of the t-th frame through local space analysis, continuous frame timing analysis, and mutation analysis of the current frame;

[0086] Step S343: Screening based on the comprehensive abnormal value to obtain abnormal second motion vector information;

[0087] It should be noted that the above embodiment of the present application first presets a comprehensive abnormal value threshold, and when the comprehensive abnormal value is greater than or equal to the comprehensive abnormal value threshold, the second motion vector information corresponding to the comprehensive abnormal value is determined to be abnormal second motion vector information.

[0088] Step S344: Smoothing the abnormal second motion vector information to obtain a target motion vector.

[0089] It should be noted that the smoothing processing in the above-mentioned embodiment of the present application includes time domain smoothing, spatial smoothing, etc., which are common knowledge of those skilled in the art and will not be described in detail in this application. During the operation of step S343 in the above-mentioned embodiment of the present application, after the abnormal second motion vector information is obtained by screening, the second motion vector information that has not been screened out is directly identified as the target motion vector information 1. After the smoothing processing in step S344, the target motion vector (also referred to as target motion vector information 2) after the abnormal second motion vector information is smoothed is obtained. Finally, the target motion vector information 1 and the target motion vector information 2 are combined to construct a complete target motion vector. Subsequently, steps S40 to S60 are executed to obtain the classification information of the child.

[0090] The above-mentioned embodiment of the present application first combines the background subtraction image with the initial motion vector information to generate motion vector information of each pixel in continuous frames (i.e., second motion vector information), so that the motion vector of each pixel or region can better reflect its actual motion in the image, reduce background information interference, and thus improve the accuracy of the motion vector information. Specifically, by using technologies such as optical flow algorithm, the motion vector of each pixel in continuous frames is calculated, which can more accurately describe the motion characteristics of the pixel, especially when the target changes greatly between continuous frames. Moreover, since the influence of the background has been excluded from the background subtraction image, the second motion vector information after preliminary combination can more accurately reflect the motion state of the target, reducing the background interference. The error caused by the interference of background information is reduced; further, by comprehensively applying local space analysis, continuous frame timing analysis and mutation analysis of the current frame, combined with the motion information of the current frame and the previous and next frames, possible outliers are identified, and those outliers (such as mutation points) that are obviously different from the motion vectors of the neighborhood or historical frames are detected; further, the comprehensive outliers are screened by a preset outlier threshold to obtain abnormal second motion vector information, and the abnormal motion vector is corrected by smoothing processing to obtain a smoother and more stable target motion vector, remove the violent fluctuations of the motion vector caused by the outliers, ensure the continuity of the motion vector in time and space, and thus obtain more accurate motion vector information.

[0091] Furthermore, during the specific implementation of the above-mentioned embodiments of the present application, technicians also found that due to the presence of interference objects in front of the child, that is, between the child and the camera, such as leaves, flying insects or other tiny interference objects, these tiny interference objects may produce misleading motion data, thereby interfering with the analysis of the child's motion vector information. At the same time, when the child moves quickly, the change in motion between consecutive frames may not be smooth, and when the target is blocked or sudden or abnormal movement occurs, the motion vector information will fluctuate greatly. These tiny interference objects, sudden fast movements and sudden actions will have a great impact on the analysis of children's behavior.

[0092] See also Figure 6 as well as Figure 7 In step S342, the comprehensive outlier value obtained by local space analysis, continuous frame timing analysis, and mutation analysis of the current frame based on the second motion vector information of the t-th frame includes the following steps:

[0093] Step S3421: obtaining a local area image of each pixel in the background subtracted image through a fixed-size neighborhood window of the background subtracted image;

[0094] It should be noted that the above embodiment of the present application is to obtain the local area image corresponding to each pixel point in the background subtraction image by setting a fixed-size neighborhood window (which can be 3*3 or 5*5);

[0095] Step S3422: Calculate a spatial outlier based on the second motion vector information of the j-th pixel position of the i-th local area image of the t-th frame and the second motion vector information of the neighboring points of the j-th pixel position of the i-th local area image of the t-th frame;

[0096] Step S3423: Calculate a temporal smoothing coefficient based on the second motion vector information of the j-th pixel position of the ith local area image of the t-th frame and the second motion vector information of the j-th pixel position of the ith local area image of the t-1-th frame;

[0097] Step S3424: analyzing the local region second motion vector information of the consecutive frames in combination with the global dynamic analysis to obtain a local mutation evaluation value;

[0098] Step S3425: Calculating a comprehensive outlier value based on the spatial outlier, the temporal smoothing coefficient, and the local mutation assessment value;

[0099] The composite outlier The calculation method is:

[0100] = + γ;

[0101] Where, is a spatial outlier; is the time domain smoothing coefficient; is the mutation assessment value; 、 and γ are weight coefficients (especially, is a spatial outlier The weight coefficient of is the time domain smoothing coefficient The weight coefficient; γ is the mutation evaluation value weight coefficient of );

[0102] is the second motion vector information of the j-th pixel position in the i-th local area of ​​the t-th frame; is the mean of the second motion vector information of the S neighborhoods of the j-th pixel position in the i-th local area of ​​the t-th frame (that is, the mean of the second motion vector information of the pixel position j in the S local area i in the t-th frame);

[0103] is the standard deviation of the second motion vector information of the S neighborhoods of the j-th pixel position in the i-th local area in the t-th frame; is a smoothing factor used to prevent the denominator from being zero;

[0104] is the second motion vector information of the j-th pixel position in the i-th local area of ​​the t-1-th frame; is the standard deviation of the second motion vector information of the t-th frame;

[0105] It should be noted that the above-mentioned embodiment of the present application first obtains a local area image of the pixel point through a neighborhood window in the background subtraction image, captures the motion characteristics of each pixel point and its surrounding pixels, and not only focuses on the motion vector of a single pixel, but also considers the motion characteristics of its surrounding neighborhood; through local spatial analysis, the motion vectors of the surrounding pixels can be considered, thereby eliminating the slight interference information in the background, ensuring that the motion vector of each pixel more accurately reflects the actual movement of the target.

[0106] Furthermore, by comparing the motion vector of each pixel with the motion vectors of the pixels in its neighborhood, the spatial outlier is calculated to identify abnormal data that deviates significantly from its neighborhood value in space; further, by comparing the motion vector of the current frame with the previous frame, the time domain smoothing coefficient is calculated to measure the continuity and smoothness of the motion vector in time, and filter out abnormal data caused by sudden noise or movement; through continuous frame timing analysis, the continuity of the target in the time dimension is analyzed, the dynamic changes of the motion vector are identified and tracked, and abnormal motion patterns such as sudden pauses and rapid changes are detected. Unexpected abnormal motion is discovered in a timely manner, thereby ensuring the continuity and consistency of the motion vector in time;

[0107] Furthermore, by utilizing the second motion vector information of the local area of ​​continuous frames and combining it with global dynamic analysis, the degree of mutation in the local area is evaluated to capture abnormal situations caused by rapid changes or sudden movements; finally, the spatial outlier, the temporal smoothing coefficient and the local mutation evaluation value are combined for comprehensive calculation to obtain the final comprehensive outlier value, identify and process anomalies in the motion vector. By comprehensively considering space, time and mutation behavior, abnormal data can be more accurately identified and marked, thereby providing more stable motion vector data for subsequent smoothing, correction and further analysis.

[0108] Specifically, see Figure 7 In step S3424, the local region second motion vector information of the consecutive frames is analyzed in combination with the global dynamic analysis to obtain a local mutation evaluation value, including the following steps:

[0109] Step S34241: Calculate all second motion vector information of each local area of ​​the current frame to obtain global motion vector information parameters;

[0110] The global motion vector information parameters include the global motion direction mean and the global motion amplitude standard deviation;

[0111] It should be noted that, in the above-mentioned embodiment of the present application, since the motion vector information generally includes the motion velocity vector, the motion velocity amplitude, and the motion direction angle, the embodiment of the present application can directly calculate the above-mentioned global motion vector information parameters from the second motion vector information;

[0112] Step S34242: extracting the motion direction angle corresponding to the second motion vector information at each pixel position in the current frame and N consecutive frames before the current frame, constructing a direction field matrix based on the motion direction angle; performing principal component extraction on the direction field matrix to obtain a global direction angle;

[0113] Each element in the direction field matrix represents the motion direction angle of the j-th pixel position in the t-th frame;

[0114] Step S34243: constructing a global projection matrix P based on the global motion vector information parameters and the global direction angle in combination with the second motion vector information of each pixel position; calculating the global motion vector information V'' corresponding to the second motion vector information of each pixel position based on the global projection matrix P; calculating the Mahalanobis distance based on the global motion vector information V'' to obtain the difference between each global motion vector information V'' and the global motion vector parameter information; and constructing a local projection anomaly marker matrix A after screening multiple global motion vector information V'' based on the difference;

[0115] It should be noted that in the above-mentioned embodiment of the present application, the global projection matrix P refers to the second motion vector information of each pixel position mapped to the global space, thereby analyzing and evaluating whether these motion vectors conform to the global dynamic law. The core of the projection is to transform the motion vector from the coordinate system of each pixel position into the global coordinate system, thereby performing global consistency analysis to help identify anomalies and mutations.

[0116] First, a global model V is constructed using the global motion vector information parameters and global direction angles. Specifically, V=[ ];in, is the global motion direction mean in the global motion vector information parameter, is the global direction angle;

[0117] Then, the second motion vector information of each pixel position is mapped to the global model V to construct a global projection matrix P. The global projection matrix P can be expressed as: P = f(V, V'); where f() is the projection mapping function; V' is the second motion vector information of each pixel position;

[0118] Furthermore, after establishing the global projection matrix P, the second motion vector information of each pixel position can be mapped to the global space through matrix operation. Specifically, the second motion vector information of each pixel position is transformed into the global motion vector information V'' through matrix transformation, which is expressed as: V''=P×V';

[0119] Next, the global motion vector information V'' and the global motion vector information parameters are used to calculate the Mahalanobis distance to measure the distance (i.e. the above-mentioned gap) between each global motion vector information V'' and the distribution center (i.e. the above-mentioned global motion vector information parameters); when the above-mentioned gap is greater than the set gap threshold, it is proved that the second motion vector information of the pixel position corresponding to the global motion vector information V'' is abnormal; by analyzing multiple abnormal second motion vector information, a local projection abnormality marking matrix A is established.

[0120] Step S34244: performing temporal continuity detection and spatial consistency detection on each element in the local projection abnormality mark matrix A to generate comprehensive mark information;

[0121] It should be noted that, in the above-mentioned embodiment of the present application, each element in the projection abnormality mark matrix A is first tested for temporal continuity. Specifically, the velocity autocorrelation coefficient is calculated using the motion velocity amplitude of the abnormal second motion vector information of the current frame and the motion velocity amplitude of the second motion vector information of the corresponding pixel position of the consecutive k frames before the current frame. When it is determined that the velocity autocorrelation coefficient is less than a preset velocity autocorrelation coefficient threshold, the second motion vector information of the current frame is determined to be temporally discontinuous. The direction difference coefficient (the direction difference coefficient is a characterization of the directional change of the second motion vector information of the pixel position of adjacent frames) is further calculated by using the motion direction angle of the abnormal second motion vector information of the current frame and the motion direction angle of the second motion vector information of the corresponding pixel position of the consecutive k frames before the current frame. When it is determined that the directional difference coefficient changes twice in a row are both greater than the preset directional difference coefficient threshold, it is determined that the second motion vector information of the current frame has an abnormal turning (for example, a child suddenly stands up). Second motion vector information that meets both the requirements of the second motion vector information temporal discontinuity and the second motion vector information abnormal turning is deemed to have failed the temporal continuity test (for example, a child suddenly stands up but one leg is bent at the knee).

[0122] Furthermore, the elements in the local projection abnormality label matrix A are clustered to obtain a main motion cluster and an outlier motion cluster; wherein the number of second motion vector information within the bounding box of the main motion cluster is greater than a preset threshold value of the number of elements in the main motion cluster;

[0123] Furthermore, the second motion vector information corresponding to the unqualified temporal continuity test and the second motion vector information in the outlier motion cluster are marked to obtain a comprehensive abnormality mark set (i.e., the above-mentioned comprehensive mark information);

[0124] The velocity autocorrelation coefficient is calculated as follows:

[0125] ;in, is the motion speed amplitude of the second motion vector information at the current pixel position of the current frame; The previous frame of the current frame; is the average of the motion speed amplitudes of the current frame and the consecutive k frames before the current frame; is the standard deviation of the global motion amplitude;

[0126] Step S34245: Calculating the mutation intensity based on the comprehensive abnormality marker information (the mutation intensity is the above-mentioned local mutation evaluation value);

[0127] The calculation method of the mutation intensity is:

[0128] M=w'×ST+w''×LT+w''×STLT;

[0129] Wherein, w', w'' and w'' are weight coefficients; ST is the number of second motion vector information that fails the temporal consistency test; LT is the number of second motion vector information in the outlier motion cluster; STLT is the number of second motion vector information that fails the temporal consistency test and belongs to the outlier motion cluster at the same time;

[0130] It should be noted that the above-mentioned embodiment of the present application first calculates the second motion vector information of each local area of ​​the current frame to obtain global motion vector information parameters (including the global motion direction mean and the global motion amplitude standard deviation) to identify the overall motion pattern in the image sequence; further, the direction field matrix is ​​constructed to reveal the changing trend of the motion vector in different time frames, and the global direction angle is used to further determine the global dynamic pattern in the image, thereby improving the accuracy of anomaly recognition; further, the local motion vectors are mapped to the global space through the construction of the projection matrix, so as to analyze whether these motion vectors conform to the global dynamic law, and further determine the anomaly points that do not conform to the global motion pattern through the calculation of the Mahalanobis distance; by first performing a temporal continuity test on each element in the local projection anomaly marker matrix A , calculate the velocity autocorrelation coefficient and directional difference coefficient, and determine whether there is a temporal discontinuity or abnormal turning. Secondly, perform spatial consistency detection, identify and mark outlier motion clusters, and identify dynamic abnormal events, such as sudden changes in motion direction (such as a child suddenly standing up). It effectively identifies behaviors or abnormal states that are inconsistent with normal motion patterns, and improves the accuracy and reliability of detection. Finally, by integrating the abnormal marking information (including the second motion vector information that fails the temporal continuity test, the second motion vector information in the outlier motion cluster, and the number of second motion vector information that simultaneously fails the temporal continuity test and belongs to the outlier motion cluster), the mutation intensity, that is, the local mutation evaluation value, is calculated, which objectively reflects the severity of the local mutation events in the image sequence, thereby helping to determine whether there are significant dynamic changes or mutation behaviors.

[0131] An example of the technical solution employed in the above-described embodiment of the present application is provided below: After acquiring five consecutive frames of images, the second motion vector information for each pixel position is shown in Table 1. Furthermore, a comprehensive outlier value is calculated and analyzed through steps S3421 to S3425. Furthermore, during the execution of step S3424, the second motion vector information for the pixel position is marked, resulting in the marking information shown in Table 2. Table 2 shows that some pixels have abnormal motion vectors in consecutive frames (for example, (1,3) in Frame 1 and Frame 5). This may be misleading motion data caused by rapid motion or obstructions (such as leaves or flying insects). By calculating the comprehensive outlier value, this abnormal data can be effectively marked and further corrected or smoothed to obtain more stable motion vector data.

[0132] Table 1

[0133] Frame number Pixel position Second motion vector information (V) Frame 1 (1,1) (3, 2) Frame 1 (1,2) (2, 1) Frame 1 (1,3) (4, 3) Frame 2 (1,1) (3, 3) Frame 2 (1,2) (2, 2) Frame 2 (1,3) (3, 3) Frame 3 (1,1) (5, 2) Frame 3 (1,2) (1, 1) Frame 3 (1,3) (4, 2) Frame 4 (1,1) (4, 3) Frame 4 (1,2) (3, 2) Frame 4 (1,3) (2, 2) Frame 5 (1,1) (5, 4) Frame 5 (1,2) (2, 3) Frame 5 (1,3) (4, 4)

[0134] Table 2

[0135] Frame number Pixel position Second motion vector information (V) Spatial outliers Time domain smoothing coefficient Local mutation assessment value Comprehensive outliers Exception Marker Frame 1 (1,1) (3, 2) 0.2 0.1 0.4 0.3 normal Frame 1 (1,2) (2, 1) 0.1 0.2 0.3 0.2 normal Frame 1 (1,3) (4, 3) 0.3 0.4 0.5 0.4 abnormal Frame 2 (1,1) (3, 3) 0.1 0.05 0.2 0.15 normal Frame 2 (1,2) (2, 2) 0.15 0.05 0.25 0.2 normal Frame 2 (1,3) (3, 3) 0.2 0.3 0.4 0.3 abnormal Frame 3 (1,1) (5, 2) 0.4 0.3 0.6 0.5 abnormal Frame 3 (1,2) (1, 1) 0.2 0.3 0.35 0.3 normal Frame 3 (1,3) (4, 2) 0.25 0.2 0.4 0.35 normal Frame 4 (1,1) (4, 3) 0.3 0.4 0.5 0.4 abnormal Frame 4 (1,2) (3, 2) 0.2 0.3 0.4 0.3 normal Frame 4 (1,3) (2, 2) 0.1 0.15 0.3 0.2 normal Frame 5 (1,1) (5, 4) 0.5 0.3 0.7 0.6 abnormal Frame 5 (1,2) (2, 3) 0.4 0.25 0.5 0.45 normal Frame 5 (1,3) (4, 4) 0.6 0.3 0.8 0.7 abnormal

[0136] Example 2

[0137] On the other hand, the second embodiment of the present invention is based on the method for early warning monitoring of children's ADHD behavior based on video analysis provided in the first embodiment of the invention, and further provides a computer storage medium (hereinafter referred to as storage medium), which includes:

[0138] A memory for storing computer programs; a communication interface for connecting the memory to a processor; and a processor for executing the computer program to implement an embodiment disclosed in combination with any of the above-mentioned embodiments, involving a method for early warning monitoring of children's ADHD behavior based on video analysis.

[0139] In summary, the present invention proposes a method and storage medium for early warning monitoring of children's ADHD behavior based on video analysis, which realizes accurate extraction of children's motion features through the coordinated optimization of the optical flow algorithm and the background subtraction algorithm, and by combining the optical flow algorithm (capturing motion trajectory) and the background subtraction algorithm (separating dynamic targets from static background). The optical flow algorithm reduces motion blur errors through a multi-scale pyramid model, and the background subtraction algorithm introduces a time-weighted adaptive update mechanism to effectively deal with sudden changes in illumination and dynamic interference; further, a multi-dimensional anomaly assessment model is constructed through local spatial analysis (neighborhood window comparison), continuous frame timing analysis (calculation of time domain smoothing coefficients), and global dynamic analysis (direction field matrix and principal component extraction). The spatial outliers, time domain smoothing coefficients, and local mutation assessment values ​​are integrated to form a comprehensive anomaly value with dynamic weight adjustment, which significantly improves the robustness of mutation point detection;

[0140] Furthermore, for small distractors, rapid or sudden movements of the target during child behavior recognition, this application analyzes the comprehensive outliers from three aspects: local spatial analysis, continuous frame timing analysis, and mutations in the current frame. By comprehensively considering spatial, temporal, and mutation behaviors, it can more accurately identify and mark abnormal data, thereby providing more stable motion vector data for subsequent smoothing, correction, and further analysis.

[0141] Furthermore, projection mapping is used in combination with time series detection and spatial detection to identify outliers, thereby helping to determine whether there are significant dynamic changes or mutation behaviors.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for early warning monitoring of children's ADHD behavior based on video analysis, characterized in that: The steps are as follows: Acquire continuous surveillance video data from a preset surveillance area through a multi-channel video acquisition device; Preprocessing the surveillance video data to obtain a target area image; The motion feature extraction based on the optical flow algorithm and background subtraction algorithm is used on the segmented target motion area to obtain the motion vector information between consecutive frames; Calculating behavior parameters based on motion vector information between consecutive frames; the behavior parameters include motion amplitude, motion direction change, and duration of motion; Normalizing the behavioral parameters and assembling them into a multi-dimensional behavioral feature vector; The multi-dimensional behavior feature vector is compared with a pre-established standard behavior template to form classification information of the child's behavior in the target video.

2. The method for early warning monitoring of children's ADHD behavior based on video analysis according to claim 1, characterized in that: The segmented target motion region is subjected to motion feature extraction based on an optical flow algorithm and a background subtraction algorithm to obtain motion vector information between consecutive frames, including the following steps: Performing image enhancement and brightness normalization processing on the target motion area to obtain an enhanced image; Processing the enhanced image based on the continuous frames through a background subtraction algorithm to obtain a background subtracted image; The enhanced images based on the continuous frames are processed by multi-scale optical flow to estimate and obtain initial motion vector information; The target motion vector is obtained by analyzing the background subtracted image in combination with the initial motion vector information through regional correlation correction processing.

3. The method for early warning monitoring of children's ADHD behavior based on video analysis according to claim 2, characterized in that: The process of analyzing and obtaining a target motion vector based on the background subtracted image in combination with the initial motion vector information through regional correlation correction processing includes the following steps: Preliminarily combining the background subtraction image with the initial motion vector information to obtain second motion vector information corresponding to each pixel position in the background subtraction image; A comprehensive outlier value obtained based on the second motion vector information of the t-th frame through local space analysis, continuous frame timing analysis, and mutation analysis of the current frame; Screening based on the comprehensive abnormal value to obtain abnormal second motion vector information; The abnormal second motion vector information is smoothed to obtain a target motion vector.

4. The method for early warning monitoring of children's ADHD behavior based on video analysis according to claim 3, characterized in that: The comprehensive outlier value obtained based on the second motion vector information of the t-th frame through local space analysis, continuous frame timing analysis, and mutation analysis of the current frame includes the following steps: Obtaining a local area image of each pixel point in the background subtraction image through a neighborhood window of a fixed size from the background subtraction image; A spatial outlier is obtained by calculating the second motion vector information of the j-th pixel position of the i-th local area image of the t-th frame and the second motion vector information of the neighboring point of the j-th pixel position of the i-th local area image of the t-th frame; A temporal smoothing coefficient is obtained by calculating according to the second motion vector information of the j-th pixel position of the i-th local area image of the t-th frame and the second motion vector information of the j-th pixel position of the i-th local area image of the t-1-th frame; The local mutation evaluation value is obtained by analyzing the second motion vector information of the local area of ​​the continuous frames in combination with the global dynamic analysis; A comprehensive outlier value is obtained by calculation based on the spatial outlier, the temporal smoothing coefficient, and the local mutation evaluation value.

5. The method for early warning monitoring of children's ADHD behavior based on video analysis according to claim 4, characterized in that: The composite outlier The calculation method is: Where, is a spatial outlier; is the time domain smoothing coefficient; is the mutation assessment value; α, β and γ are weight coefficients; is the second motion vector information of the j-th pixel position in the i-th local area of ​​the t-th frame; is the mean of the second motion vector information of the S neighborhoods of the j-th pixel position in the i-th local area of ​​the t-th frame; is the standard deviation of the second motion vector information of the S neighborhoods of the j-th pixel position in the i-th local area in the t-th frame; ε is a smoothing factor used to prevent the denominator from being zero; is the second motion vector information of the jth pixel position in the ith local area of ​​the t-1th frame; t is the standard deviation of the second motion vector information of the t-th frame.

6. The method for early warning monitoring of children's ADHD behavior based on video analysis according to claim 5, characterized in that: The method of analyzing the local region second motion vector information of the continuous frames in combination with the global dynamic analysis to obtain the local mutation evaluation value includes the following steps: Calculating all second motion vector information of each local area of ​​the current frame to obtain a global motion vector information parameter; The global motion vector information parameters include a global motion direction mean and a global motion amplitude standard deviation; Extracting the motion direction angle corresponding to the second motion vector information at each pixel position in the current frame and N consecutive frames before the current frame, constructing a direction field matrix based on the motion direction angle; performing principal component extraction on the direction field matrix to obtain a global direction angle; Each element in the direction field matrix represents the motion direction angle of the j-th pixel position in the t-th frame; Based on the global motion vector information parameter and the global direction angle and in combination with the second motion vector information of each pixel point posture, a Mahalanobis distance obtained by projection analysis is used to perform screening and marking to obtain a local projection abnormality marking matrix A; Each element in the local projection abnormality mark matrix A is processed by temporal continuity detection and spatial consistency detection to generate comprehensive mark information; The mutation intensity is calculated based on the comprehensive abnormal marker information.

7. The method for early warning monitoring of children's ADHD behavior based on video analysis according to claim 6, characterized in that: Based on the global motion vector information parameter and the global direction angle and in combination with the second motion vector information of each pixel point posture, a Mahalanobis distance obtained by projection analysis is used to perform screening and marking to obtain a local projection abnormality marking matrix A, including the following steps: A global projection matrix P is constructed based on the global motion vector information parameters and the global direction angle and combined with the second motion vector information of each pixel position; the global motion vector information V" corresponding to the second motion vector information of each pixel position is calculated based on the global projection matrix P; the Mahalanobis distance is calculated based on the global motion vector information V", and the gap between each global motion vector information V" and the global motion vector parameter information is obtained; based on the gap, multiple global motion vector information V" are screened to obtain local projection anomaly mark matrix A.

8. A storage medium, characterized in that: The storage medium includes: a memory, a communication interface and a processor; the processor is used to execute a computer program to implement a method for early warning monitoring of children's ADHD behavior based on video analysis as described in any one of claims 1 to 7 above.

Images