Social intervention training method and system for children with autism

Abstract

The present invention relates to the field of image interpolation and enhancement technology, and in particular to a social intervention training method and system for autistic children. The present invention obtains the necessity of interpolation of each pixel point in the current child area image based on the grayscale difference between each pixel point in the child area image and the neighboring pixel points, combined with the position change of each pixel point in the continuous frame image; further obtains the dimension to be interpolated based on the overall characteristics of the interpolation necessity of all pixels in each dimension, combined with the difference characteristics of the interpolation necessity of pixels in adjacent dimensions; finally, interpolates the current child area image based on the grayscale change characteristics of the pixels in the dimension to be interpolated and the adjacent dimensions to obtain a child area enhanced image. The present invention accurately locates the area that needs to be interpolated in the child area image, enhances the details of the key parts of the child area image, ensures the accuracy of auxiliary recognition, and improves the effect of intervention training for autistic children.

Description

Technical Field

[0001] The present invention relates to the technical field of image interpolation enhancement, and in particular to a social intervention training method and system for autistic children. Background Art

[0002] Children with autism, also known as autism, often face difficulties understanding and responding to social emotions. Therefore, social storytelling training can help children with autism improve their social skills, creating more favorable conditions for their growth and development. To avoid the subjective bias of manual identification of children's behavior, image-assisted systems are now being used to identify children's behavior more objectively.

[0003] However, in the current process of social story training for children with the help of auxiliary systems, due to the large individual differences among autistic children, and the small size of the target children in the collected images, unclear target features, incomplete details and other problems, it is easy to cause the system's misrecognition rate to increase, affecting the effectiveness of social story training for autistic children. Summary of the Invention

[0004] In order to solve the technical problem that the auxiliary system is affected by individual differences and image acquisition problems, resulting in unsatisfactory recognition accuracy and affecting the effectiveness of social story training for autistic children, the purpose of the present invention is to provide a social intervention training method and system for autistic children. The technical solutions adopted are as follows:

[0005] A social intervention training method for children with autism, comprising:

[0006] Real-time acquisition of images of the children's area in the test area;

[0007] In the current child region image, a grayscale highlighting coefficient of each pixel is obtained based on the grayscale difference between each pixel of the child region image and neighboring pixels in a preset neighborhood; the necessity of interpolation of each pixel is obtained based on the position change of each pixel in consecutive frame images in combination with the grayscale highlighting coefficient; the dimension to be interpolated is obtained based on the overall characteristics of the interpolation necessity of all pixels in each dimension in combination with the difference characteristics of the interpolation necessity of pixels in adjacent dimensions; the dimension is divided into row dimension and column dimension;

[0008] The current child region image is interpolated according to the grayscale variation characteristics of the pixel points in the dimension to be interpolated and the adjacent dimensions to obtain a child region enhanced image.

[0009] Furthermore, the method for obtaining the grayscale highlight coefficient includes:

[0010] Obtaining the number of neighboring pixels whose grayscale values ​​differ between each pixel and neighboring pixels in a preset neighborhood as a first grayscale difference parameter for each pixel;

[0011] The average of the absolute values ​​of the grayscale differences between each pixel and its neighboring pixels in the preset neighborhood is used as the second grayscale difference parameter of each pixel;

[0012] The product of the first grayscale difference parameter and the second grayscale difference parameter of each pixel is used as the grayscale highlighting coefficient of each pixel.

[0013] Furthermore, the method for obtaining the necessity of interpolation includes:

[0014] Obtain two adjacent historical frames of the current child region image, and use the sum of the Euclidean distances of the coordinates of each pixel point between the temporally adjacent images as the position variation coefficient of each pixel point;

[0015] The product of the position variation coefficient and the grayscale highlighting coefficient of each pixel point is used as the interpolation necessity of each pixel point.

[0016] Furthermore, the method for obtaining the dimension to be interpolated includes:

[0017] Select any dimension as the target dimension; take the average value of the interpolation necessity of all pixels of the target dimension as the first necessary coefficient; normalize the sum of the differences between the interpolation necessity of the pixel points of the target dimension and the interpolation necessity of the pixel points with the same sequence number in the adjacent dimension as the second necessary coefficient; and take the product of the first necessary coefficient and the second necessary coefficient of the target dimension as the interpolation necessity of the target dimension;

[0018] The dimensions to be interpolated are obtained by screening according to the interpolation necessity of each dimension.

[0019] Furthermore, the method of screening and obtaining the dimensions to be interpolated according to the interpolation necessity of each dimension includes:

[0020] Arrange the interpolation necessity of all the row dimensions from large to small, and select the row dimensions of a preset proportion as dimensions to be interpolated;

[0021] The interpolation necessity of all the column dimensions is arranged in order from large to small, and the column dimensions of a preset proportion are selected as dimensions to be interpolated.

[0022] Furthermore, the method for obtaining the children's region enhanced image includes:

[0023] Insert an interpolation dimension between each of the to-be-interpolated dimensions and the adjacent dimension; select any pixel point of any of the interpolation dimensions as a target pixel point;

[0024] In a preset interpolation neighborhood of the target pixel point, the average of the absolute values ​​of the grayscale value differences between the pixels in the adjacent dimensions of the same interpolation dimension to which the target pixel point belongs is used as the numerator; the maximum absolute value of the grayscale value differences between the pixels in the adjacent dimensions of the same interpolation dimension to which the target pixel point belongs is used as the denominator, and the fractional ratio is used as the grayscale variation coefficient;

[0025] The product of the absolute value of the grayscale value difference between the pixels of known grayscale values ​​on both sides of the target pixel and the grayscale variation coefficient is used as the subtrahend, the maximum grayscale value of the pixels of known grayscale values ​​on both sides of the target pixel is used as the minuend, and the difference is used as the grayscale value of the target pixel;

[0026] The grayscale values ​​of all pixels in all the interpolation dimensions are obtained to obtain an enhanced image of the child region.

[0027] Furthermore, after obtaining the children's region enhanced image, the method further includes:

[0028] Edge detection is performed on the enhanced image of the child area, and the key points of the human body are extracted using a key point detection algorithm. The feature vector of each key point is obtained using a SIFT feature descriptor algorithm, and the extracted feature vector is input into a pre-trained classifier to identify whether the child's action is wrong.

[0029] Furthermore, the preset neighborhood is an eight-neighborhood centered on the pixel point.

[0030] Furthermore, the preset ratio is 50%.

[0031] The present invention also proposes a social intervention training system for children with autism, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it implements any one of the steps of the social intervention training method for children with autism.

[0032] The present invention has the following beneficial effects:

[0033] The present invention first obtains the child area image in the test site in real time, provides a basis for subsequent analysis, and reduces the interference of irrelevant background areas; further, in the current child area image, according to the grayscale difference between each pixel point of the child area image and the neighboring pixel points in the preset neighborhood, combined with the position change of each pixel point in the continuous frame image, from the two perspectives of grayscale change and spatial position change, the necessity of pixel interpolation is evaluated, so as to determine the appropriate interpolation position in the future and prepare for detail enhancement of the edge, contour and motion area in the image; further facilitates interpolation of the entire row or column, and according to the overall characteristics of the interpolation necessity of all pixels in each dimension, combined with the difference characteristics of the interpolation necessity of pixels in adjacent dimensions, obtains the dimension to be interpolated, and accurately locates the interpolation area; finally, according to the grayscale change characteristics of the pixels in the dimension to be interpolated and the adjacent dimensions, the current child area image is interpolated to obtain a child area enhanced image, improve the quality of the key areas in the image, and provide more reliable data support for intervention training. The present invention accurately locates the area that needs to be interpolated in the child area image, performs detail enhancement on the key parts of the child area image, ensures the accuracy of auxiliary recognition, and improves the effect of intervention training for children with autism. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A flowchart of a social intervention training method for autistic children provided by one embodiment of the present invention;

[0036] Figure 2 A flowchart of a method for acquiring an enhanced image of a child region provided by one embodiment of the present invention;

[0037] Figure 3 A schematic diagram of column-dimensional interpolation provided by an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of row-dimensional interpolation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail a social intervention training method and system for children with autism proposed in accordance with the present invention, its specific implementation, structure, features and effects. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics of one or more embodiments may be combined in any suitable form.

[0040] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0041] The following describes in detail a social intervention training method and system for autistic children provided by the present invention with reference to the accompanying drawings.

[0042] See also Figure 1 , which shows a flow chart of a social intervention training method for autistic children provided by one embodiment of the present invention, specifically including:

[0043] Step S1: Acquire an image of the children's area in the test site in real time.

[0044] This invention primarily relates to social intervention training for children with autism. Before conducting intervention training for autistic children, it is necessary to first observe the children's current problems and difficulties and record them in detail. Based on these observations, themes for social stories are then determined and presented in a specific format to observe and intervene in the children's social behavior.

[0045] Implementers can choose to present the story in the form of text, illustrations, PPT or video, and after the presentation, a demonstrator such as a teacher or parent will demonstrate the correct behavior; an image acquisition device is installed in the test site, and after the demonstrator makes an action gesture, a camera or other shooting device captures the demonstrator's behavior image; the image in the test site is captured in real time by the camera, and the image is pre-processed and denoised using an adaptive histogram equalization algorithm, and the facial image information of the autistic child is input into the neural network model to identify the child area in the captured image, and perform character recognition to obtain the child area image, which provides a basis for subsequent analysis and reduces interference from irrelevant background areas.

[0046] It should be noted that social stories should be brief, understandable, and narrated in the first person. They should also describe expected behaviors. Implementers can plan and install their own image capture equipment to ensure that each child's complete posture is captured. A single camera can be set to track and capture a single child, or multiple cameras can be set to capture from multiple angles. From the captured images, select images in which the child and the demonstrator are facing the same direction, such as both facing forward or back.

[0047] It should be noted that the image acquisition frequency can be set by the implementer. As an example, it is set to 30Hz. To improve computing efficiency, the last image of every three images is analyzed, and the other two images are used to analyze the position changes of pixel points. For example, in images with serial numbers 1, 2, 3, 4, 5, 6, 7, 8, and 9, images 3, 6, and 9 are analyzed. The implementer can write or select existing social stories. It is an existing technology to use neural networks to identify people and locate children's areas in images to obtain children's area images, which will not be repeated here.

[0048] Step S2: In the current child area image, the grayscale highlighting coefficient of each pixel is obtained based on the grayscale difference between each pixel of the child area image and the neighboring pixels in the preset neighborhood; the interpolation necessity of each pixel is obtained based on the position change of each pixel in the continuous frame image and the grayscale highlighting coefficient; the dimension to be interpolated is obtained based on the overall characteristics of the interpolation necessity of all pixels in each dimension and the difference characteristics of the interpolation necessity of pixels in adjacent dimensions; the dimensions are divided into row dimensions and column dimensions.

[0049] During the intervention training process, in order to prevent the camera from putting pressure on children or the children from changing their natural behavior and becoming restrained due to the awareness of the camera, which will affect the training effect, the image acquisition equipment is usually installed at a distance from the children. At the same time, the children's activity range is large, and it is necessary to avoid the target being lost or incomplete due to children's activities. The focal length is limited, resulting in the target child being smaller in the captured image, the target features being unclear, and the details being incomplete. Therefore, it is necessary to enhance the details of the child area image to ensure the accuracy of auxiliary recognition and improve the effect of intervention training for children with autism.

[0050] Considering that a certain posture of a child is generally judged based on the arms, legs and other parts, and the grayscale difference between the pixels on the edges of these parts and the surrounding pixels is large, interpolation near these pixels can enhance the continuity of these edge areas, making it easier for the feature extraction algorithm to accurately identify key parts and facilitate analysis of posture details. Therefore, in the current child area image, the grayscale highlighting coefficient of each pixel is obtained based on the grayscale difference between each pixel in the child area image and the neighboring pixels in the preset neighborhood, in preparation for determining the appropriate interpolation position.

[0051] Preferably, in one embodiment of the present invention, considering that the greater the number of neighboring pixels having differences between a pixel point and pixels in the neighborhood, the greater the average value of the absolute value of the grayscale value difference, indicating that the grayscale difference between the pixel point and its corresponding neighboring pixels is greater, based on this, the number of neighboring pixels having grayscale values ​​different between each pixel point and neighboring pixels in a preset neighborhood is obtained as the first grayscale difference parameter of each pixel point;

[0052] The average of the absolute values ​​of the grayscale differences between each pixel and its neighboring pixels in the preset neighborhood is used as the second grayscale difference parameter of each pixel;

[0053] The product of the first grayscale difference parameter and the second grayscale difference parameter of each pixel is used as the grayscale highlighting coefficient of each pixel.

[0054] As an example, the preset neighborhood of a pixel point is an eight-neighborhood neighborhood centered on the pixel point. The grayscale difference between the pixel point and the neighboring pixels in the preset neighborhood is jointly expressed by the number of neighboring pixels in the eight neighborhoods that have a grayscale value difference with the central pixel point, and the average value of the absolute value of the difference in grayscale values ​​between the central pixel point and the neighboring pixels.

[0055] It should be noted that the implementer may also adopt other preset neighborhoods such as four neighborhoods, which and the eight neighborhoods are already well known technical means to those skilled in the art and will not be described in detail.

[0056] Taking into account that the position change of pixel points in continuous frame images reflects the characteristics of children's posture changes, since autistic children may show inattention when engaging in a certain activity (see "Neural Mechanism of Motor Development Disorders in Autistic Children" by Wang Lin), which leads to erratic movements and difficulty in fixing, and when children's arms or legs are used quickly between continuous frames, single-frame images may be unclear due to motion blur. Therefore, based on the position change of each pixel point in the continuous frame image and the grayscale highlighting coefficient, the necessity of interpolation of each pixel point is obtained, and the degree of necessity of interpolation around the pixel point is characterized, which facilitates the precise positioning of key areas that require interpolation enhancement.

[0057] Preferably, in one embodiment of the present invention, considering that by establishing a coordinate system to analyze the coordinate changes of pixel points, the position changes of pixel points can be intuitively reflected, and the Euclidean distance can measure the distance length between coordinates, the Euclidean distance of the coordinates of pixel points in adjacent frame images is analyzed in the coordinate system to represent the position changes of pixel points in consecutive frame images;

[0058] Based on this, we obtain two adjacent historical images of the current child area image, and use the sum of the Euclidean distances of the coordinates of each pixel point between the temporally adjacent images as the position variation coefficient of each pixel point; that is, we obtain the Euclidean distance between the coordinates of the pixel point in the first historical image and the coordinates of the second historical image, and the Euclidean distance between the coordinates of the second historical image and the coordinates of the current child area image, and use the sum of these two Euclidean distances as the position variation coefficient of this pixel point;

[0059] The product of the position variation coefficient of each pixel and the grayscale highlight coefficient is used as the interpolation necessity of each pixel.

[0060] It should be noted that in multiple consecutive frames of images, locating the same pixel point can be achieved through optical flow method, corner detection and other methods. A two-dimensional rectangular coordinate system is established at any corner of the image to obtain the coordinates of the pixel point and thus calculate the Euclidean distance. These are all existing technologies and will not be repeated here.

[0061] It should be noted that the interpolation necessity is a specific numerical value.

[0062] Taking into account that when interpolating an image, usually a row or a column is interpolated, the dimension to be interpolated should be determined based on the overall characteristics of the interpolation necessity of all pixels in each dimension; and considering that the interpolation necessity integrates the position change of the pixel points and the local highlight characteristics of the grayscale, the difference characteristics of the interpolation necessity of the pixels in adjacent dimensions reflect the differences of the pixels in adjacent dimensions and reflect the local changes of the image, especially the changes in the edges, textures or motion areas. Therefore, the difference characteristics of the interpolation necessity of the pixels in adjacent dimensions are combined to obtain the dimension to be interpolated and accurately locate the interpolation area, so as to improve the quality of the key areas in the image, ensure the accuracy of auxiliary recognition, and improve the effect of intervention training for children with autism.

[0063] In a preferred embodiment of the present invention, any dimension is selected as the target dimension to facilitate analysis one by one;

[0064] Considering that the larger the mean of the interpolation necessity of all pixels in a certain dimension is, the greater the demand for interpolation of the overall pixels in the dimension is, the average of the interpolation necessity of all pixels in the target dimension is used as the first necessary coefficient;

[0065] Considering the necessity of interpolation of pixels in the target dimension, the greater the interpolation necessity of pixels with the same sequence number in adjacent dimensions, the more interpolation is needed in the target dimension compared with the adjacent dimensions. Therefore, the sum of the differences between the interpolation necessity of pixels in the target dimension and the interpolation necessity of pixels with the same sequence number in adjacent dimensions is normalized and used as the second necessary coefficient.

[0066] The product of the first necessary coefficient and the second necessary coefficient of the target dimension is used as the interpolation necessity of the target dimension; and the dimension to be interpolated is obtained by screening according to the interpolation necessity of each dimension.

[0067] As an example, the target dimension is compared only with the next adjacent dimension, and the last dimension is compared only with the adjacent dimension. For example, if the target dimension is the row dimension and the dimension sequence number is 5, the sixth dimension in the row dimension is compared with the target dimension. The calculation formula for interpolation necessity includes:

[0068] ;

[0069] Where i represents the serial number of the child region image; k represents the serial number of the child region; x represents the serial number of the row dimension; Indicates the necessity of interpolation of the x-th row dimension of the k-th child region in the i-th child region image; It represents the average value of the interpolation necessity of the pixels in the x-th row dimension of the k-th child region in the i-th child region image, which is also the first necessary coefficient; v represents the sequence number of the pixel point; V represents the number of pixels; Indicates the necessity of interpolation of the vth pixel in the xth row dimension of the kth child region in the i-th child region image; Indicates the necessity of interpolation of the vth pixel in the x+1th row dimension of the kth child region in the i-th child region image; represents linear normalization; Represents the second necessary coefficient of the x-th row dimension of the k-th child region in the i-th child region image.

[0070] In the calculation formula of interpolation necessity, the overall characteristics of interpolation necessity of all pixels in the dimension are expressed in the form of average value; The necessity of interpolation of pixels in the target dimension is expressed by the difference between the pixels with the same sequence number in the adjacent dimension. Then the value range is adjusted by normalization. Supplement and obtain the necessity of interpolation of the target dimension, providing a basis for accurately determining the dimension to be interpolated.

[0071] It should be noted that an image may contain multiple children, and multiple child areas are identified, so the child areas are numbered; for the located child area, which is more detailed, that is, the edge contour of the child area is the contour of the child's body, and the edge of the child area is irregular, such as there are 100 pixels in the 5th row, 105 pixels in the 6th row, and 113 pixels in the 7th row, the implementer can set the comparison of pixels of different dimensions with the same column coordinates. For example, if the first pixel in the 5th row is in the 20th column, it is compared with the pixel in the 20th column of the 6th row. When there is no pixel in the 6th row on the column where a pixel in the 5th row is located, the interpolation necessity is set to 0; the implementer can also set the extraction of a square child area image containing part of the background, so that the number of columns in each row is the same and the number of rows in each column is the same.

[0072] It should be noted that the implementer can also express the overall characteristics of the interpolation necessity of all pixel points in the dimension by taking the mean, mode and median of the interpolation necessity of all pixel points in the target dimension as the first necessary coefficient, with weighted weights of 0.5, 0.3 and 0.2.

[0073] Preferably, in one embodiment of the present invention, considering that the greater the interpolation necessity of each dimension, the more interpolation enhancement is needed, the interpolation necessity of all row dimensions is arranged from large to small, and a preset proportion of row dimensions are selected as the dimensions to be interpolated;

[0074] Arrange the interpolation necessity of all column dimensions from large to small, and select the column dimensions with a preset ratio as the dimensions to be interpolated.

[0075] As an example, the preset ratio is 50%, that is, among all row dimensions, the row dimensions with the top 50% interpolation necessity are selected as the dimensions to be interpolated; among all column dimensions, the column dimensions with the top 50% interpolation necessity are selected as the dimensions to be interpolated.

[0076] It should be noted that dimensions are divided into row dimensions and column dimensions. Adjacent dimensions refer to dimensions with adjacent serial numbers of the same type, such as row dimensions are adjacent to row dimensions, and column dimensions are adjacent to column dimensions. The preset ratio can be adjusted according to actual application.

[0077] Step S3: interpolate the current child region image according to the grayscale change characteristics of the pixel points in the interpolated dimension and the adjacent dimensions to obtain the child region enhanced image.

[0078] By screening the dimensions to be interpolated and determining the specific rows or columns that need to be interpolated, the image can be interpolated and enhanced. Considering that the purpose of interpolation is to enhance the details and clarity of key areas in the image while maintaining the naturalness and continuity of the image, the current child area image is interpolated according to the grayscale change characteristics of the pixels in the dimension to be interpolated and the adjacent dimensions to obtain an enhanced image of the child area, which better preserves the natural transition of the image, improves the image details and clarity, and provides higher quality data support for intervention training.

[0079] Preferably, in one embodiment of the present invention, the method for obtaining the enhanced image of the children's region includes: Figure 2 , which shows a flow chart of a method for obtaining a child region enhanced image provided by an embodiment of the present invention, specifically comprising:

[0080] Step S301: insert an interpolation dimension between each dimension to be interpolated and the adjacent dimension; select any pixel point of any interpolation dimension as a target pixel point.

[0081] See also Figure 3 , which shows a schematic diagram of column dimension interpolation provided by an embodiment of the present invention; please refer to Figure 4 , which shows a schematic diagram of row-dimensional interpolation provided by an embodiment of the present invention.

[0082] Figure 3 In the figure, the pixels marked with numbers 1 and 3 are in a column and are the dimensions to be interpolated. The pixels marked with numbers 2 and 4 are in a column. The pixel marked with letter a is the target pixel, and its column is the interpolation dimension. Figure 4 In the figure, the pixels marked with numbers 1 and 3 are in one row and are the dimensions to be interpolated. The pixels marked with numbers 2 and 4 are in one row. The pixels marked with letter a are the target pixels and are in one row of interpolation dimensions.

[0083] It should be noted that whether to perform column interpolation or row interpolation first can be set by the implementer. When the dimension to be interpolated is at the edge of the image, interpolation is performed toward the inner side of the image. For example, if the image originally has 200 columns and the 200th column is the dimension to be interpolated, the corresponding interpolation dimension is inserted between the 199th and 200th columns. In one embodiment of the present invention, column interpolation is performed first and then row interpolation.

[0084] Step S302: Within the preset interpolation neighborhood of the target pixel point, the average of the absolute values ​​of the grayscale value differences between the pixels in the adjacent dimensions of the same interpolation dimension to which the target pixel point belongs is used as the numerator; the maximum value of the absolute value of the grayscale value differences between the pixels in the adjacent dimensions of the same interpolation dimension to which the target pixel point belongs is used as the denominator, and the fractional ratio is used as the grayscale variation coefficient.

[0085] When the target pixel is column interpolated, see Figure 3 The preset interpolation neighborhood is the pixels marked by 1, 2, 3, and 4, that is, the pixels to the left, right, lower left, and lower right of the target pixel. When the target pixel is to be interpolated in a row, please refer to Figure 4 The preset interpolation neighborhood is the pixels marked by 1, 2, 3, and 4, that is, the pixels above, below, upper right, and lower right of the target pixel.

[0086] Considering that the naturalness and continuity of the image need to be maintained when the target pixel is interpolated between pixels in two dimensions, it is necessary to analyze the grayscale changes between pixels in the same adjacent dimensions of the interpolation dimension to which the target pixel belongs within the preset interpolation neighborhood, so as to ensure that the interpolated image is smoother.

[0087] Based on this, the calculation formula of the grayscale variation coefficient includes:

[0088] ;

[0089] in, Indicates the grayscale variation coefficient of the a-th target pixel; Indicates the absolute value of the grayscale value difference between the first pair of pixels in the adjacent dimension of the same interpolation dimension to which the target pixel belongs within the preset interpolation neighborhood of the a-th target pixel. , Indicates the grayscale value of the pixel marked as 1 corresponding to the a-th target pixel, Indicates the grayscale value of the pixel marked as 2 corresponding to the a-th target pixel; Indicates the absolute value of the grayscale value difference between the second pair of pixels in the adjacent dimension of the same interpolation dimension to which the target pixel belongs within the preset interpolation neighborhood of the a-th target pixel. , Indicates the grayscale value of the pixel marked as 3 corresponding to the a-th target pixel, Indicates the grayscale value of the pixel marked as 4 corresponding to the a-th target pixel; Indicates taking the maximum value.

[0090] In the calculation formula of the grayscale variation coefficient, the average value of the absolute value of the grayscale value difference between pixels in the adjacent dimensions of the same interpolation dimension to which the target pixel belongs is used to express the degree of grayscale variation around the target pixel, reflecting the grayscale variation characteristics of the pixels in the interpolation dimension and the adjacent dimensions; and the maximum value of the absolute value of the grayscale value difference between pixels in the adjacent dimensions of the same interpolation dimension to which the target pixel belongs is used for further normalization, thereby limiting the value range of the grayscale variation coefficient and expressing the grayscale change ratio that the target pixel should have.

[0091] Step S303: The absolute value of the grayscale value difference between the pixels with known grayscale values ​​on both sides of the target pixel point is multiplied by the grayscale variation coefficient as the subtrahend, the maximum grayscale value of the pixels with known grayscale values ​​on both sides of the target pixel point is taken as the minuend, and the difference is taken as the grayscale value of the target pixel point.

[0092] The calculation formula for the grayscale value of the target pixel includes:

[0093] ;in, Indicates the grayscale value of the a-th target pixel.

[0094] In the calculation formula of the gray value of the target pixel, Indicates the grayscale change amplitude of the target pixel point, and uses the maximum grayscale value of the pixels with known grayscale values ​​on both sides of the target pixel point as the minuend, subtracting The method ensures that the grayscale change direction of the target pixel and the pixels on both sides are consistent, that is, the gradient direction is consistent. Keeping the intensity of grayscale changes consistent maintains the continuity of the image, making the image appear smoother.

[0095] Step S304: Obtain the grayscale values ​​of all pixels in all interpolation dimensions to obtain the enhanced image of the child region.

[0096] Each pixel point in the interpolation dimension is further analyzed one by one to obtain the grayscale values ​​of all pixels in all interpolation dimensions, thereby obtaining an enhanced image of the child area, enhancing the characteristic expression of the child's behavior and posture, and providing more reliable data support for the auxiliary system.

[0097] It should be noted that, in one embodiment of the present invention, after obtaining the enhanced image of the children's region, the following steps are further included:

[0098] Edge detection is performed on the enhanced image of the child area, and the key points of the human body are extracted using the key point detection algorithm. The feature vector of each key point is obtained using the SIFT feature descriptor algorithm. The extracted feature vector is input into the pre-trained classifier to identify whether the child's action is wrong.

[0099] Specifically, Canny edge detection is performed on the processed, enhanced image of the child region to help extract the child's outline and posture information. The Open Pose keypoint detection algorithm is then used to detect key points (such as joints) in the edge-detected image. The positions and relative relationships of these key points can reflect the body's movements and posture.

[0100] Then use the SIFT feature descriptor algorithm to calculate the feature vector of each key point:

[0101] a) First calculate the information around the key point: including pixel value, gradient, and direction.

[0102] b) We then compute some statistics, dividing the gradient around the keypoint by direction and calculating the gradient strength in each direction. We then combine these gradient strengths into a histogram. We can also compute other statistics, such as average brightness, contrast, and gradient magnitude around the keypoint.

[0103] c) Encode the calculated statistical information into a feature vector.

[0104] Finally, the extracted feature vectors are fed into a pre-trained classifier, which reflects the child's specific movements based on the learned mapping relationships. The identified child's movements are then compared with the teacher's to identify any errors. If errors are detected, a prompt is issued to the demonstrator or other relevant personnel, helping to identify problems encountered by children with autism in social storytelling and improving the effectiveness of social storytelling training for children with autism.

[0105] It should be noted that the technologies used in the further processing of the enhanced image of the children's region are all existing technologies and will not be described in detail here.

[0106] An embodiment of the present invention also provides a social intervention training system for children with autism, which includes a memory, a processor, and a computer program, wherein the memory is used to store the corresponding computer program, and the processor is used to run the corresponding computer program. When the computer program runs in the processor, it can implement a social intervention training method for children with autism described in steps S1-S3.

[0107] In summary, in order to address the technical problem that the auxiliary system is affected by individual differences and image acquisition problems, resulting in unsatisfactory recognition accuracy and affecting the effect of social story training for autistic children, the present invention obtains the child area image in the test site in real time; further, in the current child area image, the grayscale highlighting coefficient of each pixel point is obtained according to the grayscale difference between each pixel point of the child area image and the neighboring pixel points in the preset neighborhood; further, according to the position change of each pixel point in the continuous frame image, the necessity of interpolation of each pixel point is obtained in combination with the grayscale highlighting coefficient; further, according to the overall characteristics of the interpolation necessity of all pixels in each dimension, the difference characteristics of the interpolation necessity of pixels in adjacent dimensions are combined to obtain the dimension to be interpolated; finally, according to the grayscale change characteristics of the pixels in the dimension to be interpolated and the adjacent dimensions, the current child area image is interpolated to obtain the child area enhanced image.

[0108] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0109] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A social intervention training method for autistic children, characterized by: The method comprises: Real-time acquisition of images of the children's area in the test area; In the current child region image, a grayscale highlighting coefficient of each pixel is obtained based on the grayscale difference between each pixel of the child region image and neighboring pixels in a preset neighborhood; the necessity of interpolation of each pixel is obtained based on the position change of each pixel in consecutive frame images in combination with the grayscale highlighting coefficient; the dimension to be interpolated is obtained based on the overall characteristics of the interpolation necessity of all pixels in each dimension in combination with the difference characteristics of the interpolation necessity of pixels in adjacent dimensions; the dimension is divided into row dimension and column dimension; Interpolating the current child region image according to the grayscale variation characteristics of the pixel points in the dimension to be interpolated and the adjacent dimensions to obtain an enhanced image of the child region; The method for obtaining the grayscale highlight coefficient includes: Obtaining the number of neighboring pixels whose grayscale values ​​differ between each pixel and neighboring pixels in a preset neighborhood, and using the number as a first grayscale difference parameter for each pixel; The average of the absolute values ​​of the grayscale differences between each pixel and its neighboring pixels in the preset neighborhood is used as the second grayscale difference parameter of each pixel; The product of the first grayscale difference parameter and the second grayscale difference parameter of each pixel is used as the grayscale highlighting coefficient of each pixel; The method for obtaining the children's region enhanced image includes: Insert an interpolation dimension between each of the to-be-interpolated dimensions and the adjacent dimension; select any pixel point of any of the interpolation dimensions as a target pixel point; In a preset interpolation neighborhood of the target pixel point, the average of the absolute values ​​of the grayscale value differences between the pixels in adjacent dimensions of the same interpolation dimension to which the target pixel point belongs is used as the numerator; the maximum value of the absolute values ​​of the grayscale value differences between the pixels in adjacent dimensions of the same interpolation dimension to which the target pixel point belongs is used as the denominator, and the fractional ratio is used as the grayscale variation coefficient; The product of the absolute value of the grayscale value difference of the pixels of known grayscale values ​​on both sides of the target pixel and the grayscale variation coefficient is used as the subtrahend, the maximum grayscale value of the pixels of known grayscale values ​​on both sides of the target pixel is used as the minuend, and the difference between the minuend and the subtrahend is used as the grayscale value of the target pixel; An enhanced image of the child region is obtained according to the grayscale values ​​of the target pixels corresponding to all the interpolation dimensions.

2. A social intervention training method for autistic children according to claim 1, characterized in that: The method for obtaining the interpolation necessity includes: Obtain two adjacent historical frames of the current child region image, and use the sum of the Euclidean distances of the coordinates of each pixel point between the temporally adjacent images as the position variation coefficient of each pixel point; The product of the position variation coefficient and the grayscale highlighting coefficient of each pixel point is used as the interpolation necessity of each pixel point.

3. A social intervention training method for autistic children according to claim 1, characterized in that: The method for obtaining the dimension to be interpolated includes: Select any dimension as the target dimension; take the average value of the interpolation necessity of all pixels in the target dimension as the first necessary coefficient; normalize the sum of the differences between the interpolation necessity of the pixels in the target dimension and the interpolation necessity of the pixels with the same sequence number in the adjacent dimension and use it as the second necessary coefficient; and take the product of the first necessary coefficient and the second necessary coefficient of the target dimension as the interpolation necessity of the target dimension; The dimensions to be interpolated are obtained by screening according to the interpolation necessity of each dimension.

4. A social intervention training method for autistic children according to claim 3, characterized in that: The method for selecting and obtaining the dimensions to be interpolated according to the interpolation necessity of each dimension includes: Arrange the interpolation necessity of all the row dimensions from large to small, and select the row dimensions of a preset proportion as dimensions to be interpolated; The interpolation necessity of all the column dimensions is arranged in order from large to small, and the column dimensions of a preset proportion are selected as dimensions to be interpolated.

5. The social intervention training method for autistic children according to claim 1, characterized in that: After obtaining the children's region enhanced image, the method further includes: Edge detection is performed on the enhanced image of the child area, and the key points of the human body are extracted using a key point detection algorithm. The feature vector of each key point is obtained using a SIFT feature descriptor algorithm, and the extracted feature vector is input into a pre-trained classifier to identify whether the child's action is wrong.

6. The social intervention training method for autistic children according to claim 1, characterized in that: The preset neighborhood is an eight-neighborhood neighborhood centered on the pixel point.

7. The social intervention training method for autistic children according to claim 4, characterized in that: The preset ratio is 50%.

8. A social intervention training system for children with autism, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the social intervention training method for autistic children as described in any one of claims 1 to 7 are implemented.

Images