Social intervention training method and system for autistic children

Abstract

The invention relates to the technical field of image interpolation enhancement, in particular to a social intervention training method and system for autistic children. The method comprises the following steps: in a current child region image, according to the gray difference between each pixel point of the child region image and a neighborhood pixel point, and in combination with the position change of each pixel point in a continuous frame image, obtaining the interpolation necessity of each pixel point; further obtaining a dimension to be interpolated according to the overall features of the interpolation necessity of all the pixel points of each dimension in combination with the difference features of the interpolation necessity of the pixel points of adjacent dimensions; and finally, interpolating the current child region image according to the gray level change characteristics of the pixel points of the dimension to be interpolated and the adjacent dimension to obtain a child region enhanced image. According to the method, the region needing interpolation in the child region image is accurately positioned, detail enhancement is carried out on the key part of the child region image, the accuracy of auxiliary recognition is ensured, and the effect of intervention training on autistic children is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image interpolation enhancement, and particularly relates to a social intervention training method and system for autistic children. Background Art

[0002] Autism, also known as autism spectrum disorder, autistic children often face difficulties in understanding and coping with social emotions. Therefore, through social story training, it helps autistic children improve their social skills, creates more favorable conditions for the growth and development of autistic children, and is beneficial to the growth and development of autistic children. To avoid the subjective deviation of artificially identifying children's behaviors, nowadays, an image-assisted system is used to identify children's behaviors, making the identification more objective.

[0003] However, currently, during the process of using the auxiliary system for social story training of children, due to the large individual differences among autistic children, and the small size of the target children in the collected image frames, the target features are not obvious, and the details are incomplete, etc., it is easy to cause an increase in the system's misrecognition rate, affecting the effect of social story training for autistic children. Summary of the Invention

[0004] In order to solve the technical problem that the auxiliary system is affected by individual differences and image acquisition problems, resulting in an unsatisfactory recognition accuracy and affecting the effect of social story training for autistic children, the purpose of the present invention is to provide a social intervention training method and system for autistic children, and the specific technical solutions adopted are as follows: A social intervention training method for autistic children, the method includes: Real-time obtain the image of the children's area in the test site; In the current image of the children's area, according to the gray-scale difference between each pixel point of the children's area image and the neighboring pixel points within the preset neighborhood, obtain the gray-scale prominence coefficient of each pixel point; according to the position change of each pixel point in consecutive frame images, combined with the gray-scale prominence coefficient, obtain the interpolation necessity of each pixel point; according to the overall characteristics of the interpolation necessity of all pixel points in each dimension, combined with the difference characteristics of the interpolation necessity of pixel points in adjacent dimensions, obtain the dimension to be interpolated; the dimension is divided into a row dimension and a column dimension; According to the gray-scale change characteristics of the pixel points in the dimension to be interpolated and the adjacent dimensions, interpolate the current image of the children's area to obtain an enhanced image of the children's area.

[0005] Further, the method for obtaining the gray-scale prominence coefficient includes: Obtain the number of neighboring pixel points whose gray-scale values are different from the gray-scale value of each pixel point within the preset neighborhood, as the first gray-scale difference parameter of each pixel point; Taking the average value of the absolute value of the difference between the gray value of each pixel and the gray values of the neighboring pixels within a preset neighborhood as the second gray difference parameter of each pixel; Taking the product of the first gray difference parameter and the second gray difference parameter of each pixel as the gray highlighting coefficient of each pixel.

[0006] Further, the method for obtaining the interpolation necessity includes: Obtaining two adjacent historical images of the current child region image, and taking the sum value of the Euclidean distances of the coordinates of each pixel between the temporally adjacent images as the position change coefficient of each pixel; Taking the product of the position change coefficient of each pixel and the gray highlighting coefficient as the interpolation necessity of each pixel.

[0007] Further, the method for obtaining the dimension to be interpolated includes: Selecting any dimension as the target dimension; taking the average value of the interpolation necessities of all the pixels in the target dimension as the first necessity coefficient; normalizing the sum value of the differences between the interpolation necessities of the pixels in the target dimension and the interpolation necessities of the pixels with the same serial number in the adjacent dimension, and taking it as the second necessity coefficient; taking the product of the first necessity coefficient and the second necessity coefficient in the target dimension as the interpolation necessity of the target dimension; Screening and obtaining the dimension to be interpolated according to the interpolation necessity of each dimension.

[0008] Further, the method for screening and obtaining the dimension to be interpolated according to the interpolation necessity of each dimension includes: Arranging the interpolation necessities of all the row dimensions from large to small, and selecting the first preset proportion of the row dimensions as the dimensions to be interpolated; Arranging the interpolation necessities of all the column dimensions from large to small, and selecting the first preset proportion of the column dimensions as the dimensions to be interpolated.

[0009] Further, the method for obtaining the enhanced image of the child region includes: Inserting an interpolation dimension between each dimension to be interpolated and the adjacent dimension; selecting any pixel in any interpolation dimension as the target pixel; Within the preset interpolation neighborhood of the target pixel, taking the average value of the absolute value of the gray value difference between the pixels in the adjacent dimension of the same type as the target pixel's interpolation dimension as the numerator; taking the maximum value of the absolute value of the gray value difference between the pixels in the adjacent dimension of the same type as the target pixel's interpolation dimension as the denominator, and taking the fractional ratio as the gray change coefficient; Take the product of the absolute value of the gray value difference between the pixels with known gray values on both sides of the target pixel and the gray change coefficient as the subtrahend, take the maximum gray value of the pixels with known gray values on both sides of the target pixel as the minuend, and the difference as the gray value of the target pixel; Obtain the gray values of all pixels in all the interpolation dimensions to obtain the enhanced image of the children's area.

[0010] Further, after obtaining the enhanced image of the children's area, it further includes: Perform edge detection on the enhanced image of the children's area, extract the key points of the human body using the key point detection algorithm, obtain the feature vector of each key point using the SIFT feature descriptor algorithm, and input the extracted feature vector into the pre-trained classifier to identify whether the actions of the children are incorrect.

[0011] Further, the preset neighborhood is an eight-neighborhood centered on the pixel.

[0012] Further, the preset ratio is 50%.

[0013] The present invention also proposes a social intervention training system for autistic children. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of any one of the social intervention training methods for autistic children.

[0014] The present invention has the following beneficial effects: The present invention first obtains the image of the children's area in the test site in real time, provides an analysis basis for the follow-up, and reduces the interference of irrelevant background areas; further, in the current image of the children's area, according to the gray value difference between each pixel of the children's area image and the neighborhood pixels within the preset neighborhood, combined with the position change of each pixel in the consecutive frame images, the necessity of pixel interpolation is evaluated from two perspectives of gray value change and spatial position change, preparing for determining the appropriate interpolation position and enhancing the details of the edges, contours, and moving areas in the image; further facilitating interpolation by row or column, according to the overall characteristics of the interpolation necessity of all pixels in each dimension, combined with the differential characteristics of the interpolation necessity of adjacent dimension pixels, obtaining the dimension to be interpolated and accurately positioning the interpolation area; finally, according to the gray value change characteristics of the pixels in the dimension to be interpolated and the adjacent dimensions, interpolating the current image of the children's area to obtain the enhanced image of the children's area, improving the quality of the key areas in the image, and providing more reliable data support for the intervention training. The present invention accurately locates the area that needs to be interpolated in the children's area image, enhances the details of the key parts of the children's area image, ensures the accuracy of auxiliary recognition, and improves the effect of the intervention training for autistic children. Description of the Drawings

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of a social intervention training method for autistic children provided by an embodiment of the present invention; Figure 2 It is a flowchart of a method for obtaining a child area enhanced image provided by an embodiment of the present invention; Figure 3 It is an interpolation schematic diagram of the column dimension provided by an embodiment of the present invention; Figure 4 It is an interpolation schematic diagram of the row dimension provided by an embodiment of the present invention. Detailed implementation manners

[0017] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features and effects of a social intervention training method and system for autistic children proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0019] The following specifically describes the specific solutions of a social intervention training method and system for autistic children provided by the present invention with reference to the accompanying drawings.

[0020] Please refer to Figure 1 , which shows a flowchart of a social intervention training method for autistic children provided by an embodiment of the present invention, specifically including: Step S1: Real-time obtain the child area image in the test site.

[0021] The present invention mainly relates to social intervention training for autistic children. Before intervening and training autistic children, it is necessary to first observe the current problems and difficulties of autistic children and make detailed records. Then, based on the observation records, the theme of the social story is determined, and the social story is presented in a certain way to observe and intervene in the social behavior of children.

[0022] The implementer can choose ways such as text, illustrations, PPT, or video to present the story. After presentation, the demonstrator, such as a teacher or parent, demonstrates the correct behavior; an image acquisition device is installed in the test site. After the demonstrator signals an action, the behavior image of the demonstrator is collected by a camera or other shooting device; the image in the test site is collected in real time by the camera, and the adaptive histogram equalization algorithm is used to preprocess and denoise the image. The facial image information of the autistic child is input into the neural network model to identify the child area in the collected image and perform person recognition to obtain the child area image, providing an analysis basis for subsequent steps and reducing the interference of irrelevant background areas.

[0023] It should be noted that the social story should be short, easy to understand, and should be narrated in the first person. At the same time, the story should describe the expected behavior performance. The image acquisition device can be planned and installed by the implementer to ensure that the complete body posture of each child can be collected. It can be set to track and shoot a single child with a single lens, or multiple lenses can be set to shoot from multiple angles. Images in which the child and the demonstrator have the same orientation, such as both front-facing or both back-facing images, are selected from the collected images.

[0024] It should be noted that the image acquisition frequency can be set by the implementer himself / herself. As an example, it is set to 30Hz. To improve the calculation efficiency, the last image of every three images is analyzed, and the other two are used to analyze the position change of pixel points. For example, among the images numbered 1, 2, 3, 4, 5, 6, 7, 8, 9, images 3, 6, and 9 are analyzed; the implementer can write by himself / herself or select existing social stories; using a neural network for person recognition and locating the child area in the image to obtain the child area image is already prior art and will not be elaborated here.

[0025] Step S2: In the current child area image, according to the gray-scale difference between each pixel point of the child area image and the neighborhood pixel points within the preset neighborhood, obtain the gray-scale prominence coefficient of each pixel point; according to the position change of each pixel point in consecutive frame images, combined with the gray-scale prominence coefficient, obtain the interpolation necessity of each pixel point; according to the overall characteristics of the interpolation necessity of all pixel points in each dimension, combined with the difference characteristics of the interpolation necessity of pixel points in adjacent dimensions, obtain the dimension to be interpolated; the dimensions are divided into row dimension and column dimension.

[0026] During the intervention training process, to avoid the camera causing stress to children or children changing their natural behaviors due to being aware of the camera's presence, becoming constrained, and affecting the training effect, the image acquisition device is usually installed at a relatively far position from the children. At the same time, the children have a large activity range, and to avoid the loss or incompleteness of the target caused by the children's activities, the focal length is limited, resulting in the target child being relatively small in the captured image, the target features being unclear, and the details being incomplete. Therefore, it is necessary to enhance the details of the children's area images to ensure the accuracy of auxiliary recognition and improve the effect of the intervention training for children with autism.

[0027] Considering that for a certain pose assumed by a child, it is generally judged based on parts such as the arms and legs. The gray-level differences between the pixel points on the edges of these parts and the surrounding pixel points are relatively large. Interpolation near these pixel points can enhance the continuity of these edge areas, making it easier for the feature extraction algorithm to accurately identify the key parts and facilitating the analysis of pose details. Therefore, first, in the current children's area image, according to the gray-level differences between each pixel point of the children's area image and the neighborhood pixel points within the preset neighborhood, the gray-level prominence coefficient of each pixel point is obtained to prepare for determining the appropriate interpolation position.

[0028] Preferably, in an embodiment of the present invention, considering that the more the number of neighborhood pixel points with differences from the pixel point within the neighborhood, the greater the average value of the absolute value of the gray-level difference, it indicates that the gray-level difference between the pixel point and its corresponding neighborhood pixel points is greater. Based on this, the number of neighborhood pixel points with gray-level differences from the pixel point within the preset neighborhood is obtained as the first gray-level difference parameter of each pixel point; The average value of the absolute value of the difference between the gray-level values of each pixel point and the neighborhood pixel points within the preset neighborhood is used as the second gray-level difference parameter of each pixel point; The product of the first gray-level difference parameter and the second gray-level difference parameter of each pixel point is used as the gray-level prominence coefficient of each pixel point.

[0029] As an example, the preset neighborhood of the pixel point is the eight-neighborhood centered on the pixel point. The number of neighborhood pixel points with gray-level differences from the central pixel point within the eight-neighborhood and the average value of the absolute value of the difference between the gray-level values of the central pixel point and the neighborhood pixel points together represent the gray-level difference between the pixel point and the neighborhood pixel points within the preset neighborhood.

[0030] It should be noted that the implementer can also adopt other preset neighborhoods such as the four-neighborhood. Both the eight-neighborhood and the four-neighborhood are well-known technical means in the art and will not be elaborated further.

[0031] Considering that the position change of pixels in consecutive frame images reflects the posture change characteristics of children, since children with autism may show inattentiveness when engaged in an activity (refer to "The Neural Mechanisms of Motor Development Disorders in Children with Autism" by Wang Lin), resulting in erratic movements that are difficult to fix, and when the arms or legs of children are used relatively quickly between consecutive frames, single-frame images may be unclear due to motion blur. Therefore, further based on the position change of each pixel in consecutive frame images and combined with the gray-scale highlighting coefficient, the interpolation necessity of each pixel is obtained to characterize the necessary degree of interpolation around the pixel, facilitating the precise positioning of the key area that needs interpolation enhancement.

[0032] Preferably, in an embodiment of the present invention, considering that analyzing the coordinate change of pixels by establishing a coordinate system can intuitively reflect the position change of pixels, and the Euclidean distance can measure the distance length between coordinates, so the Euclidean distance of the coordinates of pixels in adjacent frame images is analyzed in the coordinate system to represent the position change of pixels in consecutive frame images; Based on this, two adjacent historical images of the current children's area image are obtained, and the sum value of the Euclidean distances of the coordinates of each pixel between temporally adjacent images is used as the position change coefficient of each pixel; that is, the Euclidean distance between the coordinates of the pixel in the first historical image and the coordinates of the second historical image, and the Euclidean distance between the coordinates of the second historical image and the coordinates of the current children's area image are obtained, and the sum value of these two Euclidean distances is used as the position change coefficient of this pixel; The product of the position change coefficient of each pixel and the gray-scale highlighting coefficient is used as the interpolation necessity of each pixel.

[0033] It should be noted that in consecutive multi-frame images, locating the same pixel can be achieved by methods such as optical flow method and corner detection. Establishing a two-dimensional rectangular coordinate system at any corner of the image to obtain the coordinates of the pixel and thus calculate the Euclidean distance are all existing technologies and will not be elaborated here.

[0034] It should be noted that the interpolation necessity is a specific value.

[0035] When interpolating an image, it is usually done row by row or column by column. Therefore, the dimension to be interpolated should be determined according to the overall characteristics of the interpolation necessity of all pixel points in each dimension. Also, considering that the interpolation necessity incorporates the position change of pixel points and the local highlighting feature of gray scale, the difference feature of the interpolation necessity of pixel points in adjacent dimensions reflects the difference between pixel points in adjacent dimensions and the local changes in the image, especially the changes in edges, textures, or motion regions. Therefore, the difference feature of the interpolation necessity of pixel points in adjacent dimensions is also combined to obtain the dimension to be interpolated, accurately locate the interpolation region, facilitate improving the quality of key regions in the image, ensure the accuracy of auxiliary recognition, and enhance the effect of intervention training for autistic children.

[0036] In a preferred embodiment of the present invention, any dimension is selected as the target dimension for individual analysis. Considering that the larger the average value of the interpolation necessity of all pixel points included in a dimension, the greater the overall demand of pixel points in the dimension for interpolation. Therefore, the average value of the interpolation necessity of all pixel points in the target dimension is used as the first necessity coefficient. Considering that the greater the interpolation necessity of pixel points in the target dimension compared to the interpolation necessity of pixel points with the same serial number in the adjacent dimension, it indicates that the target dimension requires more interpolation compared to the adjacent dimension. Therefore, the sum value of the differences between the interpolation necessity of pixel points in the target dimension and the interpolation necessity of pixel points with the same serial number in the adjacent dimension is normalized and used as the second necessity coefficient. The product of the first necessity coefficient and the second necessity coefficient of the target dimension is used as the interpolation necessity of the target dimension. The dimension to be interpolated is screened and obtained according to the interpolation necessity of each dimension.

[0037] As an example, the target dimension is only compared with the next adjacent dimension, and the last dimension is compared with the only adjacent dimension. For example, if the target dimension is the row dimension with a dimension serial number of 5, the 6th dimension in the row dimension is compared with the target dimension. The calculation formula for the interpolation necessity includes: ; where i represents the serial number of the child area image; k represents the serial number of the child area; x represents the serial number of the row dimension; represents the interpolation necessity of the x-th row dimension of the k-th child area in the i-th child area image; represents the average value of the interpolation necessity of pixel points in the x-th row dimension of the k-th child area in the i-th child area image, which is also the first necessity coefficient; v represents the serial number of the pixel point; V represents the number of pixel points; represents the interpolation necessity of the v-th pixel point in the x-th row dimension of the k-th child area in the i-th child area image; Indicates the interpolation necessity of the (v + 1)-th pixel in the (x + 1)-th row dimension of the k-th child region in the i-th child region image; Indicates linear normalization; Indicates the second necessary coefficient of the x-th row dimension of the k-th child region in the i-th child region image.

[0038] In the calculation formula of interpolation necessity, the overall characteristics of the interpolation necessity of all pixel points in the dimension are represented in the form of an average value; through In this way, the difference characteristics of the interpolation necessity of adjacent dimension pixel points are represented by the difference between the interpolation necessity of the pixel points in the target dimension and the pixel points with the same serial number in the adjacent dimension. Then, the value range is adjusted through normalization processing, and is supplemented to obtain the interpolation necessity of the target dimension, providing a basis for accurately determining the dimension to be interpolated.

[0039] It should be noted that there may be multiple children in an image, and multiple child regions are recognized, so the child regions are numbered; for the located child regions being relatively detailed, that is, the edge contour of the child region is the contour of the child's body, in the case where the edge of the child region is irregular, such as there are 100 pixel points in the 5th row, 105 pixel points in the 6th row, and 113 pixel points in the 7th row, the implementer can set the pixel points in different dimensions with the same comparison column coordinates. For example, if the first pixel point in the 5th row is in the 20th column, it is compared with the pixel point in the 20th column of the 6th row. When there is no pixel point in the 6th row in the column where a certain pixel point in the 5th row is located, the interpolation necessity is set to 0; the implementer can also set to extract a square child region image containing part of the background, so that the number of columns in each row is the same, and the number of rows in each column is the same.

[0040] It should be noted that the implementer can also weight and sum the mean, mode, and median of the interpolation necessity of all pixel points in the target dimension with weighted weights of 0.5, 0.3, and 0.2 as the first necessary coefficient to represent the overall characteristics of the interpolation necessity of all pixel points in the dimension.

[0041] Preferably, in an embodiment of the present invention, considering that the greater the interpolation necessity of each dimension, the more interpolation enhancement is required, so the interpolation necessities of all row dimensions are arranged from large to small, and the first preset proportion of row dimensions is selected as the dimension to be interpolated; The interpolation necessities of all column dimensions are arranged from large to small, and the first preset proportion of column dimensions is selected as the dimension to be interpolated.

[0042] As an example, the preset proportion is 50%, that is, in all row dimensions, the row dimensions with the top 50% of the interpolation necessity are selected as the dimensions to be interpolated; in all column dimensions, the column dimensions with the top 50% of the interpolation necessity are selected as the dimensions to be interpolated.

[0043] It should be noted that the dimensions are divided into row dimensions and column dimensions. Adjacent dimensions refer to dimensions with adjacent serial numbers of the same type. For example, row dimensions are adjacent to row dimensions, and column dimensions are adjacent to column dimensions; the preset ratio can be adjusted according to actual applications.

[0044] Step S3: Interpolate the current child region image according to the gray-scale change characteristics of the pixel points of the dimension to be interpolated and the adjacent dimensions, and obtain the enhanced child region image.

[0045] By screening to obtain the dimension to be interpolated and determining the specific row or column that needs to be interpolated, the image can be interpolated and enhanced. Considering that the purpose of interpolation is to enhance the details and clarity of the key regions in the image while maintaining the naturalness and continuity of the image, the current child region image is interpolated according to the gray-scale change characteristics of the pixel points of the dimension to be interpolated and the adjacent dimensions, and the enhanced child region image is obtained, which better retains the natural transition of the image, improves the image details and clarity, and provides higher-quality data support for intervention training.

[0046] Preferably, in an embodiment of the present invention, the method for obtaining the enhanced child region image includes: Please refer to Figure 2 , which shows a flowchart of a method for obtaining an enhanced child region image provided by an embodiment of the present invention, and specifically includes: Step S301: Insert an interpolation dimension between each dimension to be interpolated and the adjacent dimension; select any pixel point of any interpolation dimension as the target pixel point.

[0047] Please refer to Figure 3 , which shows an interpolation schematic diagram of a column dimension provided by an embodiment of the present invention; Please refer to Figure 4 , which shows an interpolation schematic diagram of a row dimension provided by an embodiment of the present invention.

[0048] Figure 3 In, the pixel points marked with numbers 1 and 3 are in one column and are the dimensions to be interpolated, the pixel points marked with numbers 2 and 4 are in one column, and the pixel point marked with letter a is the target pixel point, and the column where it is located is a column interpolation dimension; Figure 4 In, the pixel points marked with numbers 1 and 3 are in one row and are the dimensions to be interpolated, the pixel points marked with numbers 2 and 4 are in one row, and the pixel point marked with letter a is the target pixel point, and the row where it is located is a row interpolation dimension.

[0049] It should be noted that whether to perform column interpolation or row interpolation first can be set by the implementer himself. When the dimension to be interpolated is at the edge of the image, interpolation is performed towards the inner side of the image. For example, if the original image has 200 columns and the 200th column is the dimension to be interpolated, the corresponding interpolation dimension is inserted between the 199th column and the 200th column. In an embodiment of the present invention, it is selected to perform column interpolation first and then row interpolation.

[0050] Step S302: Within the preset interpolation neighborhood of the target pixel point, take the average value of the absolute values of the gray value differences between the pixel points of the adjacent dimensions of the same type as the interpolation dimension of the target pixel point as the numerator; take the maximum value of the absolute values of the gray value differences between the pixel points of the adjacent dimensions of the same type as the interpolation dimension of the target pixel point as the denominator, and the fractional ratio as the gray change coefficient.

[0051] When the target pixel point is for column interpolation, please refer to Figure 3 , the preset interpolation neighborhood is the pixel points marked 1, 2, 3, and 4, that is, the pixel points to the left, right, lower left, and lower right of the target pixel point; when the target pixel point is for row interpolation, please refer to Figure 4 , the preset interpolation neighborhood is the pixel points marked 1, 2, 3, and 4, that is, the pixel points above, below, upper right, and lower right of the target pixel point.

[0052] Considering that when the target pixel point is inserted between the pixel points of two dimensions, it is necessary to maintain the naturalness and continuity of the image. Therefore, within the preset interpolation neighborhood, it is necessary to analyze the gray value change between the pixel points of the adjacent dimensions of the same type as the interpolation dimension of the target pixel point, so as to ensure that the interpolated image is more smooth; Based on this, the calculation formula of the gray change coefficient includes: ; Among them, represents the gray change coefficient of the a-th target pixel point; represents the absolute value of the gray value difference between the first pair of pixel points of the adjacent dimensions of the same type as the interpolation dimension of the target pixel point within the preset interpolation neighborhood of the a-th target pixel point, , represents the gray value of the pixel point marked 1 corresponding to the a-th target pixel point, represents the gray value of the pixel point marked 2 corresponding to the a-th target pixel point; represents the absolute value of the gray value difference between the second pair of pixel points of the adjacent dimensions of the same type as the interpolation dimension of the target pixel point within the preset interpolation neighborhood of the a-th target pixel point, , represents the gray value of the pixel point marked 3 corresponding to the a-th target pixel point, Denote the gray value of the pixel labeled 4 corresponding to the a-th target pixel; Denote taking the maximum value.

[0053] In the calculation formula of the gray change coefficient, the average value of the absolute values of the gray value differences between the pixels of the adjacent dimensions of the same type as the interpolation dimension to which the target pixel belongs represents the gray change degree around the target pixel, reflecting the gray change characteristics of the pixels in the dimension to be interpolated and the adjacent dimensions; and then further normalizing by the maximum value of the absolute values of the gray value differences between the pixels of the adjacent dimensions of the same type as the interpolation dimension to which the target pixel belongs, while restricting the value range of the gray change coefficient, represents the due gray change ratio of the target pixel.

[0054] Step S303: Take the product of the absolute value of the gray value difference between the pixels with known gray values on both sides of the target pixel and the gray change coefficient as the subtrahend, take the maximum value of the gray values of the pixels with known gray values on both sides of the target pixel as the minuend, and the difference as the gray value of the target pixel.

[0055] The calculation formula of the gray value of the target pixel includes: ; where Denote the gray value of the a-th target pixel.

[0056] In the calculation formula of the gray value of the target pixel, through Denote the gray change amplitude of the target pixel. By taking the maximum value of the gray values of the pixels with known gray values on both sides of the target pixel as the minuend and subtracting In this way, it is ensured that the gray change directions of the target pixel and the pixels on both sides are the same, that is, the gradient directions are the same. At the same time, by means of Keep the gray change intensity consistent, keep the continuity of the image, and make the image appear smoother.

[0057] Step S304: Obtain the gray values of all pixels in all interpolation dimensions to obtain the enhanced image of the child area.

[0058] Further analyze each pixel in each interpolation dimension one by one to obtain the gray values of all pixels in all interpolation dimensions, so as to obtain the enhanced image of the child area, enhance the feature representation of the child's behavior posture, and provide more reliable data support for the auxiliary system.

[0059] It should be noted that in an embodiment of the present invention, after obtaining the enhanced image of the child area, it further includes: Perform edge detection on the enhanced image of the children's area, extract the key points of the human body using the key point detection algorithm, obtain the feature vector of each key point using the SIFT feature descriptor algorithm, and input the extracted feature vector into the pre-trained classifier to identify whether the children's actions are incorrect.

[0060] Specifically, perform canny edge detection on the processed enhanced image of the children's area to help extract the outline and pose information of the children. Then use the Open Pose key point detection algorithm to detect the key points (such as joint points) of the human body in the image after edge detection. The positions and relative relationships of these key points can reflect the actions and postures of the human body.

[0061] Then use the SIFT feature descriptor algorithm to calculate the feature vector for each key point: a) First, calculate the information around the key point: including pixel values, gradients, and directions.

[0062] b) Then calculate some statistical information. Divide the gradients around the key point according to the direction and calculate the gradient intensity of each direction. Then combine these gradient intensities into a histogram. Other statistics can also be calculated, such as the average brightness, contrast, and gradient magnitude around the key point.

[0063] c) Encode the calculated statistical information into a feature vector.

[0064] Finally, input the extracted feature vector into the pre-trained classifier. The classifier will reflect the specific actions of the children according to the learned mapping relationship. Compare the identified actions of the children with the actions of the teacher to identify whether the children's actions are incorrect. When it is identified that the children's actions are incorrect, send a prompt to the demonstrator or relevant personnel to assist in discovering the problems of autistic children in social stories and improve the effect of social story training for autistic children.

[0065] It should be noted that in the further processing of the enhanced image of the children's area, the technologies used are all existing technologies and will not be elaborated here.

[0066] An embodiment of the present invention also provides a social intervention training system for autistic children. The system includes a memory, a processor, and a computer program. The memory is used to store the corresponding computer program, and the processor is used to run the corresponding computer program. When the computer program runs in the processor, it can implement a social intervention training method for autistic children described in steps S1 - S3.

[0067] In summary, in view of the technical problem that the recognition accuracy is not ideal due to the influence of individual differences and image acquisition problems on the auxiliary system, which affects the training effect of social stories for autistic children, the present invention obtains the image of the children's area in the test site in real time; further, in the current children's area image, according to the gray difference between each pixel point of the children's area image and the neighborhood pixel points in the preset neighborhood, the gray highlighting coefficient of each pixel point is obtained; further, according to the position change of each pixel point in the consecutive frame images, combined with the gray highlighting coefficient, the interpolation necessity of each pixel point is obtained; further, according to the overall characteristics of the interpolation necessity of all pixel points in each dimension, combined with the difference characteristics of the interpolation necessity of the pixel points in the adjacent dimensions, the dimension to be interpolated is obtained; finally, according to the gray change characteristics of the pixel points in the dimension to be interpolated and the adjacent dimensions, the current children's area image is interpolated to obtain an enhanced image of the children's area.

[0068] It should be noted that the above sequence of embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0069] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. The key points of each embodiment are the differences from other embodiments.

Claims

1. A social intervention training method for children with autism, characterized in that, The method includes: Obtaining the image of the children's area in the test site in real time; In the current image of the children's area, according to the gray-scale difference between each pixel point of the children's area image and the neighboring pixel points in the preset neighborhood, obtaining the gray-scale prominence coefficient of each pixel point; according to the position change of each pixel point in the consecutive frame images, combining with the gray-scale prominence coefficient, obtaining the interpolation necessity of each pixel point; according to the overall characteristics of the interpolation necessity of all pixel points in each dimension, combining with the difference characteristics of the interpolation necessity of pixel points in adjacent dimensions, obtaining the dimension to be interpolated; the dimensions are divided into row dimension and column dimension; Interpolating the current image of the children's area according to the gray-scale change characteristics of the pixel points in the dimension to be interpolated and the adjacent dimensions, to obtain an enhanced image of the children's area.

2. The social intervention training method for autistic children according to claim 1, wherein The method for obtaining the gray-scale prominence coefficient includes: Obtaining the number of neighboring pixel points whose gray-scale values are different from the gray-scale value of each pixel point in the preset neighborhood, as the first gray-scale difference parameter of each pixel point; Taking the average value of the absolute value of the difference between the gray-scale value of each pixel point and the gray-scale value of the neighboring pixel points in the preset neighborhood, as the second gray-scale difference parameter of each pixel point; Taking the product of the first gray-scale difference parameter and the second gray-scale difference parameter of each pixel point, as the gray-scale prominence coefficient of each pixel point.

3. The social intervention training method for autistic children according to claim 2, wherein The method for obtaining the interpolation necessity includes: Obtaining two adjacent historical images of the current children's area image, and taking the sum value of the Euclidean distances of the coordinates of each pixel point between the temporally adjacent images, as the position change coefficient of each pixel point; Taking the product of the position change coefficient of each pixel point and the gray-scale prominence coefficient, as the interpolation necessity of each pixel point.

4. A social intervention training method for autistic children according to claim 1, characterized in that The method for obtaining the dimension to be interpolated includes: Selecting any dimension as the target dimension; taking the average value of the interpolation necessities of all pixel points in the target dimension, as the first necessity coefficient; normalizing the sum value of the differences between the interpolation necessities of the pixel points in the target dimension and the interpolation necessities of the pixel points with the same serial number in the adjacent dimension, as the second necessity coefficient; taking the product of the first necessity coefficient and the second necessity coefficient of the target dimension, as the interpolation necessity of the target dimension; Screening and obtaining the dimension to be interpolated according to the interpolation necessity of each dimension.

5. A social intervention training method for children with autism according to claim 4, characterized in that The method for screening and obtaining the dimension to be interpolated according to the interpolation necessity of each dimension includes: Arranging the interpolation necessities of all the row dimensions from large to small, and selecting the first preset proportion of the row dimensions as the dimension to be interpolated; Arranging the interpolation necessities of all the column dimensions from large to small, and selecting the first preset proportion of the column dimensions as the dimension to be interpolated.

6. A social intervention training method for children with autism according to claim 1, characterized in that, The method for obtaining the enhanced image of the children's area includes: Inserting an interpolation dimension between each dimension to be interpolated and the adjacent dimension; selecting any pixel point in any interpolation dimension as the target pixel point; Within the preset interpolation neighborhood of the target pixel point, take the average value of the absolute values of the gray value differences between the pixel points in the adjacent dimensions of the same type as the interpolation dimension of the target pixel point as the numerator; take the maximum value of the absolute values of the gray value differences between the pixel points in the adjacent dimensions of the same type as the interpolation dimension of the target pixel point as the denominator, and the fractional ratio as the gray change coefficient; Take the product of the absolute value of the gray value difference between the pixel points with known gray values on both sides of the target pixel point and the gray change coefficient as the subtrahend, take the maximum gray value of the pixel points with known gray values on both sides of the target pixel point as the minuend, and the difference as the gray value of the target pixel point; Obtain the gray values of all pixel points in all the interpolation dimensions to obtain a child region enhanced image.

7. A social intervention training method for children with autism according to claim 1, characterized in that After obtaining the child region enhanced image, it further includes: Perform edge detection on the child region enhanced image, extract the key points of the human body using a key point detection algorithm, obtain the feature vector of each key point using the SIFT feature descriptor algorithm, and input the extracted feature vector into a pre-trained classifier to identify whether the child's action is incorrect.

8. A social intervention training method for autistic children according to claim 2, characterized in that The preset neighborhood is an eight-neighborhood centered on the pixel point.

9. A social intervention training method for children with autism according to claim 5, characterized in that The preset ratio is 50%.

10. A social intervention training system for children with autism, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the social intervention training method for autistic children according to any one of claims 1 to 9.

Images