A cross-layer multi-granularity autism risk assessment method based on diffusion model

Abstract

The embodiment of the application discloses a cross-layer multi-granularity autism risk assessment method based on a diffusion model, relates to the technical field of computers, and comprises the following steps: acquiring video data of a target to be evaluated in a teaching scene, processing the video data to obtain an image sequence, inputting the image sequence into a trained diffusion model, determining a head gaze target area of each target to be evaluated in each frame of image through the trained diffusion model, determining a gaze target category based on the head gaze target area of each frame of image, and calculating an autism risk assessment parameter in a corresponding time period range. The application can determine the risk of autism of each target to be evaluated according to the autism risk assessment parameter, can objectively and accurately predict the risk degree of autism of the target, is beneficial to relevant personnel to take preventive measures in advance, can not only reduce the cost of risk assessment, but also can guarantee the accuracy and efficiency of risk assessment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a cross-layer multi-granularity autism risk assessment method based on diffusion model. BACKGROUND

[0002] Autism is a neurodevelopmental disorder mainly characterized by difficulties in social communication, repetitive behaviors and narrow interests. Early identification of the risk degree of autism and appropriate intervention can significantly improve the development trajectory of individuals.

[0003] Currently, the risk assessment of autism in children mainly relies on behavior observation and questionnaire survey. The process of risk assessment has a certain subjectivity and depends on the experience of risk assessment personnel, which may lead to differences in assessment results. In addition, functional magnetic resonance imaging, eye tracking technology and other assessment methods can be used to provide high-precision data, but this method requires high equipment, the price of the equipment is expensive, and the operation process of the equipment depends on the experience of the operator, which is difficult to be widely applied in the conventional autism risk assessment.

[0004] Therefore, there is currently a lack of a method that can accurately, low-cost and objectively assess the risk degree of autism in the target. SUMMARY

[0005] The embodiment of the present application provides a cross-layer multi-granularity autism risk assessment method based on diffusion model, which solves the defects in the related art. The technical solution is as follows:

[0006] In a first aspect, the embodiment of the present application provides a cross-layer multi-granularity autism risk assessment method based on diffusion model, comprising:

[0007] Obtain video data containing at least one target to be evaluated in a teaching scene, and process the video data to obtain an image sequence corresponding to the teaching scene, which is composed of image frames arranged according to a preset time interval;

[0008] Input the image sequence into a trained diffusion model, and determine the head gaze target area of each target to be evaluated in each image frame through the trained diffusion model;

[0009] Determine the gaze target category of the target to be evaluated at the corresponding time based on the head gaze target area of each image frame, and calculate the autism risk assessment parameter of the target to be evaluated within the time period corresponding to the image sequence based on the gaze target category of the target to be evaluated at each time.

[0010] In an optional implementation of the first aspect, the video data containing at least one to-be-evaluated target in the teaching scene is acquired, and an image sequence corresponding to the teaching scene is obtained based on the video data, the image sequence being composed of image frames arranged according to a preset time interval, including:

[0011] Two images with an overlapping area are captured at each time point by image capture devices arranged at two different positions in the teaching scene;

[0012] The two images with the overlapping area captured at each time point are divided based on the preset time interval, to obtain image sequences corresponding to the two image capture devices respectively;

[0013] Feature points are extracted from the image sequences corresponding to the two image capture devices, to obtain corresponding sets of feature points;

[0014] Feature points are matched in the images captured by the two image capture devices based on the sets of feature points by using a transformation matrix, and the images captured by the two image capture devices are spliced, to obtain the image sequence.

[0015] In an optional implementation of the first aspect, the image sequence is input into the trained diffusion model, and a head gaze target area of each to-be-evaluated target in each image frame is determined by using the trained diffusion model, including:

[0016] Each image frame in the image sequence is compressed and a noise image is added, to obtain an image sequence containing noise;

[0017] The image sequence containing noise is input into a denoising model, to obtain a denoised image, and feature extraction is performed on the denoised image, to obtain first features containing a relationship between a to-be-evaluated target and a gaze target and second features containing the teaching scene, thereby obtaining a multi-scale scene feature map containing the first features and the second features.

[0018] In an optional implementation of the first aspect, the image sequence containing noise is input into a denoising model, and feature extraction is performed on the denoised image, to obtain first features containing a relationship between a to-be-evaluated target and a gaze target and second features containing the teaching scene, including:

[0019] The noise image is convolved to obtain a feature vector, two feature map groups are obtained based on the feature vector, and image residuals are obtained by processing the two feature map groups;

[0020] And the corresponding features are connected by using the image residuals, to extract preliminary features;

[0021] The preliminary features are taken as inputs to extract the first features and the second features;

[0022] The scene features of the teaching scene are fused based on the first features and the second features.

[0023] In an optional implementation of the first aspect, the determining, by the trained diffusion model, of the head gaze target region of each of the to-be-evaluated targets in each image frame includes:

[0024] The scene features of the teaching scene are encoded to obtain scene encoded features;

[0025] The scene encoded features are spatially expanded by a decoder to obtain decoded features;

[0026] The decoded features are processed by a single-layer convolutional layer to obtain a head detection map, head features are extracted based on the head detection map, and fusion features are obtained by fusing the head features and the decoded features;

[0027] The head gaze target region is obtained based on a head heat map, a gaze heat map, and a connection map obtained by processing the fusion features by a convolutional layer.

[0028] In an optional implementation of the first aspect, the calculating of the autism risk assessment parameter of the to-be-evaluated target in a time period corresponding to the image sequence based on the gaze target category of each of the to-be-evaluated targets at each time point includes:

[0029] The gaze target category corresponding to each image frame in the image sequence is obtained;

[0030] The proportion of the number of image frames corresponding to each gaze target category in the image sequence is determined, the weight of each gaze target category is determined based on the proportion corresponding to the gaze target category, and the autism risk assessment parameter of the to-be-evaluated target in the time period corresponding to the image sequence is obtained by weighted calculation.

[0031] In a second aspect, the embodiments of the present application further provide a cross-layer multi-granularity autism risk assessment device based on a diffusion model, including:

[0032] An image acquisition module is configured to acquire video data containing at least one to-be-evaluated target in a teaching scene, and process the video data to obtain an image sequence composed of image frames arranged according to a preset time interval corresponding to the teaching scene;

[0033] a region extraction module configured to input the image sequence into the trained diffusion model, and determine, by the trained diffusion model, a head gaze target region of each of the to-be-evaluated targets in each frame of image;

[0034] a risk assessment module configured to determine, based on the head gaze target region of each frame of image, a gaze target category of the to-be-evaluated target at a corresponding time, and calculate a self-injury risk assessment parameter of the to-be-evaluated target in a time period corresponding to the image sequence based on the gaze target category of the to-be-evaluated target at each time.

[0035] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method provided in the first aspect or any of the implementation manners of the first aspect when executing the program.

[0036] In a fourth aspect, a non-transitory computer-readable storage medium is provided, which stores a computer program, and the computer program is executable on a processor to implement the method provided in the first aspect or any of the implementation manners of the first aspect.

[0037] The technical scheme provided by some embodiments of the present application has at least the following beneficial effects:

[0038] The method for cross-layer multi-granularity self-injury risk assessment based on a diffusion model provided by the embodiments of the present application can collect video data of to-be-evaluated targets in a teaching scene, input the video data into a trained diffusion model after binocular stitching and fixed time length division, and determine the head and gaze object of each to-be-evaluated target in the form of connected heat maps, so as to more accurately determine the gaze region of a child, and then calculate a self-injury risk assessment parameter according to the change of the gaze target of each to-be-evaluated target, and further determine the risk of self-injury of each to-be-evaluated target, which can objectively and accurately predict the risk degree of self-injury of the target, and is beneficial to relevant personnel to take preventive measures in advance, which can not only reduce the cost of risk assessment, but also ensure the accuracy and efficiency of risk assessment. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical schemes in the present application or related art, the following will briefly introduce the drawings needed to be used in the embodiments or related art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0040] Figure 1is a flowchart of a cross-layer multi-granularity autism risk assessment method based on a diffusion model provided by an embodiment of the present application.

[0041] Figure 2 is a flowchart of a cross-layer multi-granularity autism risk assessment method based on a diffusion model provided by an embodiment of the present application.

[0042] Figure 3 is a structural diagram of a cross-layer multi-granularity autism risk assessment device based on a diffusion model provided by an embodiment of the present application.

[0043] Figure 4 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0045] The terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above-described drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but can optionally include steps or modules not listed, or can optionally include other steps or modules inherent to the process, method, product or device.

[0046] It should be noted that the terms "first" and "second" involved in the present application are only to distinguish similar objects, and do not represent a specific order of the objects. Understandably, "first" and "second" can be interchanged in a specific order or sequence as allowed. It should be understood that the objects distinguished by "first" and "second" can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those described or illustrated herein.

[0047] It should be noted that with the application of deep learning models in the field of autism prevention, the prevention methods of autism have changed a lot. Considering the characteristics of the target to be evaluated, such as children who are active in behavior, it is difficult for conventional detection methods to cooperate, therefore, by improving the cooperation degree of children and improving the detection efficiency, the attention target detection integrated into the visual field can effectively identify the characteristics of children in social interaction, language communication and behavior patterns, and achieve the purpose of risk assessment.

[0048] It can be understood that the autistic children have significantly lower gaze time and frequency in the face of social stimuli (such as human face, eye), and relatively higher attention concentration in the face of non-social stimuli (such as object, pattern). In this way, the unique pattern of children in attention allocation can be judged, and the risk assessment parameter can be given, which can provide the basis for early intervention of autism. In this way, not only can the risk assessment of autism be more convenient, but also the accuracy of the risk assessment can be guaranteed, so that parents and teachers can take the next step according to the results of the risk assessment.

[0049] The application will be described in detail below with specific examples.

[0050] Next, the application will be described in detail below with specific examples. Figure 1 , introduce the cross-layer multi-granularity autism risk assessment method based on the diffusion model provided by the application. For details, please see Figure 1 , Figure 1 Fig. 1 shows a flowchart of a cross-layer multi-granularity autism risk assessment method based on a diffusion model provided by an embodiment of the application. As shown in Figure 1 , the method comprises the following steps:

[0051] S101, acquiring video data containing at least one target to be evaluated in a teaching scene, and processing the video data to obtain an image sequence corresponding to the teaching scene, which is composed of image frames arranged according to a preset time interval;

[0052] S102, inputting the image sequence into a trained diffusion model, and determining the head gaze target area of each target to be evaluated in each image frame through the trained diffusion model;

[0053] S103, determining the gaze target category of the target to be evaluated at the corresponding time based on the head gaze target area of each image frame, and calculating the autism risk assessment parameter of the target to be evaluated within the time period corresponding to the image sequence based on the gaze target category of the target to be evaluated at each time.

[0054] In some embodiments, S101 can include:

[0055] Specifically, two image acquisition devices can be respectively arranged in different areas in the teaching scene, for example, in a classroom. The images of the two image acquisition devices have an overlapping area. For example, an RGB camera and an infrared camera can be arranged on the left and right sides of the teaching scene respectively to acquire multi-angle video data V RGB and V RED , and the acquisition duration can be in units of class hours.

[0056] Specifically, V RGB and V RED can be acquired for a fixed time of 3 seconds.RGB and V RED are divided respectively to obtain image sequences SEQ RGB ={IMG RGB1 ,IMG RGB2 ,…,IMG RGBN} and SEQ RED ={IMG RED1 ,IMG RED2 ,…,IMG REDN}.

[0057] Specifically, the splicing can be performed in a double-targeting manner. First, the SIFT algorithm is used to extract the feature point sets of each image in the image sequences SEQ RGB and SEQ RED respectively, and the feature point sets are stored in the sets SET RGB {Feat RGB1 ,Feat RGB2 ,…Feat RGBN ,} and SET RED {Feat RED1 ,Feat RED2 ,…Feat REDN ,}.

[0058] Further, the KD tree and the BFF algorithm can be used to perform feature matching and searching on the feature points of each image in the feature point sets SET RGB and SET RED , and a preliminary screening is performed according to the ratio of the nearest neighbor distance and the second nearest neighbor distance. The RANSAC algorithm is used to calculate a transformation matrix H, the feature points Feat RED of the image IMG RED are transformed into matching points by the transformation matrix, that is, the feature points in IMG RED are transformed into corresponding points in IMG RGB . Then, the image IMG RED is transformed into a new image by the transformation matrix H, and the image IMG RGB is spliced to the left side of the new image. Finally, the scene image sequence SEQ SCE ={IMG SCE1 ,IMG SCE2 ,…,IMG SCEN} after splicing of the images collected by the two image collection devices is obtained.

[0059] In some embodiments, S102 includes the following steps, as shown in Figure 2 .

[0060] Specifically, each frame of image in the image sequence can be compressed respectively, and a noise image is added to obtain an image sequence containing noise.

[0061] The image sequence containing noise is input into a denoising model to obtain a denoised image, and feature extraction is performed on the denoised image to obtain a first feature containing the relationship between the target to be evaluated and the fixation target and a second feature containing the teaching scene, thereby obtaining a multi-scale scene feature map containing the first feature and the second feature.

[0062] Specifically, the scene graph sequence SEQ SCE can be compressed and a noise image can be added to obtain a latent noise image sequence SEQ NOI ={IMG NOI1 ,IMG NOI2 ,…,IMG NOIN}.

[0063] Further, the latent noise IMG NOI is input into a denoising model (SU-Net).

[0064] After the input image IMG NOI is converted into a feature tensor X through 3x3 convolution, 1x1 convolution, segmentation and concatenation operations are performed, and 1x1 convolution is used to effectively and efficiently promote information fusion between SwinT blocks and RConv blocks, thereby extracting fusion features F IN ={F1,F2,F3,F4}.

[0065] The feature map is represented as J, K and L, and the attention calculation is as follows:

[0066]

[0067] where γ is a learnable scalar and B represents a bias.

[0068] After applying the Softmax function, the self-attention calculation can be represented as:

[0069] tten(J,K,L)=Softmax(M+Sim(J,K))*L;

[0070] where M represents a mask matrix.

[0071] Specifically, the denoising process using the denoising model consists of a down-sampling process and an up-sampling process composed of a cross-step convolution and a transpose convolution.

[0072] The fusion features F INAs input, it contains four scale features, each with a residual connection between downsampling based on 2×2 strided convolution and upsampling based on 2×2 transposed convolution. Additionally, four SC blocks are taken between downsampling and upsampling to enhance local and non-local modeling capabilities. Finally, after passing through four SC blocks, a multi-scale scene feature map F containing the second and first features is obtained. OT ={F1,F2,F3,F4}, where the first feature includes the relationship between the target to be evaluated and the object being gazed at, and the second feature includes scene information, etc.

[0073] Furthermore, the multi-scale scene feature map F output by SU-Net is... OT Feature encoding is performed to obtain scene-encoded features F. en F is gradually decoded en Spatial dimension expansion yields decoding feature F de Among them, the decoding feature F de One part is used for the head target prediction branch, in which F de Head detection map H is generated through convolution. d This is used to strengthen the head prior, and then head features F are extracted from the head detection map. head With F de The fusion feature F is obtained by concatenation. prop .

[0074] Furthermore, the fusion feature F prop As input, a head heatmap H, a gaze heatmap G, and a connectivity graph C are generated by a prediction head with convolutional layers, and finally the head gaze target region is obtained.

[0075] In some embodiments, the training process for determining the head orientation region using a diffusion model may include the following steps:

[0076] The total training loss is calculated based on the matching pairs between the predicted head gaze target region and the actual head gaze target region, and is expressed as follows:

[0077] L total =γ H *L H +γ G *L G +γ C *L C +γ d *L d ;

[0078] Where L H L G and L C These represent the losses from the head heatmap, gaze heatmap, and connectivity map, respectively. dis the loss of head detection map, and γ is the weight of each loss term.

[0079] where, for the head heat map loss, the predicted head heat map {P1, P2, …, P N} and the ground truth The head loss is represented as:

[0080]

[0081] where N is the number of actual head gaze target regions in the input image, w and h are the heat map width and height, P n (i,j) is the pixel value of the nth head heat map at position (i,j), L G and the MSE loss of L C is the same as L H .

[0082] Specifically, for the head detection map L d uses the MSE loss. Since there is only one head detection map for each scene image, the loss function becomes:

[0083]

[0084] In some embodiments, in S103, the gaze target class of the to-be-evaluated target at the corresponding time can be determined based on the head gaze target region of each frame of image, and the autism risk evaluation parameter of the to-be-evaluated target in the time period corresponding to the image sequence is calculated based on the gaze target class of the to-be-evaluated target at each time.

[0085] Specifically, the gaze target class corresponding to each image frame in the image sequence can be obtained;

[0086] The proportion of the number of image frames corresponding to each gaze target class in the image sequence is determined, the weight of each gaze target class is determined based on the proportion corresponding to each gaze target class, and the autism risk evaluation parameter of the to-be-evaluated target in the time period corresponding to the image sequence is calculated by weighting.

[0087] In some embodiments, the gaze target of each to-be-evaluated target can be classified to determine the gaze target class, and different weights are given to each gaze target class for calculation, and then the given risk evaluation threshold is compared to predict the risk degree of the to-be-evaluated target having autism.

[0088] Specifically, the gaze object of the target can be determined according to the region of the target gaze to be evaluated, the proportion of different gaze regions obtained from the entire picture sequence is multiplied by the weight to obtain the corresponding autism risk assessment parameter, and the higher the score, the greater the risk of autism.

[0089] For example, the autism risk assessment parameter threshold can be selected as 0.5, if the autism risk assessment parameter is less than 0.5, the evaluation result that the target to be evaluated has a lower risk of autism can be output, otherwise the child is identified as normal, otherwise the evaluation result that the child has a higher risk of autism can be output.

[0090] Exemplarily, the gaze target categories can include gaze at teacher (G tea ), gaze at playmate (G FRI ), gaze down (G low ) and gaze at other things (G oth ), and different categories are given different weights {λ1=0.1, λ1=0.2, λ1=0.4, λ1=0.3}, for example: it can be considered that the child gazes at the teacher and the playmate at the current moment, which represents an active and healthy psychological state, and the child gazes down or at other things at the current moment, which needs to be further evaluated.

[0091] The calculation of the autism risk assessment parameter can apply the formula:

[0092] Score=λ1*P tea +λ2*P FRI +λ3*P low +λ4*P oth ;

[0093] Wherein λ i represents the weight of different categories, and P represents the proportion of different categories.

[0094] In some embodiments, in S103, the electroencephalogram data corresponding to the time when each video frame in the image sequence is also determined, the autism risk assessment parameter is calculated based on the frequency of the electroencephalogram signal and the gaze target category identified in S102, and the present application is not limited thereto.

[0095] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the method embodiment of the present application.

[0096] Next, please refer to Figure 3A structural diagram of a diffusion model-based cross-layer multi-granularity autism risk assessment device is provided for an exemplary embodiment of the present application. The device can be realized as all or part of a terminal by software, hardware or a combination of both, and can also be integrated as an independent module on a server. The diffusion model-based cross-layer multi-granularity autism risk assessment device in the embodiments of the present application can be applied to a terminal or cloud, and the device 30 includes an image acquisition module 301, a region extraction module 302 and a risk assessment module 303, wherein:

[0097] The image acquisition module 301 is configured to obtain video data containing at least one target to be evaluated in a teaching scene, and process the video data to obtain an image sequence composed of image frames arranged according to a preset time interval corresponding to the teaching scene;

[0098] The region extraction module 302 is configured to input the image sequence into a trained diffusion model, and determine a head gaze target region of each target to be evaluated in each image frame through the trained diffusion model;

[0099] The risk assessment module 303 is configured to determine a gaze target category of the target to be evaluated at a corresponding time based on the head gaze target region of each image frame, and calculate an autism risk assessment parameter of the target to be evaluated within a time period corresponding to the image sequence based on the gaze target category of the target to be evaluated at each time.

[0100] It should be noted that the device 30 provided in the above embodiments, when performing the diffusion model-based cross-layer multi-granularity autism risk assessment method, is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, i.e., the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiments and the diffusion model-based cross-layer multi-granularity autism risk assessment method embodiments belong to the same concept, and the implementation process is detailed in the method embodiments. Therefore, it is not repeated here.

[0101] The embodiments of the present application also provide an electronic device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method of any of the above embodiments when executing the program.

[0102] Please refer to Figure 4 A structural block diagram of an electronic device is provided for an embodiment of the present application.

[0103] As Figure 4 shown, the electronic device 400 includes a processor 401 and a memory 402.

[0104] In the embodiments of the present application, the processor 401 is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. The processor 401 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array).

[0105] The processor 401 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state.

[0106] The memory 402 can include one or more computer-readable storage media, which can be non-transitory. The memory 402 can also include a high-speed random access memory and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction for being executed by the processor 401 to implement the method in the embodiments of the present application.

[0107] In some embodiments, the electronic device 400 further includes a peripheral device interface 403 and at least one peripheral device 404. The processor 401, the memory 402, and the peripheral device interface 403 can be connected through a bus or a signal line. Each peripheral device 404 can be connected to the peripheral device interface 403 through a bus, a signal line, or a circuit board. Specifically, the peripheral device interface 403 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 401 and the memory 402.

[0108] In some embodiments of the present application, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 401, the memory 402, and the peripheral device interface 403 can be implemented on a separate chip or circuit board. The embodiments of the present application do not make specific limitations in this regard.

[0109] The electronic device structure block diagram shown in the embodiments of the present application does not constitute a limitation on the electronic device 400, and the electronic device 400 can include more or fewer components than shown, or combine certain components, or use a different arrangement of components.

[0110] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the method of any of the preceding embodiments. The computer readable storage medium can include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nano system (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0111] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus necessary universal hardware platforms, and of course can also be implemented by hardware. Based on such an understanding, the above technical solutions, essentially or in other words, the part that contributes to the related art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some part of the embodiments.

[0112] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A diffusion model based cross-layer multi-granularity autism risk assessment method, characterized in that, The method comprises the following steps: acquiring video data containing at least one target to be evaluated in a teaching scene, and processing the video data to obtain an image sequence corresponding to the teaching scene, which is composed of image frames arranged according to a preset time interval; inputting the image sequence into a trained diffusion model, and determining, by the trained diffusion model, a head gaze target area of each target to be evaluated in each frame of image; determining a gaze target category of the target to be evaluated at a corresponding time based on the head gaze target area of each frame of image, and calculating an autism risk evaluation parameter of the target to be evaluated within a time period corresponding to the image sequence based on the gaze target category of the target to be evaluated at each time, comprising: acquiring a gaze target category corresponding to each frame of image in the image sequence; determining a proportion of the number of image frames corresponding to each gaze target category in the image sequence, determining a weight of each gaze target category based on the proportion corresponding to each gaze target category, and calculating the autism risk evaluation parameter of the target to be evaluated within the time period corresponding to the image sequence by weighting.

2. The method of claim 1, wherein, The method of acquiring video data containing at least one target to be evaluated in a teaching scene, and processing the video data to obtain an image sequence corresponding to the teaching scene, which is composed of image frames arranged according to a preset time interval, comprises: acquiring two images with overlapping areas at each time through image acquisition devices arranged at two different positions in the teaching scene; dividing the two images with overlapping areas acquired at each time based on a preset time interval to obtain image sequences corresponding to the two image acquisition devices, respectively; extracting features from the image sequences corresponding to the two image acquisition devices to obtain corresponding sets of feature points; performing feature point matching on the images acquired by the two image acquisition devices based on the sets of feature points through a transformation matrix, and splicing the images acquired by the two image acquisition devices to obtain the image sequence.

3. The method of claim 1, wherein, The method of inputting the image sequence into a trained diffusion model, and determining, by the trained diffusion model, a head gaze target area of each target to be evaluated in each frame of image, comprises: compressing each frame of image in the image sequence respectively, and adding a noise image to obtain an image sequence containing noise; inputting the image sequence containing noise into a denoising model to obtain a denoised image, and extracting features from the denoised image to obtain a first feature containing a relationship between a target to be evaluated and a gaze target, and a second feature containing the teaching scene, and obtaining a multi-scale scene feature map containing the first feature and the second feature.

4. The method of claim 3, wherein, The method of determining, by the trained diffusion model, a head gaze target area of each target to be evaluated in each frame of image, comprises: performing feature encoding on the multi-scale scene feature map to obtain scene encoding features; performing spatial dimension expansion on the scene encoding features through a decoder to obtain decoding features; The head detection map is obtained by processing the decoding feature through a single-layer convolutional layer, the head feature is extracted based on the head detection map, and the fusion feature is obtained by fusing the head feature and the decoding feature; The head heat map, the gaze heat map and the connection map are obtained by processing the fusion feature through a convolutional layer, and the head gaze target region is obtained based on the head heat map, the gaze heat map and the connection map.

5. A device based on the diffusion model based cross-layer multi-granularity autism risk assessment method of any one of claims 1-4, characterized in that, The device comprises: An image acquisition module is configured to acquire video data containing at least one to-be-evaluated target in a teaching scene, and process the video data to obtain an image sequence composed of image frames arranged at a preset time interval corresponding to the teaching scene; A region extraction module is configured to input the image sequence into a trained diffusion model, and determine the head gaze target region of each to-be-evaluated target in each image frame through the trained diffusion model; A risk evaluation module is configured to determine the gaze target category of the to-be-evaluated target at a corresponding time based on the head gaze target region of each image frame, and calculate the autism risk evaluation parameter of the to-be-evaluated target within a time period corresponding to the image sequence based on the gaze target category of the to-be-evaluated target at each time.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the method of any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method of any one of claims 1 to 4.

Images