Cross-layer multi-granularity autism risk assessment method based on diffusion model

Abstract

The embodiment of the invention discloses a cross-layer multi-granularity autism risk assessment method based on a diffusion model, and relates to the technical field of computers, and the method comprises the steps: obtaining video data of a to-be-assessed target in a teaching scene, carrying out the processing of the video data, obtaining an image sequence, inputting the image sequence into a trained diffusion model, determining a head fixation target area of each to-be-evaluated target in each frame of image through a trained diffusion model; and determining a fixation target category based on the head fixation target area of each frame of image, and calculating to obtain autism risk assessment parameters in the corresponding time period range. According to the method, the risk that each to-be-assessed target possibly has the autism can be judged according to the autism risk assessment parameters, the risk degree of the target suffering from the autism can be objectively and accurately predicted, related personnel can take prevention measures in advance, the risk assessment cost can be reduced, and the risk assessment efficiency can be improved. And meanwhile, the accuracy and efficiency of risk assessment can be ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a cross-layer and multi-granularity autism risk assessment method based on a diffusion model. Background Art

[0002] Autism is a neurodevelopmental disorder characterized by difficulties with social communication, repetitive behaviors, and narrow interests. Early identification of autism risk and appropriate intervention can significantly improve an individual's developmental trajectory.

[0003] At present, the risk assessment of autism in children mainly relies on behavioral observation and questionnaires. The process of risk assessment is subjective and depends on the experience of the risk assessor, which may lead to differences in the assessment results. In addition, functional magnetic resonance imaging, eye tracking technology and other assessment methods can also be used to provide high-precision data, but this method has high requirements for equipment, the equipment is expensive, and the operation process of the equipment depends on the experience of the operator, which is difficult to be widely used in routine autism risk assessment.

[0004] Therefore, there is currently a lack of a method that can accurately, cost-effectively and objectively assess the target's risk of autism. Summary of the invention

[0005] The embodiment of the present application provides a cross-layer multi-granularity autism risk assessment method based on a diffusion model to solve the defects in the above-mentioned related technologies. The technical solution is as follows:

[0006] In a first aspect, the present application provides a cross-layer multi-granularity autism risk assessment method based on a diffusion model, comprising:

[0007] Acquire video data containing at least one target to be evaluated in a teaching scene, and obtain an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing;

[0008] Inputting the image sequence into a trained diffusion model, and determining the head gaze target area of ​​each target to be evaluated in each frame of the image by using the trained diffusion model;

[0009] The gaze target category of the target to be evaluated at the corresponding moment is determined based on the head gaze target area of ​​each frame image, and the autism risk assessment parameters of the target to be evaluated within the time period corresponding to the image sequence are calculated based on the gaze target category of the target to be evaluated at each moment.

[0010] In an optional solution of the first aspect, the acquiring of video data containing at least one target to be evaluated in a teaching scene, and obtaining an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing, comprises:

[0011] Two images with overlapping areas are acquired at each moment by image acquisition devices respectively arranged at two different positions in the teaching scene;

[0012] Dividing the two images with overlapping areas acquired at each moment based on a preset time interval to obtain image sequences corresponding to the two image acquisition devices respectively;

[0013] Extracting features from the image sequences corresponding to the two image acquisition devices to obtain corresponding feature point sets;

[0014] Based on the feature point set, feature point matching is performed on the images captured by the two image acquisition devices through a transformation matrix, and the images captured by the two image acquisition devices are spliced ​​to obtain the image sequence.

[0015] In an optional solution of the first aspect, inputting the image sequence into a trained diffusion model, and determining the head gaze target area of ​​each target to be evaluated in each frame of the image by using the trained diffusion model, comprises:

[0016] Compressing each frame of the image in the image sequence respectively and adding a noise image to obtain an image sequence containing noise;

[0017] The noisy image sequence is input into a denoising model to obtain a denoised image, and feature extraction is performed on the denoised image to obtain a first feature including the relationship between the target to be evaluated and the gaze target and a second feature including the teaching scene, thereby obtaining a multi-scale scene feature map including the first feature and the second feature.

[0018] In an optional solution of the first aspect, the image sequence containing noise is input into a denoising model, and feature extraction is performed on the denoised image to obtain a first feature containing a relationship between the target to be evaluated and the gaze target and a second feature containing the teaching scene, including:

[0019] Convolving the noise image to obtain a feature vector, convolving the feature vector to obtain two feature map groups, and processing the two feature map groups to obtain an image residual;

[0020] And the corresponding features are connected through the image residual to extract the preliminary features;

[0021] Taking the preliminary feature as input, extracting the first feature and the second feature;

[0022] The scene feature of the teaching scene is obtained based on the fusion of the first feature and the second feature.

[0023] In an optional solution of the first aspect, determining the head gaze target area of ​​each target to be evaluated in each frame of image by using the trained diffusion model includes:

[0024] Performing feature coding on the scene features of the teaching scene to obtain scene coding features;

[0025] Expanding the spatial dimension of the scene coding feature by a decoder to obtain a decoding feature;

[0026] Processing the decoded features through a single convolutional layer to obtain a head detection image, extracting head features based on the head detection image, and fusing the head features with the decoded features to obtain fused features;

[0027] A head heat map, a gaze heat map and a connection map are obtained based on the fusion features through convolution layer processing, and the head gaze target area is obtained based on the head heat map, the gaze heat map and the connection map.

[0028] In an optional solution of the first aspect, the step of calculating the autism risk assessment parameter of the target to be assessed within the time period corresponding to the image sequence based on the gaze target category of the target to be assessed at each moment includes:

[0029] Obtaining a gaze target category corresponding to each image frame in the image sequence;

[0030] Determine the ratio of the number of image frames corresponding to each type of the gaze target category in the image sequence, determine the weight of each type of the gaze target category based on the ratio corresponding to each type of the gaze target category, and obtain the autism risk assessment parameters of the target to be evaluated within the time period corresponding to the image sequence through weighted calculation.

[0031] In a second aspect, the embodiment of the present application further provides a cross-layer multi-granularity autism risk assessment device based on a diffusion model, comprising:

[0032] An image acquisition module, used to acquire video data containing at least one target to be evaluated in a teaching scene, and obtain an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing;

[0033] A region extraction module, used for inputting the image sequence into a trained diffusion model, and determining the head gaze target region of each target to be evaluated in each frame of the image through the trained diffusion model;

[0034] The risk assessment module is used to determine the gaze target category of the target to be assessed at the corresponding moment based on the head gaze target area of ​​each frame image, and calculate the autism risk assessment parameters of the target to be assessed within the time period corresponding to the image sequence based on the gaze target category of the target to be assessed at each moment.

[0035] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method provided in the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect is implemented.

[0036] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the first aspect of the embodiment of the present application or any one of the implementations of the first aspect.

[0037] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0038] The embodiment of the present application provides a cross-layer and multi-granularity autism risk assessment method based on a diffusion model. By collecting video data of a target to be assessed in a teaching scene, the video data is subjected to binocular stitching processing and fixed-length division, and then input into a trained diffusion model. A connection heat map is used to clarify the head and the object being gazed of each target to be assessed, so as to more accurately judge the child's gaze area, and then calculate autism risk assessment parameters according to changes in the gaze target of each target to be assessed, and then judge the risk of autism for each target to be assessed according to the autism risk assessment parameters. This can objectively and accurately predict the risk level of autism in the target, which is conducive to relevant personnel taking preventive measures in advance. It can not only reduce the cost of risk assessment, but also ensure the accuracy and efficiency of risk assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1It is a flowchart of a cross-layer multi-granularity autism risk assessment method based on a diffusion model provided in an embodiment of the present application;

[0041] Figure 2 It is a flowchart of a cross-layer multi-granularity autism risk assessment method based on a diffusion model provided in an embodiment of the present application;

[0042] Figure 3 It is a structural schematic diagram of a cross-layer multi-granularity autism risk assessment device based on a diffusion model provided in an embodiment of the present application;

[0043] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] The terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.

[0046] It should be noted that the terms "first\second" involved in the present application are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those described or illustrated herein.

[0047] It should be noted that with the application of deep learning models in the field of autism prevention, the prevention methods of autism have also changed greatly. Considering the characteristics of the target to be evaluated, such as children's behavioral hyperactivity, it is difficult to cooperate with conventional detection methods. Therefore, by improving the degree of cooperation of children and improving the detection efficiency, the attention target detection integrated into the visual field can effectively identify the characteristics of children in social interaction, language communication and behavior patterns, and achieve the purpose of risk assessment.

[0048] Understandably, children with autism look at social stimuli (such as faces and eyes) for a significantly lower time and frequency than non-autistic children, but show a relatively high degree of concentration when facing non-social stimuli (such as objects and patterns). In this way, the unique pattern of children's attention allocation can be determined, and risk assessment parameters can be given, which can provide a basis for early intervention in autism. In this way, not only can the risk assessment of autism be made more convenient, but the accuracy of the risk assessment can also be guaranteed, so that parents and teachers can take the next step based on the results of the risk assessment.

[0049] The present application is described in detail below with reference to specific embodiments.

[0050] Next, combine Figure 1 , introduces the cross-layer multi-granularity autism risk assessment method based on the diffusion model provided by the embodiment of the present application. For details, please refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of a cross-layer multi-granularity autism risk assessment method based on a diffusion model provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0051] S101, obtaining video data containing at least one target to be evaluated in a teaching scene, and obtaining an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing;

[0052] S102, inputting the image sequence into a trained diffusion model, and determining the head gaze target area of ​​each target to be evaluated in each frame of the image by using the trained diffusion model;

[0053] S103, determining the gaze target category of the target to be evaluated at the corresponding moment based on the head gaze target area of ​​each frame image, and calculating the autism risk assessment parameters of the target to be evaluated within the time period corresponding to the image sequence based on the gaze target category of the target to be evaluated at each moment.

[0054] In some embodiments, S101 may include:

[0055] Specifically, two image acquisition devices can be respectively arranged in different areas of the teaching scene, such as a classroom, and the images of the two image acquisition devices have overlapping areas. For example, an RGB camera and an infrared camera can be arranged on the left and right sides of the teaching scene, respectively, to acquire multi-angle video data V of the target. RGB and V RED , the collection time can be in class hours.

[0056] Specifically, V can beRGB and V RED Divide and obtain the image sequence SEQ RGB ={IMG RGB1 ,IMG RGB2 ,…,IMG RGBN} and SEQ RED ={IMG RED1 ,IMG RED2 ,…,IMG REDN}.

[0057] Specifically, the dual-target positioning method can be used for splicing. First, the SIFT algorithm is used to extract the image sequence SEQ RGB and SEQ RED The feature point set of each image is stored in the set SET RGB {Feat RGB1 ,Feat RGB2 ,…Feat RGBN ,} and SET RED {Feat RED1 ,Feat RED2 ,…Feat REDN ,}.

[0058] Furthermore, the KD tree and BFF algorithm can be used to identify the feature point set SET RGB and SET RED The feature points corresponding to each image are matched and searched, and preliminary screening is performed based on the ratio of the nearest neighbor and the next nearest neighbor distance. The transformation matrix H is calculated using the RANSAC algorithm, and the image IMG is transformed through the transformation matrix RED Corresponding Feat RED The feature points are transformed into matching points, that is, IMG RED Feature point transformation in IMG RGB Then IMG RED After the transformation matrix H is transformed into a new image, IMG RGB Finally, the scene image sequence SEQ after the images captured by the two image acquisition devices are spliced ​​is obtained. SCE ={IMG SCE1 ,IMG SCE2 ,…,IMG SCEN}.

[0059] In some embodiments, in S102, Figure 2 As shown, specifically including:

[0060] Specifically, each frame of the image in the image sequence may be compressed respectively, and a noise image may be added to obtain an image sequence containing noise;

[0061] The noisy image sequence is input into a denoising model to obtain a denoised image, and feature extraction is performed on the denoised image to obtain a first feature including the relationship between the target to be evaluated and the gaze target and a second feature including the teaching scene, thereby obtaining a multi-scale scene feature map including the first feature and the second feature.

[0062] Specifically, a variational autoencoder can be used to encode the scene graph sequence SEQ SCE Compress and add noise images to obtain the latent noise image sequence SEQ NOI ={IMG NOI1 ,IMG NOI2 ,…,IMG NOIN}.

[0063] Furthermore, the latent noise IMG NOI Input to the denoising model (SU-Net).

[0064] Input image IMG NOI After the feature tensor X is obtained through 3×3 convolution, it is then subjected to 1×1 convolution, segmentation and concatenation operations, and the 1×1 convolution effectively and efficiently promotes the information fusion between the SwinT block and the RConv block to extract the fused feature F IN ={F1,F2,F3,F4}.

[0065] Denoting the feature maps as J, K, and L, the attention is calculated as follows:

[0066]

[0067] Among them, γ is a learnable scalar and B represents the bias.

[0068] After applying the Softmax function, the self-attention calculation can be expressed as:

[0069] tten(J,K,L)=Softmax(M+Sim(J,K))*L;

[0070] Where M represents the mask matrix.

[0071] Specifically, the denoising process using the denoising model consists of downsampling and upsampling processes consisting of strided convolution and transposed convolution.

[0072] The fusion feature F INAs input, it contains four scale features. Each scale has a residual connection between the downsampling based on 2×2 strided convolution and the upsampling based on 2×2 transposed convolution. In addition, 4 SC blocks are taken between downsampling and upsampling to enhance the local and non-local modeling capabilities. Finally, after 4 SC blocks, a multi-scale scene feature map F containing the second feature and the first feature is obtained. OT ={F1, F2, F3, F4}, wherein the first feature includes the relationship between the target to be evaluated and the gaze object, and the second feature includes scene information, etc.

[0073] Furthermore, the output multi-scale scene feature map F of SU-Net is OT Perform feature encoding to obtain scene encoding feature F en , gradually convert F en The spatial dimension of the decoded feature F is expanded de , where the decoding feature F de Part of it is used for the head target prediction branch, in which F de Generate head detection map H through convolution d Used to strengthen the head prior, and then extract the head feature F through the head detection map head With F de Concatenate to get the fusion feature F prop .

[0074] Furthermore, the fusion feature F prop As input, the head heat map H, gaze heat map G and connection map C are generated by the prediction head with convolutional layers, and finally the head gaze target area is obtained.

[0075] In some embodiments, the training process of determining the head orientation region by the diffusion model may include the following steps:

[0076] The total loss of training is calculated based on the matching pairs of predicted head gaze target regions and actual head gaze target regions, and is expressed as follows:

[0077] L total =γ H *L H +γ G *L G +γ C *L C +γ d *L d ;

[0078] Where L H , L G and L C They are represented as the losses of head heatmap, gaze heatmap and connection map respectively. L dis the loss of the head detection map, and γ is the weight of each loss term.

[0079] Among them, for the head heat map loss, the predicted head heat map {P1,P2,…,P N} and truth value The head loss is expressed as:

[0080]

[0081] Where N is the number of regions with actual head gaze targets in the input image, w and h are the width and height of the heatmap, and P n (i, j) is the pixel value of the nth head heat map at position (i, j), L G and L C The calculation of MSE loss is related to L H same.

[0082] Specifically, for the head detection image L d Use MSE loss. Since each scene image has only one head detection map, the loss function becomes:

[0083]

[0084] In some embodiments, in S103, the gaze target category of the target to be evaluated at the corresponding moment can be determined based on the head gaze target area of ​​each frame image, and the autism risk assessment parameters of the target to be evaluated within the time period corresponding to the image sequence can be calculated based on the gaze target category of the target to be evaluated at each moment.

[0085] Specifically, the gaze target category corresponding to each image frame in the image sequence may be obtained;

[0086] Determine the ratio of the number of image frames corresponding to each type of the gaze target category in the image sequence, determine the weight of each type of the gaze target category based on the ratio corresponding to each type of the gaze target category, and obtain the autism risk assessment parameters of the target to be evaluated within the time period corresponding to the image sequence through weighted calculation.

[0087] In some embodiments, the gaze target of each target to be evaluated can be classified to determine the gaze target category, and different weights can be given to each gaze target category for calculation, and then compared based on a given risk assessment threshold to predict the risk level of autism for the target to be evaluated.

[0088] Specifically, the target of the target's gaze can be determined based on the area the target is fixating on, and the proportion of different gaze areas can be obtained based on the entire image sequence. The corresponding autism risk assessment parameter can be obtained by multiplying the proportion P of different areas by the weight. The higher the score, the greater the risk of autism.

[0089] For example, the threshold of the autism risk assessment parameter can be selected as 0.5. If the autism risk assessment parameter is less than 0.5, an assessment result indicating that the target to be assessed has a low risk of autism can be output; otherwise, an assessment result indicating that the target to be assessed is a normal child can be output; otherwise, an assessment result indicating that the target to be assessed has a high risk of autism can be output.

[0090] Exemplarily, the gaze target category may include a gaze teacher (G tea ), watching the playmate (G FRI ), look down (G low ) and looking at other things (G oth ), and give different weights to different categories {λ1=0.1, λ1=0.2, λ1=0.4, λ1=0.3}. For example, it can be considered that the child’s gaze at the teacher and playmate at the current moment indicates a positive and healthy mental state, while lowering the head or looking at other things at the current moment requires further evaluation.

[0091] The calculation of autism risk assessment parameters can be applied using the formula:

[0092] Score=λ1*P tea +λ2*P FRI +λ3*P low +λ4*P oth ;

[0093] where λ i Represents the weights of different categories, and P represents the proportion of different categories.

[0094] In some embodiments, in S103, the EEG data can also be combined to determine the EEG signal corresponding to each video frame in the image sequence, and the autism risk assessment parameters can be calculated based on the frequency of the EEG signal and the gaze target category identified in S102. This embodiment of the present application is not limited to this.

[0095] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0096] See next Figure 3, a schematic diagram of the structure of a cross-layer multi-granularity autism risk assessment device based on a diffusion model provided by an exemplary embodiment of the present application. The device can be implemented as all or part of a terminal through software, hardware, or a combination of the two, and can also be integrated on a server as an independent module. The cross-layer multi-granularity autism risk assessment device based on a diffusion model in the embodiment of the present application can be applied to a terminal or a cloud, and the device 30 includes an image acquisition module 301, a region extraction module 302, and a risk assessment module 303, wherein:

[0097] The image acquisition module 301 is used to obtain video data containing at least one target to be evaluated in the teaching scene, and obtain an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing;

[0098] The region extraction module 302 is used to input the image sequence into the trained diffusion model, and determine the head gaze target region of each target to be evaluated in each frame of the image through the trained diffusion model;

[0099] The risk assessment module 303 is used to determine the gaze target category of the target to be assessed at the corresponding moment based on the head gaze target area of ​​each frame image, and calculate the autism risk assessment parameters of the target to be assessed within the time period corresponding to the image sequence based on the gaze target category of the target to be assessed at each moment.

[0100] It should be noted that the device 30 provided in the above embodiment only uses the division of the above functional modules as an example when executing the cross-layer multi-granularity autism risk assessment method based on the diffusion model. In practical applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the embodiment of the cross-layer multi-granularity autism risk assessment method based on the diffusion model belong to the same concept. The implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0101] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the program.

[0102] See also Figure 4 , which is a structural block diagram of an electronic device provided in an embodiment of the present application.

[0103] like Figure 4 As shown, the electronic device 400 includes: a processor 401 and a memory 402 .

[0104] In the embodiment of the present application, the processor 401 is the control center of the computer system, which can be a processor of a physical machine or a processor of a virtual machine. The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array).

[0105] The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also called a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0106] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is used to be executed by the processor 401 to implement the method in the embodiment of the present application.

[0107] In some embodiments, the electronic device 400 further includes: a peripheral device interface 403 and at least one peripheral device 404. The processor 401, the memory 402 and the peripheral device interface 403 can be connected via a bus or a signal line. Each peripheral device 404 can be connected to the peripheral device interface 403 via a bus, a signal line or a circuit board. Specifically, the peripheral device 404 includes: a display screen, a camera and an audio circuit. The peripheral device interface 403 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 401 and the memory 402.

[0108] In some embodiments of the present application, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 401, the memory 402, and the peripheral device interface 403 can be implemented on a separate chip or circuit board. This embodiment of the present application does not specifically limit this.

[0109] The electronic device structure block diagram shown in the embodiment of the present application does not constitute a limitation on the electronic device 400. The electronic device 400 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0110] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method of any of the above embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cross-layer and multi-granular autism risk assessment method based on diffusion model, It is characterized in that include: Acquire video data containing at least one target to be evaluated in a teaching scene, and obtain an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing; Inputting the image sequence into a trained diffusion model, and determining the head gaze target area of ​​each target to be evaluated in each frame of the image by using the trained diffusion model; The gaze target category of the target to be evaluated at the corresponding moment is determined based on the head gaze target area of ​​each frame image, and the autism risk assessment parameters of the target to be evaluated within the time period corresponding to the image sequence are calculated based on the gaze target category of the target to be evaluated at each moment.

2. The method according to claim 1, It is characterized in that The step of acquiring video data containing at least one target to be evaluated in a teaching scene, and obtaining an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing, includes: Two images with overlapping areas are acquired at each moment by image acquisition devices respectively arranged at two different positions in the teaching scene; Dividing the two images with overlapping areas acquired at each moment based on a preset time interval to obtain image sequences corresponding to the two image acquisition devices respectively; Extracting features from the image sequences corresponding to the two image acquisition devices to obtain corresponding feature point sets; Based on the feature point set, feature point matching is performed on the images captured by the two image acquisition devices through a transformation matrix, and the images captured by the two image acquisition devices are spliced ​​to obtain the image sequence.

3. The method according to claim 1, It is characterized in that The step of inputting the image sequence into a trained diffusion model and determining the head gaze target area of ​​each target to be evaluated in each frame of the image by using the trained diffusion model comprises: Compressing each frame of the image in the image sequence respectively and adding a noise image to obtain an image sequence containing noise; The noisy image sequence is input into a denoising model to obtain a denoised image, and feature extraction is performed on the denoised image to obtain a first feature including the relationship between the target to be evaluated and the gaze target and a second feature including the teaching scene, thereby obtaining a multi-scale scene feature map including the first feature and the second feature.

4. The method according to claim 3, It is characterized in that The step of inputting the image sequence containing noise into a denoising model, performing feature extraction on the denoised image, and obtaining a first feature containing the relationship between the target to be evaluated and the gaze target and a second feature containing the teaching scene comprises: Convolving the noise image to obtain a feature vector, convolving the feature vector to obtain two feature map groups, and processing the two feature map groups to obtain an image residual; And the corresponding features are connected through the image residual to extract the preliminary features; Taking the preliminary feature as input, extracting the first feature and the second feature; The scene feature of the teaching scene is obtained based on the fusion of the first feature and the second feature.

5. The method according to claim 4, It is characterized in that The step of determining the head gaze target area of ​​each target to be evaluated in each frame of image by using the trained diffusion model includes: Performing feature coding on the scene features of the teaching scene to obtain scene coding features; Expanding the spatial dimension of the scene coding feature by a decoder to obtain a decoding feature; Processing the decoded features through a single convolutional layer to obtain a head detection image, extracting head features based on the head detection image, and fusing the head features with the decoded features to obtain fused features; A head heat map, a gaze heat map and a connection map are obtained based on the fusion features through convolution layer processing, and the head gaze target area is obtained based on the head heat map, the gaze heat map and the connection map.

6. The method according to any one of claims 1 to 5, It is characterized in that The step of calculating the autism risk assessment parameter of the target to be evaluated within the time period corresponding to the image sequence based on the gaze target category of the target to be evaluated at each moment includes: Obtaining a gaze target category corresponding to each image frame in the image sequence; Determine the ratio of the number of image frames corresponding to each type of the gaze target category in the image sequence, determine the weight of each type of the gaze target category based on the ratio corresponding to each type of the gaze target category, and obtain the autism risk assessment parameters of the target to be evaluated within the time period corresponding to the image sequence through weighted calculation.

7. A cross-layer multi-granularity autism risk assessment device based on diffusion model, It is characterized in that include: An image acquisition module, used to acquire video data containing at least one target to be evaluated in a teaching scene, and obtain an image sequence consisting of image frames arranged at preset time intervals corresponding to the teaching scene based on the video data processing; A region extraction module, used for inputting the image sequence into a trained diffusion model, and determining the head gaze target region of each target to be evaluated in each frame of the image through the trained diffusion model; The risk assessment module is used to determine the gaze target category of the target to be assessed at the corresponding moment based on the head gaze target area of ​​each frame image, and calculate the autism risk assessment parameters of the target to be assessed within the time period corresponding to the image sequence based on the gaze target category of the target to be assessed at each moment.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images