Methods and devices for detecting long-distance mouth breathing in preschool children

Abstract

This invention discloses a method and device for detecting mouth breathing at a distance in preschool children, relating to the field of image detection. The method includes: inputting video frames into a trained long-distance mouth breathing detection model; first, passing the frames through a backbone network, where a first feature extraction module, a second feature extraction module, a third feature extraction module, and a SPPF module output first, second, third, and fourth features respectively; performing gradient-wise fusion to obtain four semantic features; then inputting these features into a long-distance cross-receptive field dynamic attention module; and sequentially passing them through a hole-focusing fusion module and a three-path feature cascade collaborative module to extract enhancement features; finally, inputting the first, second, third, and enhancement features into a neck network to obtain four attention features, which are then processed through a multi-stage recursive self-aggregation module to obtain aggregated features, which are then input into a detection head to output the mouth breathing state category probability and the target bounding box. This invention addresses the problem of low accuracy in existing detection models.

Description

Technical Field

[0001] This invention relates to the field of image detection, specifically to a method and apparatus for detecting the mouth breathing state of preschool children at a distance. Background Technology

[0002] In recent years, with the deepening development of the smart education concept and the continuous advancement of the new education infrastructure strategy, the construction of smart kindergartens has become a key area of ​​digital transformation in education. Preschool children are in a critical period of facial and jawbone development; habitual mouth breathing can affect normal facial development and lead to health problems such as respiratory dysfunction. Traditional kindergarten health monitoring systems are limited by the subjectivity of manual observation and the spatial resolution constraints of high-position cameras. Existing detection models, due to limited detail recognition capabilities and restricted receptive fields, struggle to effectively capture the subtle features of preschool children's mouths in distant scenes. Summary of the Invention

[0003] The purpose of this application is to provide a method and device for detecting the long-distance mouth breathing status of preschool children in response to the aforementioned technical problems.

[0004] In a first aspect, the present invention provides a method for detecting the long-distance mouth breathing state of preschool children, comprising the following steps:

[0005] A remote mouth breathing state detection model based on an improved YOLO network was constructed and trained to obtain the trained remote mouth breathing state detection model. The remote mouth breathing state detection model adds a remote cross-receptive field dynamic attention module between the SPPF module in the backbone network of the YOLO network and the upsampling unit in the neck network, and adds a multi-order recursive self-aggregation module between the neck network and the detection head. The remote cross-receptive field dynamic attention module includes a hole focusing fusion module and a three-path feature cascade collaborative module connected in sequence.

[0006] Video frames containing the preschool children to be detected are acquired and input into a trained long-range mouth breathing state detection model. The frames first pass through a backbone network. The first, second, and third feature extraction modules and the SPPF module within the backbone network output the first, second, third, and fourth features, respectively. These features are then fused gradient-wise to obtain four semantic features. These four semantic features are input into a long-range cross-receptive field dynamic attention module, where they undergo adaptive fusion via a hole-focusing fusion module to obtain focused fusion features. The focused fusion features are then processed by a three-path feature cascade collaborative module to extract reinforcement features. The first, second, third, and reinforcement features are input into a neck network to obtain four attention features. These four attention features are then processed by a multi-level recursive self-aggregation module to obtain aggregated features. The aggregated features are then processed by a detection head, which outputs the corresponding mouth breathing state category probability and bounding box. Based on the mouth breathing state category probability, the model is determined to be either mouth breathing or normal breathing.

[0007] Preferably, the first, second, third, and fourth features are first fused gradient-wise to obtain four semantic features, specifically including:

[0008] A semantic resolution hierarchical strategy is adopted for gradient-wise fusion, specifically as follows:

[0009] The first feature is used as both the input feature of the first-layer gradient fusion and the first semantic feature; the second feature is used as the input feature of the second-layer gradient fusion and is fused with the input feature of the first-layer gradient fusion to obtain the second semantic feature; the third feature is used as the input feature of the third-layer gradient fusion and is fused with the input feature of the second-layer gradient fusion to obtain the third semantic feature; the fourth feature is used as the input feature of the fourth-layer gradient fusion and is fused with the input feature of the third-layer gradient fusion to obtain the fourth semantic feature, as shown in the following formula:

[0010] ;

[0011] in, Indicates the first Input features of layer gradient fusion , This indicates a global average pooling operation. Indicates the expansion rate The dilated convolution operation, This indicates that a splicing operation is performed along the channel dimension. Indicates the first One semantic feature.

[0012] As a preferred option, the cavity focusing fusion module specifically includes:

[0013] The four semantic features are first subjected to max pooling and average pooling operations respectively, and then dilated convolution is used to extract four intermediate features, as shown in the following formula:

[0014] ;

[0015] in, Indicates the first An intermediate feature, This represents the global max pooling operation;

[0016] Each intermediate feature is multiplied by its corresponding weight using the Hadamard algorithm, and then concatenated along the channel dimension to obtain the focused fusion feature, as shown in the following formula:

[0017] ;

[0018] in, This represents the Hadamard product operation. Indicates the first Each weight, This indicates a focus on integration features.

[0019] Preferably, the three-path feature cascaded collaborative module includes a first path, a second path, and a third path. The first path includes a first channel semantic focusing module and a first spatial semantic focusing module connected in sequence. The second path is where the focused fusion feature is directly used as the output feature. The third path includes a second spatial semantic focusing module and a second channel semantic focusing module connected in sequence. Both the first and second channel semantic focusing modules use channel semantic focusing modules, and both the first and second spatial semantic focusing modules use spatial semantic focusing modules. The focused fusion feature passes through the first, second, and third paths respectively, yielding the output features of the first, second, and third paths. The output features of the first, second, and third paths are first subjected to a Hadamard product operation, followed by a cube root nonlinear transformation to obtain the enhanced feature, as shown in the following equation.

[0020] ;

[0021] in, and These represent the functions corresponding to the channel semantic focusing module and the spatial semantic focusing module, respectively. This indicates an enhanced feature.

[0022] Preferably, the calculation process of the channel semantic focusing module specifically includes:

[0023] The input features of the channel semantic focusing module are multiplied by the key weight matrix, and then subjected to global max pooling to obtain max pooled features. The input features of the channel semantic focusing module are multiplied by the query weight matrix, and then subjected to global average pooling to obtain average pooled features. The max pooled features and average pooled features are flattened to obtain the key vector and query vector of the channel semantic focusing module, as shown in the following formula:

[0024] ;

[0025] ;

[0026] in, This represents the input features of the channel semantic focusing module. Indicates the flattening operation. and These represent the query vector and key vector of the channel semantic focusing module, respectively. and These represent the query weight matrix and key weight matrix of the channel semantic focusing module, respectively;

[0027] The key vector and query vector of the channel semantic focusing module are concatenated and input into a two-layer perceptron. After masking and dimensionality expansion operations, and then group normalization and Mish activation function processing, the channel attention weights are obtained, as shown in the following formula:

[0028] ;

[0029] in, This represents a two-layer perceptron. Indicates masking operation. This indicates a dimensional expansion operation. This indicates that group normalization is performed along the channel dimension. express Activation function Indicates channel attention weights;

[0030] The input features of the channel semantic focus module are multiplied by the value weight matrix to obtain the value vector of the channel semantic focus module. The channel attention weights are then multiplied pixel by pixel by the value vector of the channel semantic focus module to obtain the output features of the channel semantic focus module, as shown in the following formula:

[0031] ;

[0032] ;

[0033] in, This represents the value weight matrix of the channel semantic focusing module. This represents the value vector of the channel semantic focusing module. This indicates that pixel-by-pixel multiplication is performed using broadcast operations. This represents the output features of the channel semantic focusing module;

[0034] The computation process of the spatial semantic focusing module specifically includes:

[0035] The input features of the spatial semantic focusing module are multiplied by the query weight matrix and the key weight matrix, respectively, to obtain the query vector and key vector of the spatial semantic focusing module, as shown in the following formula:

[0036] ;

[0037] ;

[0038] in, This represents the input features of the spatial semantic focusing module. and These represent the query vector and key vector of the spatial semantic focusing module, respectively. and These represent the query weight matrix and key weight matrix of the spatial semantic focusing module, respectively;

[0039] The similarity between the query vector and the key vector of the spatial semantic focusing module is calculated using Euclidean distance. This similarity is then processed sequentially through masking, dilated convolution, layer normalization, and the Mish activation function to obtain the spatial attention weights, as shown in the following equation:

[0040] ;

[0041] in, Represents spatial attention weights. This represents the Euclidean distance between the query vector and the key vector in the spatial semantic focusing module. This indicates a hierarchical normalization operation performed in the spatial dimension;

[0042] The input features of the spatial semantic focusing module are multiplied by the value weight matrix to obtain the value vector of the spatial semantic focusing module; the spatial attention weights are then multiplied pixel-by-pixel by the value vector of the spatial semantic focusing module to obtain the output features of the spatial semantic focusing module, as shown in the following formula:

[0043] ;

[0044] ;

[0045] in, The value vector representing the spatial semantic focusing module. The value weight matrix represents the spatial semantic focusing module. This represents the output features of the spatial semantic focusing module.

[0046] Preferably, the neck network includes a bottom-up first multi-scale fusion path and a top-down second multi-scale fusion path. The first multi-scale fusion path includes three sequentially connected first fusion units, each consisting of a sequentially connected upsampling module, a first splicing layer, and a first feature fusion module. The second multi-scale fusion path includes three sequentially connected second fusion units, each consisting of a sequentially connected convolutional layer, a second splicing layer, and a second feature fusion module. The first feature, second feature, third feature, and fused feature are input into the neck network and sequentially pass through the first and second multi-scale fusion paths to obtain the first attention feature output by the third first fusion unit and the second, third, and fourth attention features output by the three second fusion units, respectively. The first, second, third, and fourth attention features constitute four attention features.

[0047] The first, second, third, and fourth attention features are input into a multi-level recursive self-aggregation module, undergoing three recursive self-aggregation stages. In the third-to-last recursive self-aggregation stage, the fourth attention feature is first upsampled, then multiplied by the third attention feature with its corresponding weight matrix using Hadamard multiplication, and then summed pixel-by-pixel to obtain the third weighted fusion feature. In the second-to-last recursive self-aggregation stage, the third weighted fusion feature is first upsampled, then multiplied by the second attention feature with its corresponding weight matrix using Hadamard multiplication, and then summed pixel-by-pixel to obtain the second weighted fusion feature. In the last recursive self-aggregation stage, the second weighted fusion feature is first upsampled, then multiplied by the first attention feature with its corresponding weight matrix using Hadamard multiplication, and then summed pixel-by-pixel to obtain the aggregated feature, as shown in the following equation:

[0048] ;

[0049] ;

[0050] ;

[0051] in, This indicates an upsampling operation, which upsamples the image to the same size and number of channels as the previous feature map. This indicates summation pixel by pixel. These represent the first and second weight matrices used in the h-th recursive self-aggregation phase from the end, respectively. The elements in the i-th row and j-th column are respectively and The corresponding expressions are as follows:

[0052] ;

[0053] ;

[0054] and , , and They are respectively represented as The index weight parameter in the i-th row and j-th column. , , These represent the third weighted fusion feature, the second weighted fusion feature, and the aggregated feature, respectively.

[0055] Secondly, the present invention provides a device for detecting the long-distance mouth breathing state of preschool children, comprising:

[0056] The model building module is configured to build and train a remote mouth breathing state detection model based on an improved YOLO network, resulting in a trained remote mouth breathing state detection model. The remote mouth breathing state detection model adds a remote cross-receptive field dynamic attention module between the SPPF module in the backbone network of the YOLO network and the upsampling unit in the neck network, and adds a multi-order recursive self-aggregation module between the neck network and the detection head. The remote cross-receptive field dynamic attention module includes a hole focusing fusion module and a three-path feature cascade collaborative module connected in sequence.

[0057] The recognition module is configured to acquire video frames containing the preschool child to be detected. The video frames are input into a trained long-range mouth breathing state detection model. They first pass through the backbone network. The first feature extraction module, the second feature extraction module, the third feature extraction module, and the SPPF module in the backbone network output the first feature, the second feature, the third feature, and the fourth feature, respectively. The first feature, the second feature, the third feature, and the fourth feature are fused gradient-wise to obtain four semantic features. The four semantic features are input into the long-range cross-receptive field dynamic attention module. They first pass through the hole focusing fusion module for adaptive fusion to obtain focusing fusion features. The focusing fusion features are then processed by the three-path feature cascade collaborative module to extract reinforcement features. The first feature, the second feature, the third feature, and the reinforcement features are input into the neck network to obtain four attention features. The four attention features are processed by the multi-level recursive self-aggregation module to obtain aggregate features. The aggregate features pass through the detection head and output the corresponding mouth breathing state category probability and the target box. The mouth breathing state category probability determines whether it is mouth breathing or normal breathing.

[0058] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0059] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0060] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] (1) The method for detecting the long-distance mouth breathing state of preschool children proposed in this invention designs a long-distance cross-receptive field dynamic attention module. This module adaptively fuses different semantic features and collaboratively focuses on channel and spatial semantics to achieve dynamic capture of multi-scale contextual information and cross-dimensional feature interaction. Among them, the hole focusing fusion module integrates complementary features of max pooling and average pooling, combined with hole convolution and learnable weights, to adaptively adjust the receptive field range and effectively focus on small targets in children's mouths in low-resolution scenes. The channel semantic focusing module and the spatial semantic focusing module respectively achieve accurate allocation of channel weights and accurate identification of key spatial regions through a two-layer perceptron and Euclidean distance metric. Compared with the traditional YOLOv8 model, this invention can solve the feature submersion problem caused by the small target pixel area. In addition, the three-path feature cascade collaborative module introduces Hadamard product and cube root nonlinear transformation, which strengthens cross-dimensional feature interaction while suppressing the interference of complex background noise. Therefore, it solves the problem of insufficient accuracy of long-distance small target detection caused by the reliance on fixed receptive field feature extraction and single-dimensional attention mechanism in traditional target detection models.

[0063] (2) The method for detecting the long-distance mouth breathing state of preschool children proposed in this invention proposes a multi-level recursive self-aggregation module, which breaks through the linear fusion limitation of the traditional Feature Pyramid Network (FPN). This module achieves progressive alignment between shallow detail features and deep semantic features through a step-by-step upsampling and weight fusion strategy. Among the four attention features output by the neck network, the high-order features are restored to their original resolution through upsampling and dynamically weighted with the low-order features to form aggregated features that have both high resolution and strong semantic expression.

[0064] (3) The method for detecting the long-distance mouth breathing state of preschool children proposed in this invention can not only adaptively fuse local details and global semantic information in multi-scale features to capture key information of the mouth of preschool children in high-resolution feature maps, but also enhance the saliency of small target features through nonlinear noise suppression, thereby achieving high-precision detection under complex background and low-resolution conditions. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic flowchart illustrating a method for detecting the remote mouth breathing status of preschool children according to an embodiment of this application.

[0067] Figure 2 This is a schematic diagram of the structure of a remote mouth breathing state detection model for a remote mouth breathing state detection method for preschool children, as described in an embodiment of this application.

[0068] Figure 3 This is a schematic diagram of the structure of the cavity focusing fusion module and the three-path feature cascaded collaborative module in the remote cross-receptive field dynamic attention module of the remote mouth breathing state detection method for preschool children according to an embodiment of this application.

[0069] Figure 4 This is a schematic diagram of the channel semantic focusing module of the method for detecting the remote mouth breathing state of preschool children according to an embodiment of this application;

[0070] Figure 5 This is a schematic diagram of the spatial semantic focusing module of the method for detecting the long-distance mouth breathing state of preschool children according to an embodiment of this application;

[0071] Figure 6 This is a schematic diagram of a remote mouth breathing detection device for preschool children, as an embodiment of this application.

[0072] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0074] Figure 1 An embodiment of this application illustrates a method for detecting the mouth breathing status of preschool children at a distance, comprising the following steps:

[0075] S1. Construct and train a remote mouth breathing state detection model based on the improved YOLO to obtain the trained remote mouth breathing state detection model. The remote mouth breathing state detection model adds a remote cross-receptive field dynamic attention module between the SPPF module in the backbone network of the YOLO network and the upsampling unit in the neck network, and adds a multi-level recursive self-aggregation module between the neck network and the detection head. The remote cross-receptive field dynamic attention module includes a hole focusing fusion module and a three-path feature cascade collaborative module connected in sequence.

[0076] Specifically, embodiments of this application propose a long-distance mouth breathing state detection model based on an improved YOLO network, denoted as the LST-YOLO model. This model improves upon the existing YOLO network, and the version of the YOLO network is not limited. Embodiments of this application use Fasternet's YOLOv8 as an example for the feature extraction module in the backbone network. The backbone network contains four feature extraction modules, from top to bottom: the first feature extraction module, the second feature extraction module, the third feature extraction module, and the fourth feature extraction module. (Refer to...) Figure 2In the embodiments of this application, the first feature extraction module, the second feature extraction module, the third feature extraction module, and the fourth feature extraction module all use Fasternet. In other examples, other versions of the YOLO network can also be selected. First, a semantic resolution hierarchical strategy is used to reconstruct the first feature output by the first feature extraction module, the second feature output by the second feature extraction module, and the third feature output by the third feature extraction module in the backbone network with the fourth feature output by the SPPF module, i.e., gradient-wise fusion, to obtain four semantic features. Second, a long-range cross-receptive field dynamic attention (LCDA) module is added between the backbone network and the neck network to improve the model's learning weights for features of small targets at a distance. Third, a multi-level recursive self-aggregation module is designed to upsample and dynamically fuse the four attention features output by the neck network step by step to gradually restore high-resolution details. This long-distance mouth breathing state detection model uses an attention mechanism to capture key mouth information of preschool children, and integrates multiple semantic features and uses dilated convolution operations to expand the receptive field, which is beneficial to improving the ability to perceive small targets at a distance.

[0077] In a specific embodiment, the first feature, the second feature, the third feature, and the fourth feature are first fused gradient-wise to obtain four semantic features, specifically including:

[0078] A semantic resolution hierarchical strategy is adopted for gradient-wise fusion, specifically as follows:

[0079] The first feature is used as both the input feature of the first-layer gradient fusion and the first semantic feature; the second feature is used as the input feature of the second-layer gradient fusion and is fused with the input feature of the first-layer gradient fusion to obtain the second semantic feature; the third feature is used as the input feature of the third-layer gradient fusion and is fused with the input feature of the second-layer gradient fusion to obtain the third semantic feature; the fourth feature is used as the input feature of the fourth-layer gradient fusion and is fused with the input feature of the third-layer gradient fusion to obtain the fourth semantic feature, as shown in the following formula:

[0080] ;

[0081] in, Indicates the first Input features of layer gradient fusion , This indicates a global average pooling operation. Indicates the expansion rate The dilated convolution operation, This indicates that a splicing operation is performed along the channel dimension. Indicates the first One semantic feature.

[0082] Specifically, in the embodiments of this application, the first feature output by the first feature extraction module, the second feature output by the second feature extraction module, the third feature output by the third feature extraction module, and the fourth feature output by the SPPF module in the backbone network are first fused gradient-wise using a semantic resolution hierarchical strategy to obtain semantic features from four different semantic domains. The four input semantic feature LCDA modules enable adaptive cross-dimensional interaction between deep and shallow semantic features and the capture of key semantic features, resulting in enhanced features. These enhanced features, along with the first feature output by the first feature extraction module, the second feature output by the second feature extraction module, and the third feature output by the third feature extraction module in the backbone network, are then input into the neck network to obtain four attention features. These are then input into a multi-level recursive self-aggregation module to complete feature resolution reconstruction and global propagation of semantic information. Finally, the aggregated features are input into the detector head to identify the exit breathing state.

[0083] In a specific embodiment, the hole focusing fusion module specifically includes:

[0084] The four semantic features are first subjected to max pooling and average pooling operations respectively, and then dilated convolution is used to extract four intermediate features, as shown in the following formula:

[0085] ;

[0086] in, Indicates the first An intermediate feature, This represents the global max pooling operation;

[0087] Each intermediate feature is multiplied by its corresponding weight using the Hadamard algorithm, and then concatenated along the channel dimension to obtain the focused fusion feature, as shown in the following formula:

[0088] ;

[0089] in, This represents the Hadamard product operation. Indicates the first Each weight, This indicates a focus on integration features.

[0090] For details, please refer to Figure 3The LCDA module mentioned in the embodiments of this application mainly consists of a Channel Semantic Focus Module (CSFM) and a Tri-Path Cascaded Collaborative Module (TPCCM). In the Channel Semantic Focus Module, the four semantic features are first subjected to max pooling and average pooling operations to align the feature map sizes. Then, dilated convolution is used to expand the receptive field and adjust the number of channels. Weights are then added to the pixels of each feature map to fuse the different semantic features. Through training, the model's attention is focused on low-pixel, distant, small targets, namely the mouth of a preschool child.

[0091] In a specific embodiment, the three-path feature cascade collaborative module includes a first path, a second path, and a third path. The first path includes a first channel semantic focusing module and a first spatial semantic focusing module connected in sequence. The second path is where the focused fusion feature is directly used as the output feature. The third path includes a second spatial semantic focusing module and a second channel semantic focusing module connected in sequence. Both the first and second channel semantic focusing modules are channel semantic focusing modules, and both the first and second spatial semantic focusing modules are spatial semantic focusing modules. The focused fusion feature passes through the first, second, and third paths respectively, yielding the output features of the first, second, and third paths. The output features of the first, second, and third paths are first subjected to a Hadamard product operation, followed by a cube root nonlinear transformation to obtain the enhanced feature, as shown in the following equation.

[0092] ;

[0093] in, and These represent the functions corresponding to the channel semantic focusing module and the spatial semantic focusing module, respectively. This indicates an enhanced feature.

[0094] In a specific embodiment, the calculation process of the channel semantic focusing module specifically includes:

[0095] The input features of the channel semantic focusing module are multiplied by the key weight matrix, and then subjected to global max pooling to obtain max pooled features. The input features of the channel semantic focusing module are multiplied by the query weight matrix, and then subjected to global average pooling to obtain average pooled features. The max pooled features and average pooled features are flattened to obtain the key vector and query vector of the channel semantic focusing module, as shown in the following formula:

[0096] ;

[0097] ;

[0098] in, This represents the input features of the channel semantic focusing module. Indicates the flattening operation. and These represent the query vector and key vector of the channel semantic focusing module, respectively. and These represent the query weight matrix and key weight matrix of the channel semantic focusing module, respectively;

[0099] The key vector and query vector of the channel semantic focusing module are concatenated and input into a two-layer perceptron. After masking and dimensionality expansion operations, and then group normalization and Mish activation function processing, the channel attention weights are obtained, as shown in the following formula:

[0100] ;

[0101] in, This represents a two-layer perceptron. Indicates masking operation. This indicates a dimensional expansion operation. This indicates that group normalization is performed along the channel dimension. express Activation function Indicates channel attention weights;

[0102] The input features of the channel semantic focus module are multiplied by the value weight matrix to obtain the value vector of the channel semantic focus module. The channel attention weights are then multiplied pixel by pixel by the value vector of the channel semantic focus module to obtain the output features of the channel semantic focus module, as shown in the following formula:

[0103] ;

[0104] ;

[0105] in, This represents the value weight matrix of the channel semantic focusing module. This represents the value vector of the channel semantic focusing module. This indicates that pixel-by-pixel multiplication is performed using broadcast operations. This represents the output features of the channel semantic focusing module;

[0106] The computation process of the spatial semantic focusing module specifically includes:

[0107] The input features of the spatial semantic focusing module are multiplied by the query weight matrix and the key weight matrix, respectively, to obtain the query vector and key vector of the spatial semantic focusing module, as shown in the following formula:

[0108] ;

[0109] ;

[0110] in, This represents the input features of the spatial semantic focusing module. and These represent the query vector and key vector of the spatial semantic focusing module, respectively. and These represent the query weight matrix and key weight matrix of the spatial semantic focusing module, respectively;

[0111] The similarity between the query vector and the key vector of the spatial semantic focusing module is calculated using Euclidean distance. This similarity is then processed sequentially through masking, dilated convolution, layer normalization, and the Mish activation function to obtain the spatial attention weights, as shown in the following equation:

[0112] ;

[0113] in, Represents spatial attention weights. This represents the Euclidean distance between the query vector and the key vector in the spatial semantic focusing module. This indicates a hierarchical normalization operation performed in the spatial dimension;

[0114] The input features of the spatial semantic focusing module are multiplied by the value weight matrix to obtain the value vector of the spatial semantic focusing module; the spatial attention weights are then multiplied pixel-by-pixel by the value vector of the spatial semantic focusing module to obtain the output features of the spatial semantic focusing module, as shown in the following formula:

[0115] ;

[0116] ;

[0117] in, The value vector representing the spatial semantic focusing module. These represent the value weight matrices of the spatial semantic focusing module. This represents the output features of the spatial semantic focusing module.

[0118] For details, please refer to Figure 4The three-path feature cascaded collaborative module mentioned in the embodiments of this application contains three branches, two of which consist of a Channel Semantic Focus Module (CSFM) and a Spatial Semantic Focus Module (SSFM). In these two branches, the focused fusion features obtained from the hole focusing fusion module undergo two operations: first, passing through the Channel Semantic Focus Module followed by the Spatial Semantic Focus Module to prioritize key channels and then refine spatial localization; second, passing through the Spatial Semantic Focus Module followed by the Channel Semantic Focus Module to first lock in potential spatial regions and then optimize channel representation. Then, the output features of the two branches are multiplied by the focused fusion features directly passed in the third branch using a Hadamard product to achieve cross-dimensional fine-grained interaction, followed by a cube root nonlinear transformation to suppress noise and obtain enhanced features.

[0119] For details, please refer to Figure 5 The channel semantic focusing module mainly includes global max pooling, global average pooling, a two-layer perceptron, and masking. First, the input features of the channel semantic focusing module are multiplied by the key weight matrix, followed by global max pooling to enhance the most salient features. Then, the input features are multiplied by the query weight matrix, followed by global average pooling to utilize global information to adjust channel attention. The resulting feature maps are then flattened to obtain key and query vectors, which are concatenated and input into the two-layer perceptron. The two-layer perceptron learns the importance of different channels. Masking then enhances key information channels, achieving weighted selection of key semantic features. After group normalization and Mish activation, the resulting channel attention weights are multiplied pixel-wise with the value vectors to enhance key channel features.

[0120] Furthermore, the semantic focusing module of this channel mainly includes Euclidean distance measurement and masking operations. First, Euclidean distance is used to calculate the similarity between the key vector and the query vector to capture spatial information. Then, masking operations are used to focus on the target region, and dilated convolution operations are used to expand the receptive field and reduce the loss of details. After layer normalization and Mish activation function processing, the obtained spatial attention weights are multiplied pixel-by-pixel with the value vector to enhance key spatial features.

[0121] In a specific embodiment, the neck network includes a bottom-up first multi-scale fusion path and a top-down second multi-scale fusion path. The first multi-scale fusion path includes three first fusion units connected in sequence, each including an upsampling module, a first splicing layer, and a first feature fusion module connected in sequence. The second multi-scale fusion path includes three second fusion units connected in sequence, each including a convolutional layer, a second splicing layer, and a second feature fusion module connected in sequence. The first feature, second feature, third feature, and fused feature are input into the neck network and pass through the first and second multi-scale fusion paths sequentially to obtain the first attention feature output by the third first fusion unit and the second, third, and fourth attention features output by the three second fusion units, respectively. The first, second, third, and fourth attention features constitute the four attention features.

[0122] The first, second, third, and fourth attention features are input into a multi-level recursive self-aggregation module, undergoing three recursive self-aggregation stages. In the third-to-last recursive self-aggregation stage, the fourth attention feature is first upsampled, then multiplied by the third attention feature with its corresponding weight matrix using Hadamard multiplication, and then summed pixel-by-pixel to obtain the third weighted fusion feature. In the second-to-last recursive self-aggregation stage, the third weighted fusion feature is first upsampled, then multiplied by the second attention feature with its corresponding weight matrix using Hadamard multiplication, and then summed pixel-by-pixel to obtain the second weighted fusion feature. In the last recursive self-aggregation stage, the second weighted fusion feature is first upsampled, then multiplied by the first attention feature with its corresponding weight matrix using Hadamard multiplication, and then summed pixel-by-pixel to obtain the aggregated feature, as shown in the following equation:

[0123] ;

[0124] ;

[0125] ;

[0126] in, This indicates an upsampling operation, which upsamples the image to the same size and number of channels as the previous feature map. This indicates summation pixel by pixel. These represent the first and second weight matrices used in the h-th recursive self-aggregation phase from the end, respectively. The elements in the i-th row and j-th column are respectively and The corresponding expressions are as follows:

[0127] ;

[0128] ;

[0129] and , , and They are respectively represented as The index weight parameter in the i-th row and j-th column. , , These represent the third weighted fusion feature, the second weighted fusion feature, and the aggregated feature, respectively.

[0130] Specifically, in the embodiments of this application, the neck network consists of a bottom-up first multi-scale fusion path and a top-down second multi-scale fusion path. The first multi-scale fusion path includes three first fusion units, and the second multi-scale fusion path includes three second fusion units. Multi-scale fusion is performed from bottom to top and from top to bottom through the first and second multi-scale fusion paths, respectively. In one example, such as... Figure 2 As shown, the first fusion unit includes an upsampling module, a first concatenation layer, and a first feature fusion module connected in sequence. The second fusion unit includes a convolutional layer, a second concatenation layer, and a second feature fusion module connected in sequence. Both the first and second concatenation layers employ a concatenation operation, and both the first and second feature fusion modules use a C2f module. The first attention feature output from the third first fusion unit, along with the second, third, and fourth attention features output from the three second fusion units respectively, are used as the output of the neck network and then input into the multi-order recursive self-aggregation module. This module employs a multi-order recursive self-aggregation mechanism (MORSA), which upsamples the four attention features obtained through the neck network sequentially from the last attention feature and weights and fuses them with the previous attention feature, gradually restoring the resolution to an aggregated feature with the same size and number of channels as the semantic feature with the highest resolution. This results in a feature that combines high resolution and strong semantics. Finally, the aggregated feature is input into the detection head for detection.

[0131] Furthermore, after constructing the above long-distance mouth breathing state detection model, the following steps are used to train it:

[0132] 1. Database retrieval:

[0133] This application collects video data from the monitoring system of a smart kindergarten, including scenes of preschool children napping. It's important to note that the video data collected in this embodiment is legal and compliant. This video data should cover different lighting conditions, children's postures, and background environments to ensure the model's generalization ability in practical applications. Simultaneously, the data collection must ensure sufficient quantity, collecting at least 1000 hours of video data to ensure enough samples for model training and validation.

[0134] 2. Database Processing: The acquired video data is preprocessed to meet the needs of model training. The specific steps are as follows:

[0135] 2.1 Data cleaning and processing: Perform preliminary screening of video data to remove invalid data that cannot be used for training due to equipment failure, lens obstruction, or severe blurring.

[0136] 2.2 Data Augmentation Processing: Data augmentation techniques, such as random cropping, rotation, flipping, brightness adjustment, and contrast adjustment, are used to process video frames to increase data diversity and improve model robustness. Additionally, Gaussian noise can be added to simulate interference in real-world scenarios.

[0137] 2.3 Classification and Annotation Processing: Annotate the preschool children in the video frames, including selecting the area around their mouths and their corresponding mouth breathing states (normal breathing and mouth breathing). The annotation process should undergo quality checks to ensure accuracy.

[0138] 2.4. Dataset Splitting: The processed dataset is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The training set is used for model training, the validation set is used for tuning the model's hyperparameters, and the test set is used for evaluating the model's final performance.

[0139] 3. Use the dataset processed in step 2 to train the long-distance mouth breathing state detection model to obtain the trained long-distance mouth breathing state detection model.

[0140] S2: Acquire video frames containing the preschool children to be detected. Input the video frames into the trained long-range mouth breathing state detection model. First, the model passes through the backbone network. The first feature extraction module, second feature extraction module, third feature extraction module, and SPPF module in the backbone network output the first feature, second feature, third feature, and fourth feature, respectively. The first feature, second feature, third feature, and fourth feature are fused gradient-wise to obtain four semantic features. The four semantic features are input into the long-range cross-receptive field dynamic attention module. First, the model passes through the hole focusing fusion module for adaptive fusion to obtain the focusing fusion feature. The focusing fusion feature passes through the three-path feature cascade collaboration module to extract the reinforcement feature. The first feature, second feature, third feature, and reinforcement feature are input into the neck network to obtain four attention features. The four attention features pass through the multi-level recursive self-aggregation module to obtain the aggregated feature. The aggregated feature passes through the detection head and outputs the corresponding mouth breathing state category probability and target box. Based on the mouth breathing state category probability, it is determined to be mouth breathing or normal breathing.

[0141] Specifically, by deploying the trained long-range mouth breathing detection model from the embodiments of this application, after acquiring video frames containing the preschool child to be detected, the data can be input into the trained long-range mouth breathing detection model for detection and identification. The detection head outputs the corresponding mouth breathing state category probability and target box. By comparing the mouth breathing state category probability with a threshold, it can be determined whether it is mouth breathing or normal breathing. In one example, let the mouth breathing state category probability be... Mouth breathing status is determined using the following formula:

[0142] ;

[0143] in, Breathing through the mouth. This represents the confidence threshold for state determination, which is set to 0.5 in one example of this application.

[0144] The target bounding box includes the bounding box of candidate mouth locations for preschool children, and its corresponding parameters are: ,in, and These represent the center points of the target bounding box. shaft and Axis coordinates and These represent the width and height of the target bounding box, respectively. This target bounding box is selected from all candidate bounding boxes based on the highest confidence level for the mouth category.

[0145] Further reference Figure 6As an implementation of the methods shown in the above figures, this application provides an embodiment of a device for detecting the remote mouth breathing state of preschool children. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0146] This application provides a device for detecting the mouth breathing status of preschool children at a distance, including:

[0147] Model building module 1 is configured to build and train a remote mouth breathing state detection model based on an improved YOLO network, resulting in a trained remote mouth breathing state detection model. The remote mouth breathing state detection model adds a remote cross-receptive field dynamic attention module between the SPPF module in the backbone network of the YOLO network and the upsampling unit in the neck network, and adds a multi-order recursive self-aggregation module between the neck network and the detection head. The remote cross-receptive field dynamic attention module includes a hole focusing fusion module and a three-path feature cascade collaborative module connected in sequence.

[0148] The recognition module 2 is configured to acquire video frames containing the preschool child to be detected. The video frames are input into a trained long-range mouth breathing state detection model. They first pass through the backbone network. The first feature extraction module, the second feature extraction module, the third feature extraction module, and the SPPF module in the backbone network output the first feature, the second feature, the third feature, and the fourth feature, respectively. The first feature, the second feature, the third feature, and the fourth feature are fused gradient-wise to obtain four semantic features. The four semantic features are input into the long-range cross-receptive field dynamic attention module. They first pass through the hole focusing fusion module for adaptive fusion to obtain focusing fusion features. The focusing fusion features are then processed by the three-path feature cascade collaborative module to extract reinforcement features. The first feature, the second feature, the third feature, and the reinforcement features are input into the neck network to obtain four attention features. The four attention features are processed by the multi-level recursive self-aggregation module to obtain aggregate features. The aggregate features pass through the detection head and output the corresponding mouth breathing state category probability and the target box. Based on the mouth breathing state category probability, it is determined to be mouth breathing or normal breathing.

[0149] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 7 As shown, the electronic device of this embodiment includes a processor 701 and a memory 702; wherein the memory 702 is used to store computer execution instructions; and the processor 701 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0150] Alternatively, the memory 702 can be either standalone or integrated with the processor 701.

[0151] When the memory 702 is set up independently, the electronic device also includes a bus 703 for connecting the memory 702 and the processor 701.

[0152] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 701, implement the above method.

[0153] This invention also provides a computer program product, including a computer program that, when executed by a processor 701, implements the above-described method.

[0154] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0155] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0156] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0157] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 701 to execute some steps of the methods of the various embodiments of this application.

[0158] It should be understood that the processor 701 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 701 can be any conventional processor 701. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 701 executing the steps, or as a combination of hardware and software modules within the processor 701 executing the steps.

[0159] The memory 702 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0160] Bus 703 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 703 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 703 in the accompanying drawings of this application is not limited to only one bus 703 or one type of bus 703.

[0161] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0162] An exemplary storage medium is coupled to a processor 701, enabling the processor 701 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 701. The processor 701 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 701 and the storage medium can exist as discrete components in an electronic device or a host device.

[0163] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting long-distance mouth breathing in preschool children, characterized in that, Includes the following steps: A remote mouth breathing state detection model based on an improved YOLO network was constructed and trained to obtain the trained remote mouth breathing state detection model. The remote mouth breathing state detection model adds a remote cross-receptive field dynamic attention module between the SPPF module in the backbone network of the YOLO network and the upsampling unit in the neck network, and adds a multi-order recursive self-aggregation module between the neck network and the detection head. The remote cross-receptive field dynamic attention module includes a hole focusing fusion module and a three-path feature cascade collaborative module connected in sequence. Video frames containing the preschool child to be detected are acquired and input into the trained long-distance mouth breathing state detection model. The data first passes through a backbone network. The first feature extraction module, second feature extraction module, third feature extraction module, and SPPF module in the backbone network output first, second, third, and fourth features, respectively. The first, second, third, and fourth features are then fused gradient-wise to obtain four semantic features. These four semantic features are input into the long-distance cross-receptive field dynamic attention module, where they first undergo adaptive fusion through the hollow focus fusion module to obtain focus fusion features. The hollow focus fusion module specifically includes: The four semantic features are first subjected to max pooling and average pooling operations respectively, and then dilated convolution is used to extract four intermediate features, as shown in the following formula: ； in, Indicates the first A semantic feature, Indicates the first An intermediate feature, , This indicates a global average pooling operation. This indicates a dilated convolution operation. This represents the global max pooling operation; Each intermediate feature is multiplied by its corresponding weight using the Hadamard algorithm, and then concatenated along the channel dimension to obtain the focused fusion feature, as shown in the following equation: ； in, This represents the Hadamard product operation. Indicates the first Each weight, This represents the focused fusion feature; the focused fusion feature is processed by the three-path feature cascade collaborative module to extract enhanced features. The calculation process of the three-path feature cascade collaborative module is shown in the following formula. ； in, and These represent the functions corresponding to the channel semantic focusing module and the spatial semantic focusing module, respectively. The reinforcement features are represented by the first feature, the second feature, the third feature, and the reinforcement features. These features are input into the neck network to obtain four attention features. The four attention features are processed by the multi-level recursive self-aggregation module to obtain aggregated features. The aggregated features are processed by the detection head to output the corresponding mouth breathing state category probability and target box. The mouth breathing state category probability is used to determine whether the breathing is mouth breathing or normal breathing.

2. The method for detecting long-distance mouth breathing in preschool children according to claim 1, characterized in that, The first, second, third, and fourth features are first fused gradient-wise to obtain four semantic features, which specifically include: A semantic resolution hierarchical strategy is adopted for gradient-wise fusion, specifically as follows: The first feature is used as both the input feature of the first-layer gradient fusion and the first semantic feature; the second feature is used as the input feature of the second-layer gradient fusion and is fused with the input feature of the first-layer gradient fusion to obtain the second semantic feature; the third feature is used as the input feature of the third-layer gradient fusion and is fused with the input feature of the second-layer gradient fusion to obtain the third semantic feature; the fourth feature is used as the input feature of the fourth-layer gradient fusion and is fused with the input feature of the third-layer gradient fusion to obtain the fourth semantic feature, as shown in the following formula: ； in, Indicates the first Input features of layer gradient fusion This indicates that a splicing operation is performed at the channel level.

3. The method for detecting long-distance mouth breathing in preschool children according to claim 1, characterized in that, The three-path feature cascaded collaborative module includes a first path, a second path, and a third path. The first path includes a first channel semantic focusing module and a first spatial semantic focusing module connected in sequence. The second path is the path where the focused fusion feature is directly used as the output feature. The third path includes a second spatial semantic focusing module and a second channel semantic focusing module connected in sequence. Both the first and second channel semantic focusing modules are channel semantic focusing modules, and both the first and second spatial semantic focusing modules are spatial semantic focusing modules. The focused fusion feature passes through the first, second, and third paths respectively to obtain the output features of the first path, the second path, and the third path. The output features of the first, second, and third paths are first subjected to a Hadamard product operation, and then to a cube root nonlinear transformation to obtain the enhanced feature.

4. The method for detecting long-distance mouth breathing in preschool children according to claim 3, characterized in that, The calculation process of the channel semantic focusing module specifically includes: The input features of the channel semantic focusing module are multiplied by the key weight matrix, and then subjected to global max pooling to obtain max pooled features. The input features of the channel semantic focusing module are multiplied by the query weight matrix, and then subjected to global average pooling to obtain average pooled features. The max pooled features and average pooled features are flattened to obtain the key vector and query vector of the channel semantic focusing module, as shown in the following formula: ； ； in, This represents the input features of the channel semantic focusing module. Indicates the flattening operation. and These represent the query vector and key vector of the channel semantic focusing module, respectively. and These represent the query weight matrix and key weight matrix of the channel semantic focusing module, respectively; The key vector and query vector of the channel semantic focusing module are concatenated and input into a two-layer perceptron. Then, through masking and dimensionality expansion operations, followed by group normalization and Mish activation function processing, the channel attention weights are obtained, as shown in the following formula: ； in, This represents a two-layer perceptron. Indicates masking operation. This indicates a dimensional expansion operation. This indicates that group normalization is performed along the channel dimension. express Activation function Indicates channel attention weights; The input features of the channel semantic focusing module are multiplied by the value weight matrix to obtain the value vector of the channel semantic focusing module. The channel attention weights are then multiplied pixel-by-pixel by the value vector of the channel semantic focusing module to obtain the output features of the channel semantic focusing module, as shown in the following formula: ； ； in, This represents the value weight matrix of the channel semantic focusing module. This represents the value vector of the channel semantic focusing module. This indicates that pixel-by-pixel multiplication is performed using broadcast operations. This represents the output feature of the channel semantic focusing module; The calculation process of the spatial semantic focusing module specifically includes: The input features of the spatial semantic focusing module are multiplied by the query weight matrix and the key weight matrix, respectively, to obtain the query vector and key vector of the spatial semantic focusing module, as shown in the following formula: ； ； in, This represents the input features of the spatial semantic focusing module. and These represent the query vector and key vector of the spatial semantic focusing module, respectively. and These represent the query weight matrix and the key weight matrix of the spatial semantic focusing module, respectively. The similarity between the query vector and the key vector of the spatial semantic focusing module is calculated using Euclidean distance, and then processed sequentially through masking, dilated convolution, layer normalization, and Mish activation function to obtain the spatial attention weights, as shown in the following equation: ； in, Represents spatial attention weights. This represents the Euclidean distance between the query vector and the key vector of the spatial semantic focusing module. This indicates a hierarchical normalization operation performed in the spatial dimension; The input features of the spatial semantic focusing module are multiplied by the value weight matrix to obtain the value vector of the spatial semantic focusing module; the spatial attention weights are then multiplied pixel-by-pixel by the value vector of the spatial semantic focusing module to obtain the output features of the spatial semantic focusing module, as shown in the following formula: ； ； in, This represents the value vector of the spatial semantic focusing module. This represents the value weight matrix of the spatial semantic focusing module. This represents the output feature of the spatial semantic focusing module.

5. The method for detecting long-distance mouth breathing in preschool children according to claim 1, characterized in that, The neck network includes a bottom-up first multi-scale fusion path and a top-down second multi-scale fusion path. The first multi-scale fusion path includes three sequentially connected first fusion units, each of which includes a sequentially connected upsampling module, a first splicing layer, and a first feature fusion module. The second multi-scale fusion path includes three sequentially connected second fusion units, each of which includes a sequentially connected convolutional layer, a second splicing layer, and a second feature fusion module. The first feature, second feature, third feature, and fused feature are input into the neck network and pass through the first and second multi-scale fusion paths sequentially to obtain a first attention feature output by the third first fusion unit and a second attention feature, a third attention feature, and a fourth attention feature output by the three second fusion units, respectively. The first attention feature, second attention feature, third attention feature, and fourth attention feature constitute four attention features. The first attention feature, second attention feature, third attention feature, and fourth attention feature are input into the multi-order recursive self-aggregation module, undergoing three recursive self-aggregation stages. In the third-to-last recursive self-aggregation stage, the fourth attention feature is first upsampled, and then Hadamard multiplication is performed on it and the third attention feature with their respective weight matrices, followed by pixel-by-pixel summation to obtain the third weighted fusion feature. In the second-to-last recursive self-aggregation stage, the third weighted fusion feature is first upsampled, and then Hadamard multiplication is performed on it and the second attention feature with their respective weight matrices, followed by pixel-by-pixel summation to obtain the second weighted fusion feature. In the first recursive self-aggregation stage, the second weighted fusion feature is first upsampled, and then Hadamard multiplication is performed on the first attention feature and its corresponding weight matrix, followed by pixel-by-pixel summation to obtain the aggregated feature, as shown in the following formula: ； ； ； in, This indicates an upsampling operation, which upsamples the image to the same size and number of channels as the previous feature map. This indicates summation pixel by pixel. These represent the first and second weight matrices used in the h-th recursive self-aggregation phase from the end, respectively. The elements in the i-th row and j-th column are respectively and The corresponding expressions are as follows: ； ； and , , and They are respectively represented as The index weight parameter in the i-th row and j-th column. , , These represent the third weighted fusion feature, the second weighted fusion feature, and the aggregated feature, respectively.

6. A device for detecting the mouth breathing status of preschool children at a distance, characterized in that, The method for detecting the remote mouth breathing status of preschool children according to any one of claims 1-5 includes: The model building module is configured to build and train a remote mouth breathing state detection model based on an improved YOLO network, resulting in a trained remote mouth breathing state detection model. The remote mouth breathing state detection model adds a remote cross-receptive field dynamic attention module between the SPPF module in the backbone network of the YOLO network and the upsampling unit in the neck network, and adds a multi-order recursive self-aggregation module between the neck network and the detection head. The remote cross-receptive field dynamic attention module includes a hole focusing fusion module and a three-path feature cascade collaborative module connected sequentially. The recognition module is configured to acquire video frames containing the preschool child to be detected, input the video frames into the trained long-distance mouth breathing state detection model, first pass through the backbone network, the first feature extraction module, the second feature extraction module, the third feature extraction module and the SPPF module in the backbone network output the first feature, the second feature, the third feature and the fourth feature respectively, the first feature, the second feature, the third feature and the fourth feature are fused gradientwise to obtain four semantic features, the four semantic features are input into the long-distance cross-receptive field dynamic attention module, first pass through the hole focusing fusion module for adaptive fusion to obtain focusing fusion features, the focusing fusion features are then passed through the three-path feature cascade collaborative module to extract reinforcement features, the first feature, the second feature, the third feature and reinforcement features are input into the neck network to obtain four attention features, the four attention features are passed through the multi-order recursive self-aggregation module to obtain aggregate features, the aggregate features are passed through the detection head to output the corresponding mouth breathing state category probability and target box, and the mouth breathing state category probability is determined to be mouth breathing or normal breathing.

7. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images