Home ventilator anomaly detection method, device and electronic equipment

Abstract

Embodiments of the present disclosure disclose a home ventilator anomaly detection method, device and electronic equipment. A specific embodiment of the method comprises: performing parameter detection on a target ventilator to obtain a first detection information set, and performing shooting calibration on a camera; issuing a ventilator wearing prompt information; controlling the calibrated camera to shoot a current scene video, and obtaining a current running data set of the target ventilator; performing scene dynamic detection on the current scene video to generate a dynamic detection information set; performing ventilator anomaly detection according to the dynamic detection information set and the current running data set to generate a second detection information set; in response to the presence of second detection information in the second detection information set that satisfies a preset anomaly condition, generating a first anomaly description information set, and sending a ventilator anomaly warning. The embodiment can timely issue a ventilator anomaly warning, thereby reducing the probability of patient danger.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, particularly to the field of data detection technology and image recognition technology, and specifically to a method, apparatus and electronic device for detecting abnormalities in home ventilators. Background Technology

[0002] A ventilator is a device that assists or replaces a patient's breathing. If a ventilator malfunctions, it can easily lead to danger. Currently, ventilator malfunction detection often involves professional nurses regularly checking ventilator data and verifying the integrity of connections to determine if there are any abnormalities. However, this method requires nurses to possess specialized ventilator operation skills. In a home setting, due to the complexity of family members and the lack of such skills, it is difficult to effectively monitor the ventilator's status. Furthermore, home activities (such as accidental contact with the ventilator tubing, mask, or circuitry) can easily lead to ventilator malfunctions or misuse (e.g., tubing detachment, mask dislodgement, circuit breakage), thus increasing the probability of danger to the patient. Summary of the Invention

[0003] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0004] Some embodiments of this disclosure provide methods, apparatus, and electronic devices for detecting abnormalities in home ventilators to address the technical problems mentioned in the background section above.

[0005] In a first aspect, some embodiments of this disclosure provide a method for detecting anomalies in a home ventilator. The method includes: responding to the startup of a target ventilator and completion of parameter configuration, performing parameter detection on the target ventilator to obtain a first detection information set, and calibrating a camera for shooting, wherein the target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located; issuing a ventilator wearing prompt message in response to the first detection information set satisfying preset parameter detection conditions; and controlling the calibrated camera to capture a video of the current scene in response to completion of wearing and camera calibration, and obtaining a current operating dataset of the target ventilator, wherein the current operating dataset is the number of ventilator devices corresponding to the current scene video during the use of the target ventilator. The following steps are taken: Dynamic scene detection is performed on the current scene video to generate a dynamic detection information set; ventilator anomaly detection is performed based on the dynamic detection information set and the current operating dataset to generate a second detection information set. The second detection information set is a detection result representing whether the target ventilator is abnormal, generated by combining the dynamic behavioral features corresponding to the current scene video and the data features corresponding to the ventilator operating data. The second detection information includes the abnormal time period and abnormal association label of the target ventilator; in response to the existence of second detection information in the second detection information set that meets preset abnormal conditions, a first abnormal description information set is generated based on each second detection information that meets the preset abnormal conditions, and a ventilator anomaly warning is sent based on the first abnormal description information set.

[0006] Secondly, some embodiments of this disclosure provide a home ventilator anomaly detection device, the device comprising: a parameter detection unit configured to, in response to the startup of a target ventilator and the completion of parameter configuration, perform parameter detection on the target ventilator to obtain a first detection information set, and perform shooting calibration on a camera, wherein the target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located; a sending unit configured to, in response to the first detection information set satisfying preset parameter detection conditions, issue a ventilator wearing prompt message; and a shooting and acquisition unit configured to, in response to the completion of wearing and camera calibration, control the calibrated camera to capture a video of the current scene, and acquire a current operating dataset of the target ventilator, wherein the current operating dataset is ventilator equipment data corresponding to the current scene video during the use of the target ventilator; The scene dynamic detection unit is configured to perform scene dynamic detection on the current scene video to generate a dynamic detection information set; the ventilator anomaly detection unit is configured to perform ventilator anomaly detection based on the dynamic detection information set and the current running dataset to generate a second detection information set, wherein the second detection information set is a detection result representing whether the target ventilator is abnormal, generated by combining the dynamic behavior features corresponding to the current scene video and the data features corresponding to the ventilator running data, and the second detection information includes the abnormal time period and abnormal association label of the target ventilator; the generation unit is configured to, in response to the existence of second detection information that meets preset abnormal conditions in the second detection information set, generate a first anomaly description information set based on each second detection information that meets the preset abnormal conditions, and send a ventilator anomaly warning based on the first anomaly description information set.

[0007] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0008] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0009] The above-described embodiments of this disclosure have the following beneficial effects: The home ventilator anomaly detection method of some embodiments of this disclosure can be used to detect ventilator anomalies or the impact of environmental factors on the ventilator in real time during home ventilator use scenarios, thereby issuing timely anomaly warnings and enabling the ventilator to provide timely assistance to the patient, reducing the probability of patient danger. Specifically, the home ventilator anomaly detection method of some embodiments of this disclosure firstly, in response to the target ventilator being started and its parameters being configured, performs parameter detection on the target ventilator to obtain a first detection information set, and performs camera shooting calibration, wherein the target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located. Secondly, in response to the first detection information set meeting preset parameter detection conditions, a ventilator wearing prompt message is issued. Then, in response to the completion of wearing and camera calibration, the calibrated camera is controlled to capture video of the current scene and obtain the current operating dataset of the target ventilator, wherein the current operating dataset is the ventilator equipment data during the use of the target ventilator corresponding to the current scene video. In practice, home settings often present numerous unexpected factors related to ventilator use (e.g., pet activity, children's activities). Compared to professional medical settings, home settings require not only monitoring the ventilator's own data but also understanding the impact of these unexpected factors on the ventilator. This allows for timely warnings, reducing the time spent investigating ventilator anomalies and thus mitigating the risk to the patient's life. This application introduces a camera into the home setting to capture video of the current ventilator usage scenario. Next, dynamic scene detection is performed on the video to generate a dynamic detection information set. Then, based on the dynamic detection information set and the current operating dataset, ventilator anomaly detection is performed to generate a second detection information set. This second detection information set combines the dynamic behavioral features corresponding to the current scene video and the data features corresponding to the ventilator's operating data, representing the detection results indicating whether the target ventilator is abnormal. The second detection information includes the abnormal time period and anomaly association label of the target ventilator. By performing dynamic scene detection on the current scene video, dynamic behavioral features present in the home ventilator usage scenario can be identified, facilitating the determination of whether there is a correlation between dynamic behavioral features and the ventilator. Finally, in response to the existence of second detection information that meets the preset abnormal conditions in the second detection information set, a first abnormal description information set is generated based on each second detection information that meets the preset abnormal conditions, and a ventilator abnormality warning is sent based on the first abnormal description information set.Therefore, it is possible not only to detect abnormal data of the ventilator itself in a timely manner, but also to identify the dynamic behavioral characteristics that lead to ventilator abnormalities in home activities. This allows for timely provision of specific circumstances leading to ventilator abnormalities, thereby reducing the time required for ventilator abnormality investigation and enabling timely issuance of ventilator abnormality warnings, thus lowering the probability of danger to patients. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0011] Figure 1 This is a flowchart of some embodiments of the home ventilator abnormality detection method according to the present disclosure; Figure 2 This is a schematic diagram of a dynamic feature extraction model; Figure 3 This is a time alignment diagram; Figure 4 This is a schematic diagram of the structure of some embodiments of the home ventilator abnormality detection device according to the present disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0014] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0015] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0017] Before performing any of the operations involving the collection, storage, and use of user personal information (such as ventilator data and scene videos in home ventilator scenarios) disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0018] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] Figure 1 A flow 100 of some embodiments of a home ventilator malfunction detection method according to the present disclosure is shown. This home ventilator malfunction detection method includes the following steps: Step 101: In response to the start of the target ventilator, perform parameter detection on the target ventilator to obtain the first detection information set, and perform shooting calibration on the camera.

[0020] In some embodiments, the execution entity (e.g., a computing device) of the home ventilator anomaly detection method may, in response to the target ventilator being started and its parameters being configured, perform parameter detection on the target ventilator to obtain a first detection information set, and perform image calibration on a camera, wherein the target ventilator is a home ventilator, and the camera is used to capture images of the environment in which the target ventilator is located. The first detection information characterizes the result of detecting the setting parameters of the target ventilator.

[0021] In practice, "when the target ventilator is powered on and running" signifies that the target ventilator has started. "Parameter configuration complete" signifies that the operator has completed the parameter configuration of the target ventilator. Here, the parameters corresponding to parameter configuration can include, but are not limited to, at least one of the following: tidal volume, respiratory rate, respiratory ratio, expiratory pressure, inspiratory pressure, and inspiratory flow rate. Secondly, each parameter configured on the target ventilator can be checked using a preset ventilator configuration constraint table to obtain the first detection information. Specifically, the ventilator configuration constraint table includes the constraint relationships established for each ventilator configuration parameter of the target ventilator. For example, the constraint relationships in the ventilator configuration constraint table are: minute ventilation = tidal volume × respiratory rate; airway pressure is less than the preset maximum pressure value. In addition, each piece of first detection information corresponds to a ventilator parameter that has been configured. The first detection information can include a parameter detection identifier indicating whether the parameter configuration meets the constraint relationships.

[0022] In practical applications, the target ventilator is often placed in a relatively fixed location during use (e.g., on a table). Therefore, to better capture video of potential abnormal contact with the target ventilator in a home setting, the camera can be fixed to a wall within a preset height range on the side of the target ventilator, or to a vertically mounted pole. This allows the camera to capture the operator's handling of the target ventilator and its connected devices (e.g., tubing, mask) from the side.

[0023] Secondly, in a home setting, the position of the target ventilator, the patient's position, and the camera's shooting angle can all vary slightly. Therefore, to prevent the environment where the target ventilator is located from exceeding the camera's shooting range, the camera's shooting (angle) can be calibrated. The aforementioned executing entity can control the camera to capture an initial scene image. Here, the initial scene image can be an image captured by the camera at the current moment. Then, the ventilator's center point is identified from the initial scene image. Subsequently, if the ventilator's center point is not within a preset range, the camera's shooting angle is adjusted based on the ventilator's center point to ensure it falls within the preset range. Here, the preset range can be a coordinate region in an image pre-marked based on the relative positions of the camera and the target ventilator. This can be used to determine if the camera's shooting angle has deviated, allowing for timely angle calibration.

[0024] The ventilator center point identification can be performed through the following steps: First, the location region of the ventilator image in the initial scene image is determined by template matching (this location region represents the static environment of the target ventilator). The ventilator image is a pre-set image captured by a calibrated camera containing only the ventilator. Then, the binary image of the ventilator image is extracted. Next, the maximum connected component of the pixels matching the binary image within this location region can be extracted to calculate the corresponding minimum bounding rectangle. Finally, the center point of the minimum bounding rectangle can be taken as the ventilator center point.

[0025] It should be noted that the aforementioned computing devices can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed on the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0026] In some optional implementations of certain embodiments, the aforementioned execution entity performs shooting calibration on the camera, including: Step S1: Control the camera to capture an initial angle image, and based on the initial angle image, perform the following first processing step: Step S11: Based on a preset static feature extraction model, static scene recognition is performed on the initial angle image to generate static scene information. This static scene information includes: static scene regions and corresponding static label groups, representing the static environment of the target ventilator. The static scene region is the area where the target ventilator and the user are fixed in position. The static region label is a feature label within the static scene region associated with the target ventilator. Here, the static region label can represent features such as the ventilator position, ventilator circuitry, ventilator tubing, ventilator mask, and patient head position in the initial angle image. The static feature extraction model includes: a backbone network, an aggregation network, a detection module, and a static post-processing module. Here, the ventilator position label represents the location of the target ventilator in the initial angle image. The ventilator circuitry label represents the identified circuitry of the target ventilator. The ventilator tubing label represents the identified gas delivery tubing of the ventilator. The user position label represents the head position region of the user (i.e., the position of the user in the initial angle image when wearing the ventilator mask). The initial angle image is a scene image captured before camera calibration is completed.

[0027] Specifically, the backbone network includes: 1 × 1 convolutional layers (e.g., input scale 640 × 640 × 3), feature extraction branch 1, feature extraction branch 2, residual connections and output fusion layers, and SPPF (Spatial Pyramid Pooling-Fast) fast spatial pyramid pooling layers. Feature extraction branch 1 extracts detailed features through three sequentially connected 3 × 3 convolutional layers to adapt to the texture features of small targets such as tubing and wires of the target ventilator. Here, each convolutional layer is padded after convolution to ensure the feature map scale remains unchanged. Feature extraction branch 2 can retain the original features without processing to avoid feature loss. Residual connections are used to fuse the outputs of feature extraction branch 1 and feature extraction branch 2 (e.g., element-wise addition) to obtain a multi-scale feature map (e.g., scale 160 × 160 × 128) to alleviate the gradient vanishing problem in deep networks and ensure that features of both large targets (target ventilator) and smaller targets (wires, tubing) can be effectively extracted. Finally, the multi-scale feature maps can be input into the SPPF fast spatial pyramid pooling layer to output a global feature map (e.g., at a scale of 160 × 160 × 256). Here, the SPPF fast spatial pyramid pooling layer can be used to enhance global features.

[0028] The aggregation network, acting as the neck network, is a bidirectional feature pyramid network structure composed of a top-down structure and a bottom-up structure. This structure enhances feature maps at different scales and strengthens edge features of small objects. The top-down structure includes four consecutive 1 × 1 convolutional layers. The bottom-up structure includes four consecutive 3 × 3 convolutional layers. The output of each layer in the top-down structure serves as the input to the corresponding layer in the bottom-up structure.

[0029] The detection module is a decoupled head, comprising a classification branch, a regression branch, and a segmentation branch. The classification branch outputs the label of the detected target. Both the classification and regression branches employ a multilayer perceptron. The segmentation branch consists of two 3 × 3 convolutional layers, a transposed convolutional layer, and an activation layer.

[0030] The static post-processing module is used to perform non-maximum suppression, mask thresholding, and edge coordinate extraction on the output results of the three branches in the detection module. Specifically, the classification branch: for the duplicate bounding boxes output by the detection module, they are first sorted from high to low confidence. The bounding box with the highest confidence is used as the reference box. The intersection-union ratio (IU) of the remaining bounding boxes with the reference box is calculated. When the IU is ≥ 0.5, it indicates that the bounding box is duplicated. The bounding box is then removed to avoid duplicate detection. Finally, the unique bounding box corresponding to each detected target is retained. Then, the regression branch: it traverses the pixel-level mask output by the detection module. Pixels with a mask value ≥ 0.5 are determined as target pixels (1), and pixels with a mask value < 0.5 are determined as background pixels (0). At the same time, small connected components with an area < 10 pixels are removed to avoid noise interference. Finally, the branch is segmented: based on the filtered binary mask, the Canny edge detection algorithm is used. First, the mask is grayscaled, and low threshold (50) and high threshold (150) are set to extract edge pixels. Then, the contour detection algorithm is used to traverse the edge pixels and record the pixel coordinates. Finally, the continuous and complete edge coordinates corresponding to the static area of ​​the scene are output.

[0031] Step S12: Generate an angle offset vector based on the scene static information. This angle offset vector is a two-dimensional vector representing the camera calibration direction. Here, if none of the static areas in the scene static information contain image boundary coordinates (except for the static area corresponding to the ventilator circuit label), an empty vector is generated as the angle offset vector. This indicates that the camera does not require shooting calibration. If the scene static information contains static areas that contain image boundary coordinates (except for the static area corresponding to the ventilator circuit label), it is determined that the ventilator device has not been fully captured. Therefore, an angle offset vector can be generated based on the contained image boundary coordinates.

[0032] Specifically, an angle offset vector can be generated based on the image boundary coordinates. Secondly, to improve camera calibration efficiency, the value of the angle offset vector can be adjusted according to the number of image boundary coordinates contained in the static area of ​​the scene. For example, if the image boundary coordinates are the coordinates of the upper boundary of the initial angle image, then the angle offset vector is a vector pointing vertically to the upper boundary. For the number of image boundary coordinates contained in the static area of ​​the scene, the corresponding vector values ​​can be selected as the vector values ​​of the angle offset vector according to a pre-defined interval between the number of image boundary coordinates and the vector values. Therefore, for static areas of the scene with a large number of image boundary coordinates, a larger angle offset vector can be set, thereby increasing the camera's adjustment angle and improving calibration efficiency.

[0033] Step S13: In response to the angle offset vector satisfying a preset offset condition, the camera is identified as the calibrated camera. The preset offset condition is that the angle offset vector is empty. An empty angle offset indicates that the camera calibration is complete.

[0034] Additionally, during calibration, identified static areas of the scene can be used as calibration constraints to prevent static areas in other directions from exceeding the image range after the camera moves in one direction. If relative directions exist simultaneously (e.g., upper and lower boundaries), meaning a static area contains the upper boundary coordinates while another static area contains the lower boundary coordinates, the camera is controlled to expand its shooting range.

[0035] Step S2: In response to the angle offset vector not meeting the preset offset condition, the camera is controlled to adjust its shooting angle according to the angle offset vector to obtain the current angle camera, and the current angle camera is controlled to capture the current angle image as the initial angle image. The first processing step described above is then executed again. Here, the camera can be controlled to move by the corresponding angle according to the direction of the angle offset vector. Secondly, after the movement, the first processing step can be used to determine again whether the camera calibration is complete. Therefore, camera shooting calibration can not only be used to calibrate the camera shooting angle, but also to simultaneously determine the static environment of the target ventilator, so as to determine the position area of ​​the target ventilator in the current scene, in order to determine the correlation between the dynamic features of the scene and the static environment.

[0036] Optionally, when processing images or videos captured by a camera, if the executing entity is a local device, encryption is not required. If the executing entity is a cloud device, encryption can be performed according to user instructions.

[0037] Step 102: In response to the first detection information set meeting the preset parameter detection conditions, a ventilator wearing prompt message is issued.

[0038] In some embodiments, the aforementioned executing entity may issue a ventilator wearing prompt message in response to the first detection information set satisfying preset parameter detection conditions. The preset parameter detection conditions may involve each piece of first detection information in the first detection information set including a parameter detection identifier indicating that the parameter configuration satisfies a constraint relationship. This indicates that the operator's parameter settings for the target ventilator comply with the parameter constraints. Therefore, a ventilator wearing prompt message can be issued. The ventilator wearing prompt message is a prompt message instructing the operator to put a ventilator mask on the patient.

[0039] Step 103: In response to the completion of wearing and camera calibration, control the calibrated camera to capture the current scene video and obtain the current operating dataset of the target ventilator.

[0040] In some embodiments, the aforementioned execution entity can, in response to completion of wearing and camera calibration, control the calibrated camera via wired or wireless means to capture video of the current scene and acquire the current operating dataset of the target ventilator. The current operating dataset is the ventilator device data during the use of the target ventilator, corresponding to the current scene video. Secondly, after confirming that the target ventilator is configured correctly, its operating data can be acquired as the current operating dataset. The current operating data corresponds one-to-one with the configuration parameters. The current scene video is a video containing the environment of the target ventilator and all its associated devices. In practice, capturing the current scene video can be used in home settings to identify objects entering the shooting area, in order to determine whether the object's behavior is related to ventilator malfunctions.

[0041] It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (Ultra Wide Band) connections, and other currently known or future wireless connection methods.

[0042] Step 104: Perform scene dynamic detection on the current scene video to generate a dynamic detection information set.

[0043] In some embodiments, the aforementioned executing entity may perform scene dynamic detection on the current scene video to generate a dynamic detection information set. This dynamic detection information set characterizes the dynamic behavioral features of the target ventilator within its environment.

[0044] First, the YOLO-v5 (You Only Look Once – version 5, object detection network model) network model can be used to detect moving targets in the current scene video, obtaining a dynamic detection information set. The dynamic detection information corresponds one-to-one with the dynamic target. The dynamic detection information includes the detection region of the dynamic target and its corresponding dynamic behavior feature label. The dynamic behavior feature label represents the behavior action of the dynamic target.

[0045] As an example, if the dynamic target is "children," the corresponding dynamic behavioral feature label is "touch." Another example is if the dynamic target is "pets," and the corresponding dynamic behavioral feature label is "biting." Yet another example is if the dynamic target is "patients," and the corresponding dynamic behavioral feature is "touch."

[0046] In practice, in home settings, when users are using ventilators, especially when family members include children and / or pets, they are easily exposed to the ventilator, potentially leading to ventilator malfunctions. Therefore, identifying external factors (i.e., dynamic environmental factors) is crucial for ventilator use in home settings. Furthermore, in complex home environments (i.e., with numerous dynamic targets), directly identifying the specific behavior of each dynamic target would be extremely difficult. Therefore, this application identifies only the dynamic target's own behavioral state for dynamic targets, and then combines this with the interaction between the dynamic target and static areas of the scene to determine if there is a correlation, thereby determining whether it is causing ventilator malfunctions. This avoids overly complex identification schemes and, through a generalized detection mechanism, reduces identification errors, thus improving the accuracy of ventilator malfunction detection.

[0047] Optionally, the dynamic detection information includes: dynamic target location information, dynamic target association labels, and association label confidence levels. Dynamic target location information refers to the location information of the moving target detected within the environment where the target ventilator is located. Dynamic target association labels characterize the association relationship between the dynamic target and the static environment. Association label confidence levels characterize the degree of association between the dynamic target and the static environment.

[0048] In some optional implementations of certain embodiments, the execution entity performs scene dynamic detection on the current scene video to generate a dynamic detection information set, including: Step S1: Determine the static scene information corresponding to the calibrated camera, and use it as the current scene static information. Specifically, the static scene information generated by the calibrated camera in the final round of angle calibration can be used as the current scene information. The angle offset corresponding to the static scene information generated by the calibrated camera in the final round of angle calibration must satisfy a preset offset condition.

[0049] Step S2 involves extracting frames from the current scene video to obtain a current scene image sequence. Frame extraction reduces computation and improves detection efficiency. Furthermore, frame extraction can be performed at a preset frequency (e.g., 10 frames / second) to obtain the current scene image sequence. The current scene images in the current scene image sequence are arranged in chronological order.

[0050] Step S3: Group the current scene image sequence to obtain a set of grouped scene image sequences. Specifically, the current scene image sequence can be grouped according to the time unit "second" to obtain a set of grouped scene image sequences. Here, each grouped scene image sequence contains multiple frames of the current scene image within a unit of time (e.g., 1 second).

[0051] Step S4: For each grouped scene image sequence in the above grouped scene image sequence set, generate dynamic detection information in the dynamic detection information set through the following steps: Step S41: Based on the aforementioned static information of the current scene and the preset dynamic feature extraction model, perform dynamic scene recognition on the aforementioned grouped scene image sequences to generate dynamic detection information, including dynamic target location information, dynamic target association labels, and association label confidence scores. The dynamic detection information characterizes the dynamic behavioral features of dynamic targets interacting with the static environment in the current scene video. The dynamic feature extraction model includes: a target detection branch, a target tracking branch, a spatiotemporal feature extraction network, and a detection head. The target detection branch reuses the network structure of the static feature extraction model. Furthermore, the input to the dynamic feature extraction model can include the current scene static information and the grouped scene image sequences.

[0052] Here, as Figure 2 As shown, by including a static feature extraction model in the network structure of the dynamic feature extraction model, not only is it possible to avoid repeatedly building the feature extraction network and reduce computational load, but it can also ensure the consistency between the extracted dynamic features and static features, avoiding misjudgments in dynamic associations caused by feature mismatch. Therefore, the object detection branch can extract features from each grouped scene image in the grouped scene image sequence to output the corresponding dynamic target location label, dynamic target detection box, and dynamic target detection region. Secondly, the object detection branch, through the static post-processing module, can calculate the intersection-union ratio (IUU) of targets in adjacent frames based on the detection results of 10 grouped scene images, associate the consecutive frame positions of the same dynamic target, and eliminate duplicate detection results across frames, ensuring the continuity of target tracking. In addition, the object detection branch also includes a matching module, which can match the dynamic target detection box of each frame with the target mask corresponding to the current scene static information (a successful match is determined when the overlap between the bounding box and the static mask is ≥ 0.6), calibrating dynamic detection deviations and preventing the dynamic target from becoming disconnected from the static baseline. Finally, the effective detection results and matched features of the 10 grouped scene images are fused to output the dynamic target detection results corresponding to multiple frames of images. The dynamic target detection results include the location region of the dynamic target in each frame of the grouped scene image and the corresponding confidence score.

[0053] The target tracking branch splits the single-channel mask (scale 640 × 640) output by the static feature extraction model according to the target category, generating independent mask region tensors for each target (e.g., pipeline mask scale 640 × 640 × 1, ventilator mask scale 640 × 640 × 1). Then, for each independent target mask, the minimum bounding rectangle is extracted through contour detection as the static reference region. Next, the static target label is bound to the reference region and category label. Invalid targets are eliminated by combining intersection-union ratio (IU) and threshold comparison, and the Hungarian algorithm is used to solve for the optimal match to avoid confusion in multi-target cross-matching, retain the tracking identifier, and obtain the detection features of the dynamic target (including tracking identifier, dynamic bounding box, associated static region, and region static label). Here, the bounding box of the dynamic target can be used as the dynamic target location information.

[0054] The spatiotemporal feature extraction network is a long short-term memory network used to extract spatiotemporal correlation features from consecutive frames and output a spatiotemporal feature map. The spatiotemporal feature map contains the region correlation feature vectors (128 dimensions) of each dynamic target in each image.

[0055] The detection head is a lightweight multilayer perceptron, comprising an input layer, hidden layers, and an output layer. The input layer takes 128-dimensional region association features plus 5-dimensional static label one-hot encoding (ventilator position, ventilator tubing, ventilator mask, ventilator circuitry, and patient head position) as input. The hidden layer consists of one fully connected layer (64 neurons), an activation function, and a normalization layer. The output layer consists of a 4-dimensional fully connected layer and an activation function, outputting the probability distribution of four types of association labels. The maximum probability is taken as the association label, and the corresponding probability is taken as the confidence level. This yields the dynamic target association labels and their corresponding confidence levels in the dynamic detection information. Here, the 5-dimensional static label one-hot encoding is generated by one-hot encoding each region static label in the region static label group of the scene static information.

[0056] As an example, the detection label for a dynamic target (e.g., a child) is “001”, and the associated label for the dynamic target could be “001-touch-gas delivery line”.

[0057] In practice, firstly, by reusing the model structure of the static feature extraction model, the costs of model training and deployment are reduced, while ensuring feature consistency between static localization and dynamic recognition, avoiding the problem of "accurate static localization but disconnected dynamic recognition." Secondly, by reusing the output of the static feature extraction model, the dynamic feature extraction model can extract features, track targets, and recognize behaviors only within the region corresponding to the static mask. Simultaneously, reusing static edge coordinates as a benchmark for dynamic position changes and association judgments significantly reduces the computational load of the dynamic model, avoids background interference, and ensures that dynamic analysis focuses on the core target. Finally, by associating dynamic targets with static regions, dynamic target association labels and association label confidence scores are generated, simplifying complex scenarios and facilitating the tracing and subsequent analysis of dynamic events.

[0058] Step 105: Perform ventilator anomaly detection based on the dynamic detection information set and the current running dataset to generate a second detection information set.

[0059] In some embodiments, the aforementioned execution entity may perform ventilator anomaly detection based on the aforementioned dynamic detection information set and the aforementioned current running dataset to generate a second detection information set. The second detection information set is a detection result characterizing whether the target ventilator is abnormal, generated by combining the dynamic behavioral features corresponding to the current scene video and the data features corresponding to the ventilator's running data. The second detection information includes the abnormal time period of the target ventilator and anomaly association tags.

[0060] First, parameter detection determines whether the current operating data is abnormal. If abnormality is found, dynamic detection information matching the parameter attributes of the current operating data is selected from the dynamic detection information set as abnormality detection information. Then, second detection information is generated based on the abnormality detection information. Specifically, the dynamic detection information with the highest confidence level of the association label can be selected from the dynamic detection information set as abnormality detection information. Secondly, the dynamic target association label of the abnormality detection information can be used as the abnormal association label in the second detection information, and the time period in the current operating data where abnormality occurs is determined as the abnormal time period including the target ventilator in the second detection information.

[0061] In some optional implementations of certain embodiments, the execution entity performs target ventilator anomaly detection based on the aforementioned dynamic detection information set and the aforementioned current running dataset to generate a second detection information set, including: Step S1: Time-align the current running dataset with the dynamic detection information set to obtain the aligned running dataset.

[0062] See here. Figure 3The current running data and dynamic detection information can be associated on the same time axis t according to timestamps to obtain an aligned running dataset. Two adjacent aligned running data points in the aligned running dataset correspond to the same time period as N adjacent dynamic detection information points in the dynamic detection information set. Here, time alignment can determine the temporal relationship between the target ventilator's running data and the dynamic detection information identified by image recognition, thus facilitating the analysis and tracing of abnormal situations in the target ventilator.

[0063] As an example, the time interval between two adjacent currently running data points is 1 second. The time interval between two adjacent dynamic detection information points is 2 seconds. Therefore, through time alignment, two adjacent aligned running data points can correspond to the same time interval as three adjacent dynamic detection information points.

[0064] Step S2: Select dynamic detection information that meets the preset reliability conditions from the above dynamic detection information set as candidate detection information to obtain a candidate detection information group. The preset reliability condition is that the confidence level of the associated label is greater than a preset reliability threshold. Furthermore, the candidate detection information represents the initially screened dynamic detection information with high confidence.

[0065] Step S3: Obtain the ventilator operation data corresponding to the aligned running dataset described above, thus obtaining the ventilator operation dataset. The ventilator operation data is the control data input to the ventilator via the ventilator control panel or remote control.

[0066] Step S4: Based on the ventilator operation dataset, perform data detection on the aligned operation dataset to generate an initial anomaly information set. This initial anomaly information includes an anomaly type identifier representing the type of ventilator anomaly. The initial anomaly information indicates that the target ventilator has data anomalies. First, parameter detection can determine whether there are anomalies in the aligned operation data. If anomalies exist, determine whether the timestamp of the corresponding aligned operation data is before the timestamp of the corresponding ventilator operation data. If the timestamp of the corresponding aligned operation data is before the timestamp of the corresponding ventilator operation data, an anomaly type identifier representing the ventilator data anomaly is generated as the initial anomaly information. If the timestamp of the corresponding aligned operation data is after the timestamp of the corresponding ventilator operation data, or at the same time point, an anomaly type identifier representing the ventilator operation anomaly is generated as the initial anomaly information. Here, the anomaly type identifier can represent ventilator data anomalies caused by abnormal ventilator operation.

[0067] As an example, abnormal ventilator data types may include: abnormal temperature heating data, abnormal pressure data, abnormal tidal volume data, etc.

[0068] Step S5: Based on a preset abnormal delay response table, determine the abnormal response duration corresponding to each initial abnormal information in the initial abnormal information set. The abnormal delay response table can be a pre-established abnormal delay response duration data table for each abnormal attribute. The abnormal delay response duration represents the delay between the cause of the ventilator data abnormality and the detection of the ventilator abnormality. Secondly, the abnormal response duration of the parameter attribute corresponding to the initial abnormal information can be determined using parameter attribute indexing.

[0069] Step S6: For each initial anomaly in the initial anomaly information set, perform the following second processing step: Step S61: Based on the abnormal response duration corresponding to the initial abnormal information, select candidate detection information that matches the initial abnormal information from the candidate detection information group, and use it as target detection information to obtain the target detection information group. The time point corresponding to the target detection information is within the time period corresponding to the abnormal response duration, and the dynamic target association label included in the target detection information matches the abnormal type identifier included in the initial abnormal information. The target detection information represents the detection information that is scene-related to the ventilator equipment within the abnormal response duration of the detected abnormal data. Here, by introducing the abnormal response duration, the screening scope can be narrowed. The dynamic target association label matches the abnormal type identifier included in the initial abnormal information, indicating that the action behavior corresponding to the dynamic target association label is related to the ventilator abnormality corresponding to the abnormal type identifier.

[0070] As an example, the dynamic target association label is "001-Touch-Gas Delivery Pipeline". The anomaly type identifier represents "Gas Flow Anomaly". And the gas flow anomaly is associated with the gas delivery pipeline. Therefore, it can be determined that the dynamic target association label matches the anomaly type identifier included in the initial anomaly information.

[0071] Step S62: Generate second detection information based on the above target detection information group and the above initial anomaly information.

[0072] Optionally, based on the aforementioned target detection information group and the aforementioned initial anomaly information, second detection information is generated, including: Step S1: In response to the target detection information group satisfying the first quantity condition, second detection information is generated. The first quantity condition is that the number of target detection information items in the target detection information group is 0. Satisfying the first quantity condition indicates that the ventilator abnormality is unrelated to the dynamic target. The second detection information also includes abnormal attributes and corresponding abnormal identifiers. Abnormal attributes are attributes of configuration parameters that exhibit abnormalities. Abnormal identifiers represent the abnormal type of the abnormal attribute. For example, if the abnormal parameter is "tidal volume," the corresponding abnormal type is "tidal volume exceeding the limit." Furthermore, the time period corresponding to the initial abnormal information can be used as the abnormal time period in the second detection information. The time period corresponding to the initial abnormal information is the time period during which the ventilator data shows abnormalities. Here, the abnormal association label in the second detection information can be empty.

[0073] Step S2: In response to the target detection information group satisfying the second quantity condition, second detection information is generated based on the target detection information group. The second quantity condition is that the number of target detection information in the target detection information group is 1. Here, the target detection information group satisfying the second quantity condition indicates that only one dynamic target came into contact with the target ventilator when the target ventilator was detected to be abnormal. Therefore, the dynamic target association tag corresponding to the target detection information can be used as the abnormal association tag, and the time period corresponding to the target detection information can be used as the abnormal time period. The time period corresponding to the target detection information is the time period in the current scene video when the dynamic target is in contact with the target ventilator. Specifically, the time period between each time point of the corresponding dynamic target association tag within the abnormal response duration can be determined as the abnormal time period.

[0074] Step S3: In response to the target detection information group satisfying the third quantity condition, a correlation degree sequence is generated based on the initial abnormal information and the target detection information group. The third quantity condition is that the number of target detection information items in the target detection information group is greater than 1. Here, a number greater than 1 indicates that multiple dynamic targets have come into contact with the target ventilator within the abnormal response time. Therefore, firstly, the contact location of the dynamic target with the target ventilator (e.g., the contact location is the gas delivery tubing) can be determined based on the dynamic target association tag corresponding to each target detection information item. Then, according to a preset abnormal association attribute sequence, the abnormal association number corresponding to the contact location is selected. Here, the abnormal association attribute sequence is a pre-established correlation degree for other attributes or devices that caused the abnormal attribute to occur. The abnormal association attributes in the abnormal association attribute sequence can be arranged in ascending order of importance from low to high. Finally, the abnormal association number can be used as the correlation degree to obtain a correlation degree group.

[0075] Step S4: Based on the aforementioned correlation group, select target detection information that meets the preset matching conditions from the aforementioned target detection information group as target anomaly information, and generate second detection information based on the aforementioned target anomaly information. The preset matching condition can be that the target detection information has the highest correlation. Therefore, for the target detection information with the highest correlation, the dynamic target association label corresponding to the target detection information can be used as the anomaly association label, and the time period corresponding to the target detection information can be used as the anomaly time period. The time period corresponding to the target detection information is the time period in the current scene video when the dynamic target is in contact with the target ventilator.

[0076] Optionally, if there are target detection information with the same abnormal association attribute in the target detection information group, the target detection information with the highest confidence of the corresponding association label can be selected as the target abnormal information to generate the second detection information.

[0077] Step 106: In response to the existence of second detection information that meets the preset abnormal conditions in the second detection information set, generate a first abnormal description information set based on each second detection information that meets the preset abnormal conditions, and send a ventilator abnormality warning based on the first abnormal description information set.

[0078] In some embodiments, the executing entity may, in response to the existence of second detection information in the second detection information set that satisfies a preset anomaly condition, generate a first anomaly description information set based on each second detection information that satisfies the preset anomaly condition, and send a ventilator anomaly warning based on the first anomaly description information set. The preset anomaly condition is that the second detection information includes an anomaly association tag corresponding to a dynamic target association tag. The second detection information satisfying the preset anomaly condition indicates that the ventilator anomaly is caused by contact between the dynamic target and the target ventilator. Therefore, the second detection information can be used as the first anomaly description information.

[0079] Secondly, the initial anomaly description information can be sent to the user's mobile terminal, and a voice announcement can be made. Additionally, the anomaly time period can be associated with corresponding current operational data and video clips from the current scene video.

[0080] In some optional implementations of certain embodiments, the execution entity, in response to the existence of second detection information satisfying preset anomaly conditions in the second detection information set, generates a first anomaly description information set based on each second detection information satisfying the preset anomaly conditions, including: Step S1: For each piece of second detection information in the above second detection information set that meets the preset abnormality conditions, perform the following steps: Step S1: Determine the abnormal time period corresponding to the second detection information. The abnormal time period is located within the time period corresponding to the abnormal response duration of the second detection information. Here, grouped scene images with the same dynamic target association label corresponding to the second detection information can be identified as abnormal images. Next, the duration between the maximum and minimum timestamps in each abnormal image is determined as the abnormal time period. The abnormal time period is the time period in the current scene video corresponding to the same abnormality (same dynamic target association label).

[0081] Step S2: Generate first anomaly description information based on the aforementioned abnormal time period and the aforementioned second detection information. Specifically, the aforementioned abnormal time period and the aforementioned second detection information can be filled into a preset description template to obtain the first anomaly description information.

[0082] As an example, the second detection information could be: "Abnormal association tag: Person-Touch-Ventilator Mask". Abnormal time period: [5.00 minutes, 5.10 minutes]. The preset description template could be: "Abnormal attribute: --, Abnormal factor: --, Abnormal time period: --". Then the first abnormal description information could be: "Abnormal attribute: Mask, Abnormal factor: Person touch, Abnormal time period: [5.00 minutes, 5.10 minutes]".

[0083] Optionally, the above execution topic may also include the following steps: Step S1: In response to the first detection information set not meeting the preset parameter detection conditions, second anomaly description information is generated. The first detection information set not meeting the preset parameter detection conditions indicates the existence of parameter detection information, including parameter detection identifiers indicating that the parameter configuration does not meet the constraint relationship. Furthermore, the parameter attributes of the parameter detection identifiers indicating that the parameter configuration does not meet the constraint relationship and the corresponding first detection information acquisition time point can be used as the second anomaly description information. The second anomaly description information is the description information indicating abnormal parameter settings detected during the ventilator preheating stage.

[0084] Step S2: Based on the aforementioned second anomaly description information, send a ventilator parameter anomaly warning. This can include sending the second anomaly description information to the user's mobile terminal and issuing a voice announcement. Furthermore, the anomaly time period can be associated with corresponding current operating data and video clips from the current scene video.

[0085] The above-described embodiments of this disclosure have the following beneficial effects: The home ventilator anomaly detection method of some embodiments of this disclosure can be used to detect ventilator anomalies or the impact of environmental factors on the ventilator in real time during home ventilator use scenarios, thereby issuing timely anomaly warnings and enabling the ventilator to provide timely assistance to the patient, reducing the probability of patient danger. Specifically, the home ventilator anomaly detection method of some embodiments of this disclosure firstly, in response to the target ventilator being started and its parameters being configured, performs parameter detection on the target ventilator to obtain a first detection information set, and performs camera shooting calibration, wherein the target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located. Secondly, in response to the first detection information set meeting preset parameter detection conditions, a ventilator wearing prompt message is issued. Then, in response to the completion of wearing and camera calibration, the calibrated camera is controlled to capture video of the current scene and obtain the current operating dataset of the target ventilator, wherein the current operating dataset is the ventilator equipment data during the use of the target ventilator corresponding to the current scene video. In practice, home settings often present numerous unexpected factors related to ventilator use (e.g., pet activity, children's activities). Compared to professional medical settings, home settings require not only monitoring the ventilator's own data but also understanding the impact of these unexpected factors on the ventilator. This allows for timely warnings, reducing the time spent investigating ventilator anomalies and thus mitigating the risk to the patient's life. This application introduces a camera into the home setting to capture video of the current ventilator usage scenario. Next, dynamic scene detection is performed on the video to generate a dynamic detection information set. Then, based on the dynamic detection information set and the current operating dataset, ventilator anomaly detection is performed to generate a second detection information set. This second detection information set combines the dynamic behavioral features corresponding to the current scene video and the data features corresponding to the ventilator's operating data, representing the detection results indicating whether the target ventilator is abnormal. The second detection information includes the abnormal time period and anomaly association label of the target ventilator. By performing dynamic scene detection on the current scene video, dynamic behavioral features present in the home ventilator usage scenario can be identified, facilitating the determination of whether there is a correlation between dynamic behavioral features and the ventilator. Finally, in response to the existence of second detection information that meets the preset abnormal conditions in the second detection information set, a first abnormal description information set is generated based on each second detection information that meets the preset abnormal conditions, and a ventilator abnormality warning is sent based on the first abnormal description information set.Therefore, it is possible not only to detect abnormal data of the ventilator itself in a timely manner, but also to identify the dynamic behavioral characteristics that lead to ventilator abnormalities in home activities. This allows for timely provision of specific circumstances leading to ventilator abnormalities, thereby reducing the time required for ventilator abnormality investigation and enabling timely issuance of ventilator abnormality warnings, thus lowering the probability of danger to patients.

[0086] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a home ventilator abnormality detection device, which are similar to... Figure 1 Corresponding to the method embodiments shown, this home ventilator abnormality detection device can be specifically applied to various electronic devices.

[0087] like Figure 4 As shown, a home ventilator anomaly detection device 400 in some embodiments includes: a parameter detection unit 401, a sending unit 402, an image capture and acquisition unit 403, a scene dynamic detection unit 404, a ventilator anomaly detection unit 405, and a generation unit 406. The parameter detection unit 401 is configured to, in response to the start-up of the target ventilator and completion of parameter configuration, perform parameter detection on the target ventilator to obtain a first detection information set, and perform image capture calibration on the camera, wherein the target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located; the sending unit 402 is configured to, in response to the first detection information set meeting preset parameter detection conditions, issue a ventilator wearing prompt message; the image capture and acquisition unit 403 is configured to, in response to completion of wearing and camera calibration, control the calibrated camera to capture the current scene video, and acquire the current operating dataset of the target ventilator, wherein the current operating dataset is the ventilator equipment data during the use of the target ventilator corresponding to the current scene video; the scene dynamic detection unit 404 is configured to... The current scene video is subjected to scene dynamic detection to generate a dynamic detection information set; the ventilator anomaly detection unit 405 is configured to perform ventilator anomaly detection based on the dynamic detection information set and the current running dataset to generate a second detection information set, wherein the second detection information set is a detection result representing whether the target ventilator is abnormal, generated by combining the dynamic behavior features corresponding to the current scene video and the data features corresponding to the ventilator running data, and the second detection information includes the abnormal time period and abnormal association label of the target ventilator; the generation unit 406 is configured to, in response to the existence of second detection information that meets preset abnormal conditions in the second detection information set, generate a first abnormal description information set based on each second detection information that meets the preset abnormal conditions, and send a ventilator anomaly warning based on the first abnormal description information set.

[0088] It is understandable that the units described in the home ventilator malfunction detection device 400 are similar to those in the reference device. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the home ventilator abnormality detection device 400 and the units contained therein, and will not be repeated here.

[0089] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device 500 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0090] like Figure 5 As shown, the electronic device 500 may include a processing unit 501 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in the read-only memory 502 or a program loaded from the storage device 508 into the random access memory 503. The random access memory 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, the read-only memory 502, and the random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0091] Typically, the following devices can be connected to the input / output interface 505: input devices 506 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0092] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a read-only memory 502. When the computer program is executed by the processing device 501, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0093] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0094] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0095] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: in response to the target ventilator being started and its parameters being configured, perform parameter detection on the target ventilator to obtain a first detection information set, and perform camera shooting calibration, wherein the target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located; in response to the first detection information set meeting preset parameter detection conditions, issue a ventilator wearing prompt message; in response to the wearing being completed and the camera calibration being completed, control the calibrated camera to capture the current scene video, and obtain the current operating dataset of the target ventilator, wherein the current operating dataset corresponds to the current scene video and is generated during the use of the target ventilator. The system collects ventilator equipment data; performs scene dynamic detection on the current scene video to generate a dynamic detection information set; performs ventilator anomaly detection based on the dynamic detection information set and the current running dataset to generate a second detection information set, wherein the second detection information set is a detection result representing whether the target ventilator is abnormal, generated by combining the dynamic behavior features corresponding to the current scene video and the data features corresponding to the ventilator running data, and the second detection information includes the abnormal time period and abnormal association label of the target ventilator; in response to the existence of second detection information that meets preset abnormal conditions in the second detection information set, generates a first abnormal description information set based on each second detection information that meets the preset abnormal conditions, and sends a ventilator anomaly warning based on the first abnormal description information set.

[0096] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0098] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0099] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for detecting abnormalities in a home ventilator, characterized in that, include: In response to the target ventilator being started and the parameters being configured, the parameters of the target ventilator are detected to obtain the first detection information set, and the camera is calibrated for shooting. The target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located. In response to the first set of detection information meeting the preset parameter detection conditions, a ventilator wearing prompt message is issued; In response to the completion of wearing and camera calibration, control the camera to capture the current scene video after calibration, and obtain the current operating dataset of the target ventilator. The current operating dataset is the ventilator equipment data during the use of the target ventilator, which corresponds to the current scene video. Scene dynamic detection is performed on the current scene video to generate a dynamic detection information set. The static scene information corresponding to the calibrated camera is determined as the current scene static information. The scene static information represents the static environment of the target ventilator. The dynamic detection information represents the dynamic behavior characteristics of the dynamic target in the current scene video that interacts with the static environment. The dynamic detection information includes: dynamic target location information, dynamic target association label and association label confidence. The dynamic target association label represents the association relationship between the dynamic target and the static environment. Based on the dynamic detection information set and the current running dataset, ventilator anomaly detection is performed to generate a second detection information set. The second detection information set is a detection result that characterizes whether the target ventilator is abnormal, generated by combining the dynamic behavior features corresponding to the current scene video and the data features corresponding to the ventilator running data. The second detection information includes the abnormal time period and abnormal association label of the target ventilator. In response to the presence of second detection information in the second detection information set that meets preset abnormal conditions, a first abnormal description information set is generated based on each second detection information that meets the preset abnormal conditions, and a ventilator abnormality warning is sent based on the first abnormal description information set.

2. The method according to claim 1, characterized in that, The method further includes: In response to the first detection information set not meeting the preset parameter detection conditions, a second anomaly description information is generated; Based on the second abnormality description information, a warning of abnormal ventilator parameters is sent.

3. The method according to claim 1, characterized in that, The camera shooting calibration includes: Control the camera to capture an initial angle image, and based on the initial angle image, perform the following first processing step: Based on the preset static feature extraction model, scene static recognition is performed on the initial angle image to generate scene static information. The scene static information includes: scene static region and corresponding region static label group. The scene static region is the region where the target ventilator and the user are fixed. The region static label is the feature label in the scene static region that is associated with the target ventilator. The static feature extraction model includes: backbone network, aggregation network, detection module and static post-processing module. Generate an angle offset vector based on static scene information; In response to the angle offset vector satisfying the preset offset condition, the camera is identified as the camera after shooting calibration; In response to the angle offset vector not meeting the preset offset condition, the camera is controlled to adjust the shooting angle according to the angle offset vector to obtain the current angle camera, and the current angle camera is controlled to shoot the current angle image as the initial angle image, and the first processing step is executed again.

4. The method according to claim 3, characterized in that, in, The step of performing scene dynamic detection on the current scene video to generate a dynamic detection information set includes: Frames are extracted from the current scene video to obtain the current scene image sequence; The current scene image sequence is grouped to obtain a grouped scene image sequence set; For each grouped scene image sequence in the grouped scene image sequence set, dynamic detection information in the dynamic detection information set is generated through the following steps: Based on the current scene static information and the preset dynamic feature extraction model, dynamic scene recognition is performed on the grouped scene image sequence to generate dynamic detection information including dynamic target location information, dynamic target association labels and association label confidence. The dynamic feature extraction model includes: target detection branch, target tracking branch, spatiotemporal feature extraction network and detection head. The target detection branch reuses the network structure of the static feature extraction model.

5. The method according to claim 4, characterized in that, The step of performing ventilator anomaly detection based on the dynamic detection information set and the current running dataset to generate a second detection information set includes: The current running dataset is time-aligned with the dynamic detection information set to obtain the aligned running dataset. Dynamic detection information that meets the preset confidence conditions is selected from the set of dynamic detection information and used as candidate detection information to obtain a candidate detection information group; Obtain the ventilator operation data corresponding to the aligned running dataset to obtain the ventilator operation dataset; Based on the ventilator operation dataset, data detection is performed on the aligned operation dataset in the aligned operation dataset to generate an initial abnormality information set, wherein the initial abnormality information includes an abnormality type identifier that characterizes the type of ventilator abnormality. Based on the preset abnormal delay response table, determine the abnormal response duration corresponding to each initial abnormal information in the initial abnormal information set; For each initial anomaly in the initial anomaly information set, perform the following second processing step: Based on the abnormal response duration corresponding to the initial abnormal information, candidate detection information that matches the initial abnormal information is selected from the candidate detection information group as target detection information, thus obtaining a target detection information group. The time point corresponding to the target detection information is within the time period corresponding to the abnormal response duration, and the dynamic target association label included in the target detection information matches the abnormal type identifier included in the initial abnormal information. Based on the target detection information group and the initial anomaly information, second detection information is generated.

6. The method according to claim 5, characterized in that, The step of generating second detection information based on the target detection information group and the initial anomaly information includes: In response to the target detection information group satisfying a first quantity condition, second detection information is generated, wherein the first quantity condition is that the number of target detection information in the target detection information group is 0, and the target detection information group satisfying the first quantity condition indicates that the target ventilator abnormality is unrelated to the dynamic target; In response to the target detection information group satisfying the second quantity condition, second detection information is generated based on the target detection information group, wherein the second quantity condition is that the number of target detection information in the target detection information group is 1; In response to the target detection information group satisfying the third quantity condition, a correlation group is generated based on the initial anomaly information and the target detection information group, wherein the third quantity condition is that the number of target detection information in the target detection information group is greater than 1; Based on the correlation group, target detection information that meets the preset matching conditions is selected from the target detection information group as target anomaly information, and second detection information is generated based on the target anomaly information.

7. The method according to claim 6, characterized in that, In response to the existence of second detection information satisfying preset anomaly conditions in the second detection information set, a first anomaly description information set is generated based on each second detection information satisfying the preset anomaly conditions, including: For each piece of second detection information in the second detection information set that meets the preset abnormal conditions, the following steps are performed: Determine the abnormal time period corresponding to the second detection information, wherein the abnormal time period is located within the time period of the abnormal response duration corresponding to the second detection information; Based on the abnormal time period and the second detection information, a first abnormal description information is generated.

8. A home ventilator malfunction detection device, characterized in that, include: The parameter detection unit is configured to detect parameters of the target ventilator in response to the start-up of the target ventilator and the completion of parameter configuration, obtain a first detection information set, and perform shooting calibration on the camera. The target ventilator is a home ventilator, and the camera is used to capture the environment in which the target ventilator is located. The transmitting unit is configured to send a ventilator wearing prompt message in response to the first detection information set meeting the preset parameter detection conditions. The shooting and acquisition unit is configured to, in response to completion of wearing and camera calibration, control the calibrated camera to shoot video of the current scene and acquire the current operating dataset of the target ventilator, wherein the current operating dataset is the ventilator equipment data during the use of the target ventilator corresponding to the current scene video; The scene dynamic detection unit is configured to perform scene dynamic detection on the current scene video to generate a dynamic detection information set. The static scene information corresponding to the calibrated camera is determined as the current scene static information. The scene static information represents the static environment of the target ventilator. The dynamic detection information represents the dynamic behavior characteristics of the dynamic target in the current scene video that interacts with the static environment. The dynamic detection information includes: dynamic target location information, dynamic target association label and association label confidence. The dynamic target association label represents the association relationship between the dynamic target and the static environment. The ventilator anomaly detection unit is configured to perform ventilator anomaly detection based on the dynamic detection information set and the current running dataset to generate a second detection information set. The second detection information set is a detection result that characterizes whether the target ventilator is abnormal, generated by combining the dynamic behavior features corresponding to the current scene video and the data features corresponding to the ventilator running data. The second detection information includes the abnormal time period and abnormal association label of the target ventilator. The generation unit is configured to, in response to the existence of second detection information that satisfies preset abnormal conditions in the second detection information set, generate a first abnormal description information set based on each second detection information that satisfies preset abnormal conditions, and send a ventilator abnormality warning based on the first abnormal description information set.

9. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images