Respiration monitoring method, device, equipment and computer-readable storage medium

Abstract

The present application is applicable to the field of image processing technology, and provides a respiratory monitoring method, apparatus, device and computer-readable storage medium, including: obtaining the corresponding abdominal regions from multiple frames of continuously acquired images, each abdominal region at least including the abdomen of the target body; determining multiple correlation values ​​for each abdominal region; dividing the multiple correlation values ​​of each abdominal region to obtain multiple groups of correlation values, each group of correlation values ​​being used to characterize a breath completed by the target body; and determining whether the target body is breathing normally based on the multiple groups of correlation values. Thus, the present application can obtain multiple groups of correlation values ​​based on the correlation between the abdominal regions, so that whether the target body is breathing normally can be determined based on the multiple groups of correlation values. The present application can improve the convenience of respiratory monitoring, simplify the user's operation, and improve the accuracy of respiratory monitoring. It is applicable to various daily scenarios that require monitoring the user's breathing.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a respiratory monitoring method, apparatus, device, and computer-readable storage medium. Background Art

[0002] With the continuous development of the economy, users have an increasing demand for timely monitoring of their health status. Respiratory rate, as a key parameter reflecting physiological status and sleep quality, can provide critical information closely related to a user's health status. Therefore, how to monitor respiratory rate is extremely important.

[0003] Currently, there are two main monitoring methods: contact and non-contact. Contact monitoring requires the user to come into contact with the monitoring device or a specially designed mattress to monitor their breathing, which is less comfortable and unsuitable for users with delicate skin. Non-contact monitoring uses respiratory measurement methods such as bioradar and thermal imaging to monitor the user's breathing. This requires appropriate monitoring equipment and specialized personnel, making it inconvenient for daily monitoring.

[0004] Therefore, how to monitor the user's breathing in daily scenarios is an urgent problem that needs to be solved. Summary of the Invention

[0005] The present application provides a respiratory monitoring method, apparatus, device, and computer-readable storage medium, which can avoid the discomfort caused by the user being connected to the monitoring device, saving professional human resources and medical equipment resources.

[0006] In a first aspect, the present application provides a respiratory monitoring method, comprising:

[0007] Obtaining respective corresponding abdominal regions from the multiple frames of images acquired continuously, each abdominal region at least including the abdomen of the target body;

[0008] determining a plurality of correlation values ​​for each abdominal region, the plurality of correlation values ​​for any abdominal region being used to represent a correlation between the any abdominal region and each of a preset number of abdominal regions, the preset number of abdominal regions including the any abdominal region and a plurality of abdominal regions acquired consecutively after the any abdominal region, the preset number being greater than or equal to a total number of image frames that can be acquired by completing the plurality of breaths;

[0009] Dividing the multiple correlation values ​​of each abdominal region to obtain multiple groups of correlation values, each group of correlation values ​​is used to represent a breath completed by the target body;

[0010] According to the multiple sets of correlation values, it is determined whether the target body is breathing normally.

[0011] In a possible implementation of the first aspect, the method further includes:

[0012] A plurality of respiratory frequencies are determined according to the plurality of sets of correlation values, and each respiratory frequency is used to indicate whether the target body is breathing normally or breathing abnormally.

[0013] In a possible implementation of the first aspect, after determining multiple respiratory rates according to multiple sets of correlation values, the method further includes:

[0014] A respiratory frequency curve is drawn according to each respiratory frequency and the duration of multiple frames of images. The respiratory frequency curve is used to indicate whether the target is in an exhalation state or an inhalation state.

[0015] In a possible implementation of the first aspect, determining multiple respiratory rates according to multiple sets of correlation values ​​includes:

[0016] Determine the correlation value at a preset position in each group of correlation values ​​as a target correlation value;

[0017] determining a first abdominal region and a second abdominal region associated with a target correlation value in each set of correlation values;

[0018] determining a plurality of target-related values ​​corresponding to each identical first abdominal region;

[0019] For multiple target correlation values ​​corresponding to any same first abdominal region, two second abdominal regions associated with any two target correlation values ​​are respectively determined, and a respiratory frequency is obtained according to the frame number difference between the two second abdominal regions.

[0020] In a possible implementation of the first aspect, when any same first abdominal region corresponds to multiple target correlation values, each group of correlation values ​​to which the multiple target correlation values ​​belong is used to represent normal breathing of the target body;

[0021] When any identical first abdominal region corresponds to a target correlation value, a group of correlation values ​​to which the target correlation value belongs is used to represent abnormal breathing of the target body.

[0022] In a possible implementation of the first aspect, the method further includes:

[0023] When it is determined that the target body has abnormal breathing, a warning message indicating abnormal breathing of the target body is displayed.

[0024] In a possible implementation of the first aspect, acquiring the corresponding abdominal regions from the plurality of continuously acquired image frames includes:

[0025] Acquire multiple frames of images from the target's breathing video;

[0026] Perform key point detection on each frame image to obtain the key points of the neck, left shoulder, right shoulder and mid-hip;

[0027] Determine the key points of the right abdomen and the left abdomen based on the key points of the neck, left shoulder, right shoulder, and mid-hip.

[0028] Determine the abdomen area based on the keypoints of the left shoulder, the right shoulder, the right abdomen, and the left abdomen.

[0029] In a possible implementation of the first aspect, determining key points on the right abdomen and key points on the left abdomen based on key points on the neck, left shoulder, right shoulder, and mid-hip, includes:

[0030] On the line between the key point of the neck and the key point of the mid-hip, determine the position with a preset distance from the key point of the mid-hip as the key point of the mid-abdomen;

[0031] Determine the key point of the right abdomen on the side of the key point of the midsection close to the key point of the right shoulder. The distance between the key point of the midsection and the key point of the right abdomen is equal to the distance between the key point of the neck and the key point of the right shoulder.

[0032] On the side of the midsection keypoint close to the left shoulder keypoint, determine the keypoint on the left abdomen, where the distance between the midsection keypoint and the left abdomen keypoint is equal to the distance between the neck keypoint and the left shoulder keypoint.

[0033] In a possible implementation of the first aspect, determining the abdominal region based on key points of the left shoulder, key points of the right shoulder, key points of the right abdomen, and key points of the left abdomen includes:

[0034] Project the key points of the left shoulder, right shoulder, right abdomen, and left abdomen into the same coordinate system;

[0035] Determine, according to the coordinate system, the minimum abscissa, the maximum abscissa, the minimum ordinate, and the maximum ordinate among the coordinates of the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen;

[0036] Extend the minimum horizontal coordinate and the maximum horizontal coordinate along the vertical axis of the coordinate system, and extend the minimum vertical coordinate and the maximum vertical coordinate along the horizontal axis of the coordinate system to form a quadrilateral area;

[0037] The quadrilateral area is determined as the largest abdominal area.

[0038] The respiratory monitoring method provided in this application avoids the discomfort caused by the user being connected to the monitoring device, saves professional human resources and medical equipment resources, simplifies the user's operation, improves the convenience of respiratory monitoring, ensures the accuracy of respiratory monitoring results, and is suitable for various daily scenarios that require monitoring the user's breathing.

[0039] In a second aspect, the present application provides a respiratory monitoring device, which is used to perform the respiratory monitoring method described in the first aspect or any possible implementation of the first aspect. Specifically, the device includes:

[0040] An acquisition module is used to acquire the corresponding abdominal regions from the multiple frames of images acquired continuously, each abdominal region at least including the abdomen of the target body;

[0041] a determination module, configured to determine multiple correlation values ​​for each abdominal region, the multiple correlation values ​​for any abdominal region being used to represent a correlation between the any abdominal region and each of a preset number of abdominal regions, the preset number of abdominal regions including the any abdominal region and multiple abdominal regions acquired consecutively after the any abdominal region, the preset number being greater than or equal to a total number of image frames that can be acquired by completing multiple breaths;

[0042] The determination module is further configured to divide the multiple correlation values ​​of each abdominal region into multiple groups of correlation values, each group of correlation values ​​being used to represent a breath completed by the target body;

[0043] The determination module is further used to determine whether the target body is breathing normally based on multiple sets of correlation values.

[0044] In a third aspect, the present application provides a device comprising a memory and a processor. The memory is used to store instructions; the processor executes the instructions stored in the memory, so that the device performs the respiratory monitoring method of the first aspect or any possible implementation of the first aspect.

[0045] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the respiratory monitoring method in the first aspect or any possible implementation of the first aspect.

[0046] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a device, causes the device to execute the respiratory monitoring method in the first aspect or any possible implementation of the first aspect.

[0047] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 1 is a flow chart of a respiratory monitoring method provided in one embodiment of the present application;

[0050] Figure 2 This is a schematic diagram of the correlation value calculation results provided by an embodiment of the present application;

[0051] Figure 3 1 is a flow chart of a respiratory monitoring method provided in one embodiment of the present application;

[0052] Figure 4 This is a schematic diagram of the correlation value calculation results provided by an embodiment of the present application;

[0053] Figure 5 This is a schematic diagram of a respiratory frequency curve provided by an embodiment of the present application;

[0054] Figure 6 1 is a flow chart of a respiratory monitoring method provided in one embodiment of the present application;

[0055] Figure 7 This is a schematic diagram of key points of a target object provided by an embodiment of the present application;

[0056] Figure 8 This is a schematic diagram of key points of a target object provided by an embodiment of the present application;

[0057] Figure 9 This is a schematic diagram of key points of a target object provided by an embodiment of the present application;

[0058] Figure 10 1 is a schematic structural diagram of a respiratory monitoring device provided in one embodiment of the present application;

[0059] Figure 11 It is a structural diagram of an electronic device provided in one embodiment of the present application.

[0060] Reference numerals:

[0061] Each solid point represents each key point of the target object in a frame of image;

[0062] The solid point 1 represents the key point of the neck;

[0063] The solid point 2 represents the key point of the right shoulder;

[0064] The solid dot 3 indicates the key point of the left button;

[0065] The solid point 4 indicates the key point of the middle hip;

[0066] The solid point 5 indicates the key point in the midsection;

[0067] The solid point 6 indicates the key point on the left abdomen;

[0068] The solid point 7 indicates the key point on the right abdomen;

[0069] Rectangular frame A represents the abdominal region. DETAILED DESCRIPTION

[0070] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0071] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0072] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0073] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0074] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0075] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0076] The present application proposes a respiratory monitoring method, apparatus, device and computer-readable storage medium, which are applicable to various scenarios requiring respiratory monitoring.

[0077] For example, the respiratory monitoring method of the present application is applicable to human respiratory monitoring, which is convenient for determining whether the human body is breathing normally.

[0078] By monitoring the breathing of the human body based on the sleep video, we can obtain the human body's breathing status, such as whether the human body is breathing normally and the breathing frequency. Furthermore, the human body's health status can be estimated through the human body's breathing status, which is convenient for discovering potential diseases in the human body and thus enabling timely treatment.

[0079] Among them, the respiratory monitoring method of the present application can be executed by an electronic device.

[0080] The electronic device may be a smartphone, tablet computer, desktop computer, laptop computer, handheld device, server, etc.

[0081] In addition, the electronic device may include a display screen, or be connected to an external display screen.

[0082] Thus, the electronic device can display the respiratory monitoring results through the display screen, and the respiratory monitoring results can include whether the target body is breathing normally, the target body's respiratory frequency, the target body's respiratory frequency curve, etc.

[0083] Based on the above scenario description, below, this application takes electronic equipment as an example, combined with the accompanying drawings and application scenarios, to elaborate on the respiratory monitoring method provided in the embodiment of this application.

[0084] See also Figure 1 , Figure 1 A flow chart of a respiratory monitoring method provided in an embodiment of the present application is shown.

[0085] like Figure 1 As shown, the respiratory monitoring method provided by the present application may include:

[0086] S101 , acquiring the corresponding abdominal regions from a plurality of frames of images acquired continuously.

[0087] The continuously acquired multiple frames of images refer to multiple frames of images that are continuous in time sequence, and each frame of image at least includes the abdominal region of the target body.

[0088] Thus, the electronic device can acquire the abdomen area of ​​each frame of the multiple frames.

[0089] Each abdominal region includes at least the abdomen of the target body.

[0090] It should be understood that the abdominal region will rise and fall once each time the subject completes a breath. Thus, the breathing process of the subject can be reflected through the changes in the abdominal region.

[0091] S102: Determine multiple correlation values ​​for each abdominal region.

[0092] The correlation value indicates the degree of linear correlation between variables.

[0093] The multiple correlation values ​​of any one abdominal region are used to represent the correlation between the any one abdominal region and each of the preset number of abdominal regions.

[0094] The preset number of abdominal regions includes any one abdominal region and multiple abdominal regions continuously acquired after any one abdominal region, and the preset number is greater than or equal to the total number of frames of images that can be acquired by completing multiple breaths.

[0095] Assuming that the preset number of multi-frame images is 2550 frames, then there are 2550 abdominal regions, and the multiple correlation values ​​of the first abdominal region include: the correlation value between the first abdominal region and the first abdominal region, the correlation value between the first abdominal region and the second abdominal region...the correlation value between the first abdominal region and the 2550th abdominal region.

[0096] In some embodiments, the formula for calculating the correlation value between two abdominal regions is:

[0097]

[0098] Among them, A mn is a matrix of the abdominal region, is the average value of the abdominal region matrix, B mn is another matrix in the abdominal area, is the average of another abdominal region matrix.

[0099] S103 , dividing the multiple correlation values ​​of each abdominal region to obtain multiple groups of correlation values.

[0100] Based on S102, multiple correlation values ​​of each abdominal area can be obtained. Since multiple frames of images, i.e., multiple abdominal areas, can be collected each time a breath is completed, the multiple correlation values ​​of the multiple abdominal areas can form a group of correlation values, so that the electronic device can obtain multiple groups of correlation values.

[0101] Each set of correlation values ​​is used to represent a breath completed by the target body.

[0102] like Figure 2 As shown, the preset number is 2550 frames, and the horizontal and vertical coordinates are 2550 frames of images collected continuously, wherein each frame of image is associated with an abdominal area. Figure 2 It can be seen from FIG1 that multiple correlation values ​​of multiple abdominal regions can form a group of correlation values, that is, a gray area.

[0103] Figure 2 In the figure, when the vertical axis is fixed, observe the process of the horizontal axis increasing. Each time it passes through a gray area, it can be considered as completing a breath.

[0104] For example, for 500 frames on the vertical axis, the horizontal axis corresponding to the 500 frames includes two gray areas, and the two gray areas respectively represent a breath completed by the target body.

[0105] S104: Determine whether the target body is breathing normally based on the multiple sets of correlation values.

[0106] It should be understood that when the target breathes normally, the state of the abdominal region can meet the preset breathing pattern. For example, the abdominal region rises and falls once every time the target completes a breath.

[0107] Then, based on the correlation between the abdominal areas, each set of calculated correlation values ​​must also satisfy a similar correlation value rule. For example, each time the target body completes a breath, a set of correlation values ​​may go through a process of first decreasing and then increasing, or each time the target body completes a breath, the number of frames of images that can be collected corresponding to a set of correlation values ​​is within a preset range.

[0108] Thus, the electronic device can determine whether the target body is breathing normally according to the correlation value rule of each group of correlation values.

[0109] If this set of correlation values ​​satisfies the correlation value rule, the electronic device can determine that the current target body is breathing normally.

[0110] If this group of correlation values ​​does not satisfy the correlation value rule, the electronic device can determine that the current target body is breathing abnormally.

[0111] Continue to combine Figure 2 , for the number of frames on the vertical axis, from Figure 2It can be seen that each set of correlation values ​​corresponding to the 1st frame to the 1500th frame satisfies the correlation value law, which can determine that the target object is breathing normally during the image time from the 1st frame to the 1500th frame. Two sets of correlation values ​​corresponding to the 1500th frame to the 2000th frame do not satisfy the correlation value law, which can be determined that the target object is breathing abnormally at this time.

[0112] Furthermore, according to the calculation results and analysis of the formula in S102, it can be understood that when the abdominal region of the target object reaches its maximum convexity from the starting position, an inhalation process is completed, and the correlation value between the abdominal region at the maximum convexity and the starting abdominal region at the starting position is the smallest. When the abdominal region gradually falls back from its maximum convexity to the starting position, an exhalation process is completed, and the position of the abdominal region at this time is the same as the starting position of the starting abdominal region. Therefore, when the starting abdominal region is fixed, each time the correlation value decreases and then increases, it can be considered that the target object has completed a breath.

[0113] In the respiratory monitoring method of the present application, the electronic device obtains the corresponding abdominal areas from the continuously collected multiple frames of images, and can prepare data for calculating multiple correlation values ​​of each abdominal area through the correlation between the abdominal areas. The electronic device can determine the multiple correlation values ​​of each abdominal area, and divide the multiple correlation values ​​of each abdominal area to obtain multiple groups of correlation values. The electronic device can determine whether the target body is currently breathing normally based on whether each group of correlation values ​​in the multiple groups of correlation values ​​meets the correlation value law. Thus, the electronic device can quickly and accurately determine whether the target body is breathing normally through the continuously collected multiple frames of images, simplifying the operating process, improving the efficiency of respiratory monitoring, ensuring the accuracy of respiratory monitoring results, and enhancing the convenience of respiratory monitoring. It is suitable for various daily scenarios that require monitoring the user's breathing.

[0114] Based on the above Figure 1 In the description of the illustrated embodiment, the electronic device may also determine multiple breathing frequencies based on multiple sets of correlation values, wherein each breathing frequency may also be used to indicate whether the target body is breathing normally or abnormally.

[0115] Next, combine Figure 3 , which introduces in detail the specific implementation process of the respiratory monitoring method of this application.

[0116] See also Figure 3 , Figure 3 A flow chart of a respiratory monitoring method provided in an embodiment of the present application is shown.

[0117] like Figure 3 As shown, the respiratory monitoring method provided by the present application may include:

[0118] S201 : Obtaining the corresponding abdominal regions from a plurality of frames of images acquired continuously.

[0119] S202: Determine multiple correlation values ​​for each abdominal region.

[0120] S203 , dividing the multiple correlation values ​​of each abdominal region to obtain multiple groups of correlation values.

[0121] Among them, S201, S202, and S203 are respectively Figure 1 The implementation of S101, S102, and S103 in the illustrated embodiment is similar and will not be described in detail in this application.

[0122] S204: Determine the correlation value located at a preset position in each group of correlation values ​​as a target correlation value.

[0123] The preset position is the center of gravity of the gray area corresponding to each set of correlation values.

[0124] The target correlation value is the correlation value at the center of gravity of each group of correlation values.

[0125] In addition, the target correlation value can be understood as the minimum correlation value in the group of correlation values.

[0126] The electronic device determines the correlation value of the preset position as the target correlation value, and prepares data for calculating the respiratory rate.

[0127] In addition, continue to combine Figure 2 To calculate the respiratory rate, the electronic device can Figure 2 The multiple sets of correlation values ​​in are extracted by the connected region technique, and we get Figure 4 .like Figure 4 As shown, the target correlation value is the correlation value of the center of gravity of each black solid area.

[0128] S205 : Determine a first abdominal region and a second abdominal region associated with the target correlation value in each group of correlation values.

[0129] It should be understood that each correlation value is calculated based on the correlation between two abdominal regions. Therefore, each target correlation value may correspond to two abdominal regions, namely, the first abdominal region and the second abdominal region.

[0130] Continue to combine Figure 4 ,from Figure 4 It can be seen that each target abdominal area corresponds to one horizontal coordinate and one vertical coordinate, that is, each target correlation value corresponds to two abdominal areas.

[0131] S206: Determine multiple target correlation values ​​corresponding to each identical first abdominal region.

[0132] It should be understood that the preset number in S102 is greater than or equal to the total number of frames of images that can be collected by completing multiple breaths.

[0133] Thus, the number of the plurality of target correlation values ​​is equal to the number of the plurality of breaths.

[0134] For example, the aforementioned multiple breaths are 2 times, and each identical first abdominal region corresponds to 2 target correlation values.

[0135] Continue to combine Figure 4 ,from Figure 4 It can be seen that each identical vertical coordinate corresponds to two groups of correlation values, that is, to two target correlation values.

[0136] S207 . For multiple target correlation values ​​corresponding to any same first abdominal region, respectively determine two second abdominal regions associated with any two target correlation values, and obtain a respiratory frequency based on a frame number difference between the two second abdominal regions.

[0137] Based on S206 , it can be determined that the number of target correlation values ​​is equal to the number of breaths. Therefore, the frame number difference between the two second abdominal regions is equivalent to the number of image frames that can be collected when the target completes one breath.

[0138] In some embodiments, the respiratory frequency is calculated as follows: respiratory frequency = (60*f) / m.

[0139] Here, m represents the difference in the horizontal coordinates between the centers of gravity of two adjacent groups of correlation values, that is, the difference in the number of frames between two adjacent second abdominal regions, and f represents the frame rate.

[0140] Assume that m is 120 frames, that is, the number of image frames that can be collected when the target completes one breath is 120 frames, and the frame rate is 30.

[0141] Then, respiratory rate = (60*30) / 120 = 15 times / min, that is, one breath per 4 seconds.

[0142] Therefore, the electronic device can calculate the respiratory frequency of the target body according to the number of image frames that can be collected when the target body completes one breath, so that the user can judge whether the target body is breathing normally according to the respiratory frequency.

[0143] When any identical first abdominal region corresponds to multiple target correlation values, each group of correlation values ​​to which the multiple target correlation values ​​belong is used to represent normal breathing of the target body.

[0144] When any identical first abdominal region corresponds to a target correlation value, a group of correlation values ​​to which the target correlation value belongs is used to represent abnormal breathing of the target body.

[0145] Continue to combine Figure 4 , Figure 4 Each identical vertical coordinate in corresponds to two target related values.

[0146] from Figure 4 It can be seen that when any identical first abdominal region corresponds to two target correlation values, the two groups of correlation values ​​to which the two target correlation values ​​belong are used to represent normal breathing of the target body. Figure 4 In the image, the target breathes normally from the 1st frame to the 1500th frame.

[0147] When any identical first abdominal region corresponds to one target correlation value, the group of correlation values ​​to which the target correlation value belongs is used to represent abnormal breathing of the target body. Figure 4 In the example, if multiple frames between the 1500th frame and the 2000th frame correspond to one target-related value, the electronic device can determine that the target is breathing abnormally at this time.

[0148] That is, the electronic device can determine whether the target body is breathing normally according to the number of target-related values ​​corresponding to any same first abdominal area.

[0149] S208. When it is determined that the target body has abnormal breathing, a warning message indicating abnormal breathing of the target body may be displayed.

[0150] Among them, S208 is an optional step.

[0151] The electronic device may include a display interface. When the electronic device determines that the target person is experiencing abnormal breathing, the display interface may display a warning message, such as "error," to alert the user that the target person is experiencing abnormal breathing.

[0152] This application does not limit the specific implementation method of the display interface.

[0153] S209 , drawing a respiratory frequency curve according to each respiratory frequency and the duration of the multiple frames of images.

[0154] Among them, S209 is an optional step.

[0155] The respiratory frequency may be the respiratory frequency calculated in S207 .

[0156] The duration of the multiple frames of images may be the duration corresponding to the multiple frames of images continuously acquired in S101 .

[0157] The respiratory rate curve is used to indicate whether the subject is in an exhalation state or an inhalation state.

[0158] In some embodiments, the respiratory rate curve is a sinusoidal function curve, wherein the formula of the sinusoidal function is:

[0159] y = sin(wt);

[0160] w=π*k / 30;

[0161] k*T=60;

[0162] T = 2π / w.

[0163] Where t represents the duration of the continuously acquired multi-frame images, w represents the frequency of the sine function, T represents the period of the sine function, and k represents the respiratory rate of k times / min. Each completed breath corresponds to one period of the sine function.

[0164] Figure 5 It is a sine function curve with a duration of 30 seconds corresponding to the continuously collected multiple frames of images. The horizontal axis is time in seconds, and the vertical axis represents the breathing state of the target body. An increase represents inhalation, and a decrease represents exhalation. Figure 5 In the example, if the target breathes 6 times in 20 seconds, then the target breathes 18 times per minute and the respiratory rate is 18 times / min. Figure 5 The respiratory state of the target body is determined by the changes in the respiratory frequency curve.

[0165] Thus, the electronic device can draw a respiratory rate curve and display the respiratory rate curve in the above display interface. The electronic device uses the respiratory rate curve to visualize the respiratory state of the target body, so that the user can understand the respiratory state of the target body more intuitively.

[0166] In the present application, the electronic device can determine two second abdominal regions associated with any two target-related values, and based on the frame difference between the two second abdominal regions, determine the number of frames of images that can be collected each time the target completes a breath. Based on the number of frames, a respiratory frequency can be obtained, and thus, the electronic device can use the respiratory frequency to determine whether the target is breathing normally. In addition, the electronic device can display a warning message when the target is breathing abnormally, so as to promptly remind the user that the target has abnormal breathing. In addition, the electronic device can also draw a respiratory frequency curve to facilitate the user to intuitively understand the respiratory status of the target.

[0167] Based on the above Figure 1 As described in the embodiment shown, when the electronic device obtains the corresponding abdominal areas from multiple frames of continuously acquired images, the abdominal area of ​​the target body can be determined through the key points of the target body's neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the middle hip.

[0168] Next, combine Figure 6 , which introduces in detail the specific implementation process of the respiratory monitoring method of this application.

[0169] The electronic device can obtain the key points of the target body's neck, left shoulder, right shoulder, and mid-hip key points, and determine the key points of the mid-abdomen based on the key points of the neck, left shoulder, right shoulder, and mid-hip key points. Then, the key points of the right abdomen and the key points of the left abdomen are determined on both sides of the key points of the mid-abdomen, and the abdominal area is determined based on the key points of the left shoulder, right shoulder, right abdomen, and left abdomen.

[0170] In some implementations, the electronic device can obtain key points of various parts of the target body through a human posture estimation algorithm, and determine the key points of the neck, left shoulder, right shoulder, and mid-hip from the key points of various parts.

[0171] like Figure 7 As shown, the solid points are key points of various parts of the target body that can be obtained by the electronic device through the human posture estimation algorithm. Among them, solid point 1 is the key point of the neck, solid point 2 is the key point of the right shoulder, solid point 3 is the key point of the left shoulder, and solid point 4 is the key point of the middle hip.

[0172] The present application can store the human body posture estimation algorithm in an electronic device and / or a storage device that communicates with the electronic device, so that the electronic device can conveniently call the human body posture estimation algorithm to obtain the key points of various parts of the target body.

[0173] The present application does not limit the storage method and specific type of the storage device.

[0174] See also Figure 6 , Figure 6 A flow chart of a respiratory monitoring method provided in an embodiment of the present application is shown.

[0175] like Figure 6 As shown, the respiratory monitoring method provided by the present application may include:

[0176] S301. Acquire multiple frames of images from a breathing video of a target body.

[0177] The breathing video of the target object is usually a video collected when the target object is in a sleeping state.

[0178] It should be understood that when the target is in a sleeping state, they rarely turn over or move, so the abdominal area can be exposed, making it easier for the electronic device to accurately monitor the target's breathing based on the ups and downs of the abdominal area.

[0179] In addition, the electronic device monitors the target's breathing based on the video collected when the target is in a sleeping state, and can determine the target's sleeping state and health information related to the sleeping state.

[0180] The multi-frame images are multiple frame images of the target body's breathing video that are continuous in time sequence.

[0181] In some embodiments, the multiple frames of images are multiple frames of images that are continuous in time sequence in the entire breathing video of the target.

[0182] In other embodiments, the multiple frames of images are multiple frames of images that are continuous in time sequence and captured from the entire breathing video of the target.

[0183] In addition, the duration of the video collected when the target is in a sleeping state must be greater than 10 seconds. It is understandable that if the duration of the video is less than 10 seconds, the number of respirations is too small to calculate an accurate result.

[0184] S302: Perform key point detection on each frame image to obtain key points of the neck, left shoulder, right shoulder and mid-hip.

[0185] Among them, the electronic device uses a human posture estimation algorithm to detect key points in each frame of the image, and can obtain key points of multiple parts of the target body. Each key point is used to identify each part of the target body.

[0186] For example, the key points of multiple parts may include: key points of the head, key points of the neck, key points of the left shoulder, key points of the right shoulder, key points of the elbows, key points of the wrists, key points of the mid-hips, key points of the waist, key points of the knees and other key points of various parts of the target body.

[0187] Furthermore, the electronic device may determine the key points of the neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the mid-hip from the key points of the plurality of parts of the target body.

[0188] S303: On the line connecting the key point of the neck and the key point of the mid-hip, determine a position with a preset distance from the key point of the mid-hip as the key point of the mid-abdomen.

[0189] Based on S302, the electronic device can obtain the key points of the neck, the left shoulder, the right shoulder, and the mid-hip key points. Thus, the electronic device can determine the key points of the mid-abdomen on the line connecting the key points of the neck and the mid-hip key points.

[0190] In some embodiments, the key point of the mid-abdomen may be a point on the above-mentioned connecting line that is a preset length away from the key point of the mid-hip.

[0191] The preset length is predetermined. For example, the preset length may be ¾ of the length between the key point of the neck and the key point of the mid-hip.

[0192] like Figure 8 As shown, on the line between solid point 1 and solid point 4, the electronic device can use the point at a preset length away from the key point of the middle hip, that is, solid point 5, as the key point of the middle hip.

[0193] S304. Determine the key point of the right abdomen on the side of the key point of the middle abdomen close to the key point of the right shoulder.

[0194] S305. Determine the key point of the left abdomen on the side of the key point of the middle abdomen close to the key point of the left shoulder.

[0195] Based on S303 , the electronic device may obtain the key points of the middle abdomen, and thus, the electronic device may determine the key points of the right abdomen and the key points of the left abdomen on the left and right sides of the key points of the middle abdomen.

[0196] The electronic device may use the distance between the key point of the neck and the key point of the right shoulder as the distance between the key point of the middle abdomen and the key point of the right abdomen, and use the distance between the key point of the neck and the key point of the left shoulder as the distance between the key point of the middle abdomen and the key point of the left abdomen. That is, the distance between the key point of the middle abdomen and the key point of the right abdomen is equal to the distance between the key point of the neck and the key point of the right shoulder. The distance between the key point of the middle abdomen and the key point of the left abdomen is equal to the distance between the key point of the neck and the key point of the left shoulder.

[0197] Thus, the electronic device can obtain accurate key points of the right abdomen and the left abdomen.

[0198] In some embodiments, the electronic device may project key points of multiple parts of the target body into the same coordinate system to obtain the coordinates of the key points of the neck, the left shoulder, the right shoulder, and the mid-abdomen. Thus, the electronic device may calculate the coordinates of the key points of the right abdomen and the key points of the left abdomen.

[0199] Based on the above description, the electronic device can represent the key points of the neck, the left shoulder, the right shoulder, and the mid-abdomen through coordinates. The coordinates of the key points of the neck are (x1, y1), the coordinates of the key points of the right shoulder are (x2, y2), the coordinates of the key points of the left shoulder are (x3, y3), and the coordinates of the key points of the mid-abdomen are (x4, y4).

[0200] The above formula for calculating the coordinates of the key points of the right abdomen and the key points of the left abdomen is:

[0201] (x a ,y a ) = (x1-x2, y1-y2)

[0202] (x b ,yb ) = (x1-x3, y1-y3)

[0203] (x c ,y c )=(x4-x a ,y4-y a )

[0204] (x d ,y d )=(x4-x b ,y4-y b )

[0205] Among them, (x c ,y c ) is used to represent the coordinates of the key points on the right abdomen, (x d ,y d ) is used to represent the coordinates of the key points on the left abdomen.

[0206] Continue to combine Figure 8 The electronic device determines the key point of the right abdomen, namely solid point 7, on the side of solid point 5 close to solid point 2, and determines the key point of the left abdomen, namely solid point 6, on the side of solid point 5 close to solid point 3.

[0207] Thus, the electronic device can obtain the key points of the right abdomen and the key points of the left abdomen with the help of the key points of the middle abdomen, and prepare data for determining the abdominal area based on the key points of the right abdomen and the key points of the left abdomen.

[0208] S306: Project the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen into the same coordinate system.

[0209] S307. Determine the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate, and maximum vertical coordinate among the coordinates of the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen according to the coordinate system.

[0210] Among them, the electronic device can project the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen, and the key points of the left abdomen into the same coordinate system, and obtain the coordinates of the key points of the left shoulder, the coordinates of the key points of the right shoulder, the coordinates of the key points of the right abdomen, and the coordinates of the key points of the left abdomen.

[0211] Thus, the electronic device can determine the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate, and maximum vertical coordinate among the coordinates of the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen. The electronic device can prepare data for determining the abdominal region based on the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate, and maximum vertical coordinate.

[0212] S308 , extending the minimum horizontal coordinate and the maximum horizontal coordinate along the vertical axis of the coordinate system, and extending the minimum vertical coordinate and the maximum vertical coordinate along the horizontal axis of the coordinate system, to form a quadrilateral area.

[0213] S309: Determine the quadrilateral area as the largest abdominal area.

[0214] like Figure 9 As shown, the electronic device can determine the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate, and the maximum vertical coordinate among the solid points 2, 3, 6, and 7. Figure 9 It can be seen that the minimum abscissa is the abscissa of the solid point 6, the maximum abscissa is the abscissa of the solid point 2, the minimum ordinate is the ordinate of the solid point 7, and the maximum ordinate is the ordinate of the solid point 3.

[0215] Thus, the electronic device can extend the minimum horizontal coordinate and the maximum horizontal coordinate along the vertical axis of the coordinate system, and extend the minimum vertical coordinate and the maximum vertical coordinate along the horizontal axis of the coordinate system. During the extension process, the four lines can intersect and connect into a quadrilateral area.

[0216] Furthermore, the terminal device can determine the above-mentioned quadrilateral area as the largest abdominal area.

[0217] Continue to combine Figure 9 , it can be seen that Figure 9 The quadrilateral area formed in the middle is the largest abdominal area.

[0218] The electronic device obtains the largest abdominal area and can more accurately calculate the correlation value through the abdominal area. Therefore, the electronic device can determine the respiratory frequency of the target body and judge whether the target body is breathing normally based on the accurate correlation value data.

[0219] In the present application, the electronic device can obtain the key points of the neck, the left shoulder, the right shoulder, and the middle hip through key point detection, and determine the key points of the middle abdomen on the line connecting the key points of the neck and the middle hip, thereby preparing data for determining the key points of the right abdomen and the left abdomen on both sides of the key point of the middle abdomen. The electronic device can determine the key points of the right abdomen on the side of the key point of the middle abdomen close to the key point of the right shoulder, and determine the key points of the left abdomen on the side of the key point of the middle abdomen close to the key point of the left shoulder. The electronic device can quickly obtain the key points of the right abdomen and the left abdomen, thereby determining the abdominal area based on the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen. In addition, the electronic device can also determine the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate, and the maximum vertical coordinate from the coordinates of the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen, forming a quadrilateral area, and use this area as the largest abdominal area. Thus, the electronic device can more accurately calculate the relevant value through the abdominal area based on the largest abdominal area.

[0220] Illustratively, the present application further provides a respiratory monitoring device, which is applied to an electronic device.

[0221] Below, with reference to the drawings, a respiratory monitoring device provided in one embodiment of the present application is described in detail.

[0222] See also Figure 10 , Figure 10 A schematic block diagram of a respiratory monitoring device provided in an embodiment of the present application is shown.

[0223] like Figure 10 As shown, the respiratory monitoring device provided by an embodiment of the present application includes an acquisition module 401 and a determination module 402.

[0224] An acquisition module 401 is configured to acquire, from a plurality of frames of images acquired continuously, respective corresponding abdominal regions, each abdominal region including at least the abdomen of the target body;

[0225] a determination module 402 configured to determine a plurality of correlation values ​​for each abdominal region, the plurality of correlation values ​​for any abdominal region being used to represent a correlation between the any abdominal region and each of a preset number of abdominal regions, the preset number of abdominal regions including the any abdominal region and a plurality of abdominal regions acquired consecutively after the any abdominal region, the preset number being greater than or equal to a total number of image frames that can be acquired by completing multiple breaths;

[0226] The determination module 402 is further configured to divide the multiple correlation values ​​of each abdominal region into multiple groups of correlation values, each group of correlation values ​​being used to represent a breath completed by the target body;

[0227] The determination module 402 is further configured to determine whether the target body is breathing normally based on the multiple sets of correlation values.

[0228] In some embodiments, the determination module 402 is specifically configured to:

[0229] A plurality of respiratory frequencies are determined according to the plurality of sets of correlation values, and each respiratory frequency is used to indicate whether the target body is breathing normally or breathing abnormally.

[0230] Continue to combine Figure 10 , the respiratory monitoring device 400 is Figure 10 On the basis of the structure shown, it may further include: a display module 403. Figure 10 In FIG, the display module 403 is represented by a dotted line.

[0231] The display module 403 is used to:

[0232] A respiratory frequency curve is drawn according to each respiratory frequency and the duration of multiple frames of images. The respiratory frequency curve is used to indicate whether the target is in an exhalation state or an inhalation state.

[0233] In some embodiments, the determination module 402 is specifically configured to:

[0234] Determine the correlation value at a preset position in each group of correlation values ​​as a target correlation value;

[0235] determining a first abdominal region and a second abdominal region associated with a target correlation value in each set of correlation values;

[0236] determining a plurality of target-related values ​​corresponding to each identical first abdominal region;

[0237] For multiple target correlation values ​​corresponding to any same first abdominal region, two second abdominal regions associated with any two target correlation values ​​are respectively determined, and a respiratory frequency is obtained according to the frame number difference between the two second abdominal regions.

[0238] In some embodiments, when any same first abdominal region corresponds to multiple target correlation values, each group of correlation values ​​to which the multiple target correlation values ​​belong is used to represent normal breathing of the target body;

[0239] When any identical first abdominal region corresponds to a target correlation value, a group of correlation values ​​to which the target correlation value belongs is used to represent abnormal breathing of the target body.

[0240] In some embodiments, the display module 403 is specifically configured to:

[0241] When it is determined that the target body has abnormal breathing, a warning message indicating abnormal breathing of the target body is displayed.

[0242] In some embodiments, the acquisition module 401 is specifically configured to:

[0243] Acquire multiple frames of images from the target's breathing video;

[0244] Perform key point detection on each frame image to obtain the key points of the neck, left shoulder, right shoulder and mid-hip;

[0245] Determine the key points of the right abdomen and the left abdomen based on the key points of the neck, left shoulder, right shoulder, and mid-hip.

[0246] Determine the abdomen area based on the keypoints of the left shoulder, the right shoulder, the right abdomen, and the left abdomen.

[0247] In some embodiments, the acquisition module 401 is specifically configured to:

[0248] On the line between the key point of the neck and the key point of the mid-hip, determine the position with a preset distance from the key point of the mid-hip as the key point of the mid-abdomen;

[0249] Determine the key point of the right abdomen on the side of the key point of the midsection close to the key point of the right shoulder. The distance between the key point of the midsection and the key point of the right abdomen is equal to the distance between the key point of the neck and the key point of the right shoulder.

[0250] On the side of the midsection keypoint close to the left shoulder keypoint, determine the keypoint on the left abdomen, where the distance between the midsection keypoint and the left abdomen keypoint is equal to the distance between the neck keypoint and the left shoulder keypoint.

[0251] In some embodiments, the acquisition module 401 is specifically configured to:

[0252] Project the key points of the left shoulder, right shoulder, right abdomen, and left abdomen into the same coordinate system;

[0253] Determine, according to the coordinate system, the minimum abscissa, the maximum abscissa, the minimum ordinate, and the maximum ordinate among the coordinates of the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen;

[0254] Extend the minimum horizontal coordinate and the maximum horizontal coordinate along the vertical axis of the coordinate system, and extend the minimum vertical coordinate and the maximum vertical coordinate along the horizontal axis of the coordinate system to form a quadrilateral area;

[0255] The quadrilateral area is determined as the largest abdominal area.

[0256] It should be understood that the respiratory monitoring device 400 of the present application can be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), which can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The respiratory monitoring method in the above embodiment can also be implemented using software. When the respiratory monitoring method in the above embodiment is implemented using software, the respiratory monitoring device 400 and its various modules can also be software modules.

[0257] For example, the present application also provides a structural diagram of an electronic device. Figure 11 As shown, the specific implementation of the electronic device can refer to the description of the above electronic device, which can execute the above respiratory monitoring method.

[0258] The electronic device 500 includes a processor 501, a memory 502, a communication interface 503, and a bus 504. The processor 501, the memory 502, and the communication interface 503 communicate via the bus 504, and communication can also be achieved through other means such as wireless transmission. The memory 502 is used to store instructions, and the processor 501 is used to execute the instructions stored in the memory 502. The memory 502 stores program code 5021, and the processor 501 can call the program code 5021 stored in the memory 502 to execute the respiratory monitoring method in the above embodiment.

[0259] It should be understood that in the present application, the processor 501 may be a CPU, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0260] The memory 502 may include a read-only memory and a random access memory, and provides instructions and data to the processor 501. The memory 502 may also include a non-volatile random access memory. The memory 502 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0261] In addition to the data bus, the bus 504 may also include a power bus, a control bus, and a status signal bus. Figure 11 Various buses are labeled as bus 504.

[0262] It should be understood that the electronic device 500 of the present application may correspond to the electronic device in the above-mentioned embodiments of the present application. When the electronic device 500 corresponds to the electronic device in the above-mentioned embodiments, the above-mentioned and other operations and / or functions of each module in the electronic device 500 are respectively the operating steps of the method executed by the electronic device in the above-mentioned embodiments. For the sake of brevity, they will not be repeated here.

[0263] Illustratively, the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0264] Illustratively, the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to implement the steps in the above-mentioned various method embodiments.

[0265] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0266] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0267] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0268] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0269] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0270] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the above modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0271] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0272] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A respiratory monitoring method, characterized in that: include: Obtaining respective corresponding abdominal regions from the multiple frames of images acquired continuously, each abdominal region at least including the abdomen of the target body; determining a plurality of correlation values ​​for each abdominal region, the plurality of correlation values ​​for any one abdominal region being used to represent a degree of linear correlation between the any one abdominal region and each of a preset number of abdominal regions, the preset number of abdominal regions including the any one abdominal region and a plurality of abdominal regions successively acquired after the any one abdominal region, the preset number being greater than or equal to a total number of image frames that can be acquired by completing the plurality of breaths; Dividing the multiple correlation values ​​of each abdominal region to obtain multiple groups of correlation values, each group of correlation values ​​is used to represent a breath completed by the target body; determining whether the target body is breathing normally based on the multiple sets of correlation values; The formula for any one of the multiple correlation values ​​is: Among them, A mn is a matrix of the abdominal region, is the average value of the abdominal region matrix, B mn is another matrix in the abdominal area, is the average of another abdominal region matrix.

2. The method according to claim 1, wherein The method further comprises: A plurality of respiratory frequencies are determined according to the plurality of sets of correlation values, and each respiratory frequency is used to indicate whether the target body is breathing normally or breathing abnormally.

3. The method according to claim 2, wherein After determining a plurality of respiratory frequencies according to the plurality of sets of correlation values, the method further comprises: A respiratory frequency curve is drawn according to each respiratory frequency and the duration of the multiple frames of images, and the respiratory frequency curve is used to indicate whether the target body is in an exhalation state or an inhalation state.

4. The method according to claim 2, wherein Determining a plurality of respiratory frequencies according to the plurality of sets of correlation values ​​comprises: Determine the correlation value at a preset position in each group of correlation values ​​as a target correlation value; determining a first abdominal region and a second abdominal region associated with the target correlation value in each set of correlation values; determining a plurality of target correlation values ​​corresponding to each identical first abdominal region; For multiple target correlation values ​​corresponding to any same first abdominal region, two second abdominal regions associated with any two target correlation values ​​are respectively determined, and a respiratory frequency is obtained based on the frame number difference between the two second abdominal regions.

5. The method according to claim 4, wherein When any one of the same first abdominal regions corresponds to multiple target correlation values, each group of correlation values ​​to which the multiple target correlation values ​​belong is used to represent normal breathing of the target body; When any identical first abdominal region corresponds to a target correlation value, a group of correlation values ​​to which the target correlation value belongs is used to represent abnormal breathing of the target body.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: When it is determined that the target body has abnormal breathing, warning information indicating abnormal breathing of the target body is displayed.

7. The method according to any one of claims 1 to 5, characterized in that The step of obtaining the corresponding abdominal regions from the continuously acquired multiple frames of images includes: Acquiring the multiple frames of images from the breathing video of the target body; Perform key point detection on each frame image to obtain the key points of the neck, left shoulder, right shoulder and mid-hip; Determine key points of the right abdomen and key points of the left abdomen based on the key points of the neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the mid-hip; The abdomen region is determined based on the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen, and the key points of the left abdomen.

8. The method according to claim 7, wherein Determining the key points of the right abdomen and the key points of the left abdomen based on the key points of the neck, the key points of the left shoulder, the key points of the right shoulder, and the key points of the mid-hip, includes: On the line connecting the key point of the neck and the key point of the mid-hip, a position with a preset distance from the key point of the mid-hip is determined as the key point of the mid-abdomen; Determine the key point of the right abdomen on a side of the key point of the mid-abdomen close to the key point of the right shoulder, wherein the distance between the key point of the mid-abdomen and the key point of the right abdomen is equal to the distance between the key point of the neck and the key point of the right shoulder; The key point of the left abdomen is determined on the side of the key point of the middle abdomen close to the key point of the left shoulder, wherein the distance between the key point of the middle abdomen and the key point of the left abdomen is equal to the distance between the key point of the neck and the key point of the left shoulder.

9. The method according to claim 7, wherein The determining of the abdominal region according to the key points of the left shoulder, the key points of the right shoulder, the key points of the right abdomen, and the key points of the left abdomen includes: Projecting the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen into the same coordinate system; Determine, according to the coordinate system, the minimum abscissa, the maximum abscissa, the minimum ordinate, and the maximum ordinate among the coordinates of the key points of the left shoulder, the right shoulder, the right abdomen, and the left abdomen; Extending the minimum horizontal coordinate and the maximum horizontal coordinate along the vertical axis of the coordinate system, and extending the minimum vertical coordinate and the maximum vertical coordinate along the horizontal axis of the coordinate system, to form a quadrilateral area; The quadrilateral area is determined as the largest abdominal area.

10. A respiratory monitoring device, characterized in that: include: An acquisition module is used to acquire the corresponding abdominal regions from the multiple frames of images acquired continuously, each abdominal region at least including the abdomen of the target body; a determination module, configured to determine multiple correlation values ​​for each abdominal region, the multiple correlation values ​​for any one abdominal region being used to represent a degree of linear correlation between the any one abdominal region and each of a preset number of abdominal regions, the preset number of abdominal regions including the any one abdominal region and multiple abdominal regions acquired consecutively after the any one abdominal region, the preset number being greater than or equal to a total number of image frames that can be acquired by completing multiple breaths; The determination module is further configured to divide the multiple correlation values ​​of each abdominal region into multiple groups of correlation values, each group of correlation values ​​being used to represent a breath completed by the target body; a determination module, further configured to determine whether the target body is breathing normally based on the multiple sets of correlation values; The formula for any one of the multiple correlation values ​​is: Among them, A mn is a matrix of the abdominal region, is the average value of the abdominal region matrix, B mn is another matrix in the abdominal area, is the average of another abdominal region matrix.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images