Respiration rate detection method and device and intelligent terminal

By acquiring and analyzing respiratory motion images, and using the face detection model to identify and screen feature points, the existing contact detection problem is solved, and the insensitive and highly accurate respiratory frequency detection is achieved.

CN119055222APending Publication Date: 2024-12-03SHANGHAI HUANXIU TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411165915.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing respiratory rate detection methods are mainly contact detection, which can easily cause discomfort for users, and signal interference, improper wearing of sensors or low performance leads to inaccurate calculation results.

Method used

By obtaining the breathing motion images of the target detection object in successive frames, the trained face detection model recognizes the target area of ​​the breathing motion image, filters out the target feature points, determines the breathing frequency, realizes sensory detection and improves accuracy.

Benefits of technology

While realizing sensorless detection, it improves the accuracy of breathing frequency, avoids signal interference and sensor wearing problems, and provides more accurate breathing frequency data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119055222A_ABST
    Figure CN119055222A_ABST
Patent Text Reader

Abstract

The invention provides a respiratory rate detection method and device and an intelligent terminal, and is applied to the technical field of medical detection. The method comprises the following steps: acquiring continuous frames of breathing motion images of a target detection object; and utilizing the trained face detection model to identify a target area of the breathing motion image. The breathing motion image is the breathing motion image of the target detection object in the previous frame. And screening out a target feature point from the target area. The target feature point is determined based on a previous frame of breathing motion image of the target detection object. And determining the respiratory rate of the target detection object by using the target feature point. The accuracy of respiratory rate detection can be improved while non-inductive detection is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of medical detection technology, and in particular to a respiratory rate detection method and device, and an intelligent terminal. Background Art

[0002] Respiratory rate is a very important physiological parameter of the human body. Changes in human breathing can reflect the health level of the body. As one of the basic indicators for evaluating respiratory diseases, respiratory rate is a key indicator for judging respiratory dysfunction and a major indicator for clinical medical monitoring. Detecting human respiratory rate has great medical significance.

[0003] At present, the human respiratory rate detection method is mainly contact detection. For example, the human respiratory rate detection method can be to use a respiratory rate chest strap tied to the human chest, and calculate the human respiratory rate by detecting the rise and fall of the chest. However, this method is a sensory detection, which can easily cause physical discomfort to the user. The human respiratory rate detection method can also use sensors such as blood oxygen clips and smart watch bracelets to indirectly calculate the respiratory rate by detecting the PPG signal of the human body contact sensor. However, in the process of calculating the respiratory rate, there may be problems such as signal interference, improper sensor wearing, or low sensor performance, resulting in inaccurate respiratory rate calculation results.

[0004] Based on this, the present application provides a respiratory rate detection method for achieving non-contact detection while improving the accuracy of respiratory rate detection. Summary of the invention

[0005] The purpose of this application is to provide a respiratory rate detection method and device, and an intelligent terminal to achieve non-contact detection while improving the accuracy of respiratory rate detection.

[0006] In a first aspect, an embodiment of the present application provides a breathing rate detection method, the method comprising: obtaining respiratory motion images of continuous frames of a target detection object. Using a trained face detection model, identifying a target area of ​​the respiratory motion image. The respiratory motion image is a respiratory motion image of the target detection object in a previous frame. Filtering out target feature points from the target area. The target feature points are determined based on the respiratory motion image of the target detection object in a previous frame. Using the target feature points, determining the respiratory frequency of the target detection object.

[0007] The respiratory rate detection method provided in the embodiment of the present application obtains continuous frames of respiratory motion images of the target detection object, screens out target feature points from the target area of ​​the respiratory motion images, and uses the target feature points to determine the respiratory frequency of the target detection object. This can achieve non-sensing detection of the target detection object, and at the same time, by directly capturing the target's respiratory motion images, it can directly obtain the target detection object's respiratory information, and can obtain a more accurate respiratory frequency.

[0008] A possible implementation method uses target feature points to determine the respiratory frequency of the target detection object, including: determining the sample value of the respiratory signal of the target detection object in the current frame according to the target feature points. The sample value is used to represent the displacement of the target feature point relative to the respiratory motion image of the previous frame. When the sample value meets the preset conditions, the respiratory signal of the current frame is spliced ​​to the historical signal of the target detection object to obtain the target respiratory signal. The respiratory frequency is determined according to the target respiratory signal.

[0009] In a possible implementation, the breathing rate detection method provided in the embodiment of the present application further includes: calculating the absolute value of the first-order derivative of the sample value. When the absolute value is greater than an abnormal threshold, determining that the breathing signal corresponding to the sample value is an abnormal signal. The abnormal threshold is determined based on the sample value of the historical signal of the preset time length of the target detection object. The motion breathing signal of the current frame is eliminated.

[0010] A possible implementation method is to determine the sample value of the breathing signal of the target detection object in the current frame according to the target feature point, including: obtaining the cumulative sum of the average displacement of the target feature point in the previous frame. Calculating the average vertical displacement of the target feature point in the current frame relative to the previous frame. The average displacement and the cumulative sum of the average displacement are summed to obtain the sample value.

[0011] A possible implementation method is to determine the respiratory frequency according to the target respiratory signal, including: filtering the target respiratory signal, performing window transformation on the filtered target respiratory signal through a window function, and extracting the signal main frequency of the target respiratory signal by Fourier transformation, and determining the respiratory frequency according to the signal main frequency.

[0012] A possible implementation method is to use the trained face detection model to identify the target area of ​​the respiratory motion image, including: using a preset face recognition frame to identify the face area in the respiratory motion image. Based on the preset position of the face area and the preset position of the respiratory motion image, the target area is determined.

[0013] A possible implementation method is to filter out target feature points from a target area, including: in the target area, using a detection algorithm to determine a feature point set of a respiratory motion image. The feature point set is a point set suitable for optical flow calculation. The feature point set is a set of points in a previous frame of the respiratory motion image. For any feature point in the feature point set, optical flow calculation is performed on the feature point to obtain a displacement change of the feature point in the respiratory motion image. According to the displacement change, the target feature point in the feature point set is determined.

[0014] A possible implementation method is to perform optical flow calculation on the feature point to obtain the displacement change of the feature point in the respiratory motion image, including: obtaining the first position information of the feature point in the previous frame. Based on the first position information, determining the second position information of the first position information relative to the current frame. Based on the second position information, determining the third position information of the first position information relative to the previous frame. Based on the first position information and the third position information, determining the displacement change of the feature point.

[0015] A possible implementation manner is to determine a target feature point in a feature point set according to a displacement change, including: when the displacement change is less than a preset displacement change, determining the feature point as a target feature point.

[0016] In a second aspect, an embodiment of the present application provides a respiratory rate detection device, which includes: an acquisition module, an identification module, a screening module, and a determination module.

[0017] The acquisition module is used to acquire continuous frames of respiratory motion images of the target detection object.

[0018] The recognition module is used to recognize the target area of ​​the respiratory motion image using the trained face detection model. The respiratory motion image is the respiratory motion image of the target detection object in the previous frame.

[0019] The screening module is used to screen out target feature points from the target area. The target feature points are determined based on the respiratory motion image of the previous frame of the target detection object.

[0020] The determination module is used to determine the breathing frequency of the target detection object using the target feature points.

[0021] In a third aspect, the present application provides an intelligent terminal having the function of implementing the respiratory rate detection method of the first aspect or any possible implementation method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0022] In a fourth aspect, a computer-readable storage medium is provided, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer can execute the respiratory rate detection method of the first aspect or any possible implementation method.

[0023] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the respiratory rate detection method of the first aspect or any possible implementation manner.

[0024] Among them, the technical effects brought about by any design method in the second to fifth aspects can refer to the technical effects brought about by different possible implementation methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of the structure of an intelligent terminal provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of the structure of a respiratory rate detection system provided in an embodiment of the present application;

[0028] Figure 3 A schematic flow chart of a respiratory rate detection method provided in an embodiment of the present application;

[0029] Figure 4 A specific example diagram of the target area provided in the embodiment of the present application;

[0030] Figure 5 A specific example diagram of a respiratory signal segment without eliminating abnormal signals provided in an embodiment of the present application;

[0031] Figure 6 A specific example diagram of a respiratory signal segment for removing abnormal signals provided in an embodiment of the present application;

[0032] Figure 7 A spectrum diagram generated based on a respiratory signal segment without eliminating abnormal signals provided in an embodiment of the present application;

[0033] Figure 8 A spectrum diagram generated based on a respiratory signal segment without eliminating abnormal signals provided in an embodiment of the present application;

[0034] Fig. 9 A specific example diagram of a respiratory rate detection method provided in an embodiment of the present application;

[0035] Fig.10 A schematic diagram of the structure of a respiratory rate detection device provided in an embodiment of the present application;

[0036] Fig.11 Another structural schematic diagram of a respiratory rate detection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0038] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0039] At present, the conventional respiratory rate detection technology is contact detection. For example, the human respiratory rate detection method can be to use a respiratory rate chest strap tied to the human chest, and calculate the human respiratory rate by detecting the rise and fall of the chest. However, this method is a sensory detection, which can easily cause physical discomfort to the user. The human respiratory rate detection method can also use sensors such as blood oxygen clips and smart watch bracelets to indirectly calculate the respiratory rate by detecting the PPG signal of the human body contact sensor. However, in the process of calculating the respiratory rate, there may be problems such as signal interference, improper sensor wearing, or low sensor performance, resulting in inaccurate respiratory rate calculation results.

[0040] Based on this, the embodiments of the present application provide a breathing rate detection method and device, and an intelligent terminal, the basic principle of which is: obtain continuous frames of respiratory motion images of the target detection object. Use the trained face detection model to identify the target area of ​​the respiratory motion image. The respiratory motion image is the respiratory motion image of the target detection object in the previous frame. Filter out target feature points from the target area. The target feature points are determined based on the respiratory motion image of the previous frame of the target detection object. Use the target feature points to determine the respiratory frequency of the target detection object.

[0041] The respiratory rate detection method provided in the embodiment of the present application obtains continuous frames of respiratory motion images of the target detection object, screens out target feature points from the target area of ​​the respiratory motion images, and uses the target feature points to determine the respiratory frequency of the target detection object. This can achieve non-sensing detection of the target detection object, and at the same time, by directly capturing the target's respiratory motion images, it can directly obtain the target detection object's respiratory information, and can obtain a more accurate respiratory frequency.

[0042] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0043] On the one hand, a respiratory rate detection method provided in an embodiment of the present application can be executed by a smart terminal. Figure 1 As shown, the smart terminal 100 may include: an image sensor 101 , a processor 102 , and a memory 103 . The memory 103 may be a memory in the processor 102 .

[0044] The image sensor 101 may be a complementary metal-oxide-semiconductor (CMOS), or a charge-coupled device (CCD), or an image sensor that is a combination of CMOS and CCD.

[0045] The image sensor 101 may be used to obtain respiratory motion images of successive frames of the target detection object, convert the respiratory motion images into respiratory motion image signals, and send the respiratory motion image signals to the processor 102 .

[0046] The processor 102 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0047] The processor 102 can be used to determine the respiratory frequency of the target detection object based on the respiratory motion image signal sent by the image sensor 101 through the respiratory rate detection method provided in the embodiment of the present application.

[0048] The memory 103 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct rambus RAM (DRRAM) or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory can be independent and connected to the processor 102 via a communication line. The memory 103 can also be integrated with the processor 102.

[0049] The memory 103 stores computer executable instructions for implementing the solution of the present application, and is executed under the control of the processor 102. The processor 102 is used to execute the computer executable instructions stored in the memory 103, thereby implementing the respiratory rate detection method provided in the following embodiments of the present application.

[0050] On the one hand, the respiratory rate detection method provided in the embodiment of the present application can be performed by a respiratory rate detection system. Figure 2 As shown, the respiratory rate detection system 200 may include: a smart terminal 100 and a cloud 201 .

[0051] The smart terminal 100 may be, but is not limited to, a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and no limitation is imposed on the specific type of the smart terminal.

[0052] The intelligent terminal 100 is used to support the respiratory rate detection method provided in the embodiment of the present application, and obtain the respiratory motion images of the target detection object in continuous frames. The target area of ​​the respiratory motion image is identified using the trained face detection model. The respiratory motion image is the respiratory motion image of the target detection object in the previous frame. Target feature points are selected from the target area. The target feature points are determined based on the respiratory motion image of the target detection object in the previous frame. The respiratory frequency of the target detection object is determined using the target feature points.

[0053] The smart terminal 100 can also be used to provide interactive functions for users. Users can input data and information to the smart terminal 100 or view data and information through the interactive interface of the smart terminal 100. For example, input a respiratory motion image, or view the respiratory rate, etc.

[0054] The cloud 201 may be a resource cluster shared by multiple smart terminals 100. The resource cluster can provide multiple algorithms for use by the smart terminals 100. The resource cluster may belong to a public cloud or a private cloud. In the case of a public cloud, the resource cluster may be used by different company users and individual users. In the case of a private cloud, the resource cluster generally only serves the company users described in the private cloud.

[0055] On the one hand, the embodiment of the present application discloses a method for detecting respiratory rate, which can be deployed by Figure 1 The smart terminal shown in Figure 2 The respiratory rate detection system shown is implemented. Figure 3 As shown, the method may include the following steps.

[0056] S301, acquiring respiratory motion images of consecutive frames of a target detection object.

[0057] Among them, the respiratory motion images of continuous frames can be in the form of real-time video to be detected, historical video, etc. of the target detection object, which is not limited in this application.

[0058] In a possible implementation, the target detection object uses an intelligent terminal to collect continuous frames of the user's breathing motion images in real time through an image sensor.

[0059] Exemplarily, the target detection subject places the mobile terminal on a bracket on a desktop, sits on a chair and remains still, and uses the front camera of the mobile terminal to aim at the target detection subject to obtain continuous frame breathing motion images of the target detection subject.

[0060] Another possible implementation manner is that the target detection object uploads the historically collected continuous frames of respiratory motion images to the smart terminal, so that the smart terminal obtains the continuous frames of respiratory motion images of the target detection object.

[0061] S302, using the trained face detection model to identify the target area of ​​the respiratory motion image.

[0062] The respiratory motion image is a respiratory motion image of the target detection object in the previous frame. The face detection model can be an opencv model.

[0063] In a possible implementation, a preset face recognition frame is used to recognize a face region in the respiratory motion image, and a target region is determined based on a preset position of the face region and a preset position of the respiratory motion image.

[0064] For example, Figure 4 As shown, Figure 4 is a specific example diagram of the target area. The preset position of the face area can be the lower edge position of the face recognition frame. The preset position of the preset respiratory motion image can be the lower edge position of the respiratory motion image. The range area formed by the lower edge position of the face recognition frame and the lower edge position of the respiratory motion image is the target area.

[0065] S303: Filter out target feature points from the target area.

[0066] The target feature points are determined based on the respiratory motion image of the previous frame of the target detection object.

[0067] In a possible implementation, a detection algorithm is used in the target area to determine a set of points in the respiratory motion image suitable for optical flow calculation, and the set of feature points is formed. The feature point set is a set of points in the previous frame of the respiratory motion image.

[0068] Exemplarily, by specifying the parameters of feature points in the opencv model, such as the maximum number of feature points, the quality level of feature points, and the minimum distance between feature points, the function goodFeaturesToTrack in the opencv model is used to detect feature points in the target area suitable for optical flow calculation based on the above parameters.

[0069] Furthermore, for any feature point in the feature point set, optical flow calculation is performed on the feature point to obtain the displacement change of the feature point in the respiratory motion image.

[0070] Specifically, the first position information of the feature point in the previous frame is obtained. Based on the first position information, the second position information of the first position information relative to the current frame is determined. Based on the second position information, the third position information of the first position information relative to the previous frame is determined. Based on the first position information and the third position information, the displacement change of the feature point is determined.

[0071] Exemplarily, taking feature point a in the feature point set as an example, for feature point a, obtain the first position information of feature point a in the previous frame of respiratory motion image. Use the function calcOpticalFlowPyrLK in the opencv model to perform forward optical flow calculation on the first position information of feature point a, and obtain the second position information of the first position information of feature point a relative to the current frame. Then use the function calcOpticalFlowPyrLK in the opencv model to perform backward optical flow calculation on the second position information of feature point 2, and obtain the third position information of the second position information of feature point a relative to the previous frame. According to the first position information and the third position information, the displacement change of feature point a is confirmed.

[0072] Then, according to the displacement change, the target feature point in the feature point set is determined.

[0073] Exemplarily, when the displacement change is less than a preset displacement change, the feature point is determined to be a target feature point. The number of target feature points may be one or more, and the present application does not limit the number of target feature points.

[0074] The preset displacement transformation may be a distance of two pixels, which is not limited in the present application.

[0075] S304, using the target feature points, determining the breathing frequency of the target detection object.

[0076] A possible implementation method is to determine the sample value of the breathing signal of the target detection object in the current frame according to the target feature point. The sample value is used to represent the displacement of the target feature point relative to the respiratory motion image of the previous frame. When the sample value meets the preset conditions, the breathing signal of the current frame is spliced ​​to the historical signal of the target detection object to obtain the target breathing signal. According to the target breathing signal, the breathing frequency is determined.

[0077] Specifically, the average displacement accumulation sum of the target feature point in the previous frame is obtained. The average displacement accumulation sum is the accumulation sum of the average displacement of the target feature point in the previous frame based on all frames before the target feature point.

[0078] For any target feature point among all target feature points, the fourth position information of the target feature point in the current frame is obtained, and the vertical displacement of the target feature point is calculated according to the first position information and the fourth position information of the target feature point. The average value of the vertical displacements of all target feature points is calculated to obtain the average vertical displacement.

[0079] Then add the average displacement accumulation sum of the previous frame of the target feature point to the average vertical displacement to obtain the sample value of the breathing signal of the current frame. Calculate the absolute value of the first-order derivative of the sample value. When the absolute value is greater than the abnormal threshold, the breathing signal corresponding to the sample value is determined to be an abnormal signal. The abnormal threshold is determined based on the sample value of the historical signal of the preset time length of the target detection object. When the breathing signal is an abnormal signal, the motion breathing signal of the current frame is eliminated.

[0080] When the absolute value is not greater than the abnormal threshold, the respiratory signal corresponding to the sample value is determined to be a normal signal, and the normal signal is spliced ​​to the previous normal respiratory signal.

[0081] For example, Figure 5 and Figure 6 As shown, Figure 5 is a specific example diagram of a respiratory signal segment without removing abnormal signals. Figure 5 The dotted line frame in the figure represents an abnormal signal. After the abnormal signal is detected, the abnormal signal is removed and the next normal breathing signal is spliced ​​to the previous normal breathing signal to obtain Figure 6 The figure is a specific example of a respiratory signal segment from which abnormal signals are removed.

[0082] It should be noted that the process of splicing the next normal breathing signal to the previous normal breathing signal can be to directly remove the abnormal signal when the breathing signal of the current frame is detected to be abnormal, or to retain the abnormal signal and mark it, which is not limited in this application. Figure 5 and Figure 6 It is only used to represent the motion breathing signal segment without removing abnormal signals and the breathing signal segment after removing abnormal signals, and is not a limitation on the breathing signal generated by the present application.

[0083] Furthermore, after generating a normal breathing signal, the intelligent terminal filters the target breathing signal, performs window transformation on the filtered target breathing signal through a window function, and uses Fourier transform to extract the signal main frequency of the target breathing signal. According to the signal main frequency, the breathing frequency is determined.

[0084] Exemplarily, a second-order Butterworth bandpass filter is used to filter the respiratory signal to remove abnormal segments, and the cutoff frequency of the bandpass filter is 0.1 to 1 Hz. The respiratory signal is transformed using a window function (Hamming window), and the corresponding signal main frequency is obtained using Fourier transform (FFT). The final respiratory rate is the signal main frequency multiplied by 60, that is, the number of breaths per minute.

[0085] The respiratory rate detection method provided in the embodiment of the present application obtains continuous frames of respiratory motion images of the target detection object, screens out target feature points from the target area of ​​the respiratory motion images, and uses the target feature points to determine the respiratory frequency of the target detection object. This can achieve non-sensing detection of the target detection object, and at the same time, by directly capturing the target's respiratory motion images, it can directly obtain the target detection object's respiratory information, and can obtain a more accurate respiratory frequency.

[0086] For example, Figure 7 As shown, Figure 7 A spectrum diagram generated based on a respiratory signal segment without removing abnormal signals, such as Figure 5 The main frequency of the signal without removing the abnormal signal is 0.538Hz, and the corresponding breathing rate is 32bpm. Figure 8 As shown, Figure 8 A spectrum diagram generated based on a respiratory signal segment with abnormal signals removed, for example Figure 6 . The main frequency of the signal after the abnormal signal is removed is 0.4Hz, and the corresponding respiratory rate is 24bpm. The actual respiratory rate of the target detection object is 24bpm. It can be seen that the presence of abnormal signals will make the evaluation of the respiratory rate too high, thereby causing inaccurate respiratory rate detection results. By executing the respiratory rate detection method provided in the embodiment of the present application, after the abnormal signal in the respiratory signal is removed, a more accurate respiratory rate can be obtained.

[0087] Furthermore, the respiratory rate detection method provided in the embodiment of the present application can also be used to provide a respiratory rate assessment for a target detection object.

[0088] Specifically, by providing a specific breathing exercise for the target detection object, taking 4-3-4 (inhale for 4 seconds, hold the breath for 3 seconds and exhale for 4 seconds) as an example, the target detection object can perform breathing exercises following the audio of the guided breathing exercise. Fig. 9As shown, by comparing the measured breathing signal with the standard breathing signal and combining the following indicators: Dynamic Time Warping (DTW), cross correlation, mean square error and Pearson correlation coefficient, the delay and phase difference that may exist between the measured breathing signal and the standard breathing signal are eliminated, and then the practice accuracy of the target detection object is evaluated to provide breathing suggestions for the target detection object.

[0089] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the working principle of the device. It is understandable that in order to realize the above functions, the intelligent terminal includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0090] The embodiment of the present application can divide the functional modules of the intelligent terminal according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or software functional module.

[0091] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. Fig.10 FIG. 2 shows a possible schematic diagram of the composition of the respiratory rate detection device involved in the above-mentioned embodiments. Fig.10 As shown, the respiratory rate detection device 1000 may include: an acquisition module 1001, an identification module 1002, a screening module 1003, and a determination module 1004.

[0092] The acquisition module 1001 is used to support the respiratory rate detection device 1000 to execute Figure 3 S301 in the respiratory rate detection method shown.

[0093] Identification module 1002, used to support the respiratory rate detection device 1000 to perform Figure 3 S302 in the respiratory rate detection method shown.

[0094] The screening module 1003 is used to support the respiratory rate detection device 1000 to perform Figure 3 S303 in the respiratory rate detection method shown.

[0095] Determining module 1004, used to support the respiratory rate detection device 1000 to perform Figure 3 S304 in the respiratory rate detection method shown.

[0096] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0097] The respiratory rate detection device 1000 provided in the embodiment of the present application is used to perform the above Figure 3 The respiratory rate detection method shown can therefore achieve the same effect as the above-mentioned respiratory rate detection method.

[0098] The present application also provides a smart terminal, which can be the smart terminal described in the above embodiment. Figure 1 The smart terminal 100 shown can execute the respiratory rate detection method and related steps in the above method embodiment.

[0099] An embodiment of the present application also provides a computer-readable storage medium having instructions stored thereon, which, when executed, execute the respiratory rate detection method and related steps in the above method embodiment.

[0100] An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the respiratory rate detection method and related steps in the above method embodiment.

[0101] In some embodiments, the methods described herein may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or encoded on other non-transitory media or articles of manufacture.

[0102] The present application also provides a respiratory rate detection system 1100. Fig.11 As shown, the respiratory rate detection system 1100 includes at least one processor 1101 and at least one interface circuit 1102 .

[0103] As an example, when the respiratory rate detection system 1100 includes a processor and an interface circuit, the processor may be Fig.11 The processor 1101 shown in the solid line frame (or the processor 1101 shown in the dotted line frame) may be Fig.11 The interface circuit 1102 shown in the solid line frame (or the interface circuit 1102 shown in the dotted line frame). When the chip system 1100 includes two processors and two interface circuits, the two processors include Fig.11 The processor 1101 shown in the solid line frame and the processor 1101 shown in the dotted line frame, the two interface circuits include Fig.11 The interface circuit 1102 shown in the solid line frame and the interface circuit 1102 shown in the dotted line frame are not limited to this.

[0104] The processor 1101 and the interface circuit 1102 may be interconnected via a line. For example, the interface circuit 1102 may be used to receive a signal. In another example, the interface circuit 1102 may be used to send a signal to another device (e.g., the processor 1101). For example, the interface circuit 1102 may read a computer instruction stored in a memory and send the computer instruction to the processor 1101. The processor 1101 executes the instruction and implements the various steps in the above embodiments in combination with the input and output devices, such as implementing Figure 3 Fig. 9 Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0105] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0106] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0110] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A respiratory rate detection method, characterized in that: Applied to a smart terminal, the method comprises: Acquire continuous frames of respiratory motion images of the target detection object; Using the trained face detection model, identifying the target area of ​​the respiratory motion image; the respiratory motion image is the respiratory motion image of the target detection object in the previous frame; Filtering target feature points from the target area; the target feature points are determined based on a respiratory motion image of a previous frame of the target detection object; The target feature points are used to determine the breathing frequency of the target detection object.

2. The method according to claim 1, characterized in that The step of determining the breathing frequency of the target detection object by using the target feature points includes: Determine, according to the target feature point, a sample value of the breathing signal of the target detection object in the current frame; the sample value is used to represent the displacement of the target feature point relative to the breathing motion image of the previous frame; When the sample value meets a preset condition, the breathing signal of the current frame is spliced ​​to the historical signal of the target detection object to obtain a target breathing signal; The respiratory frequency is determined according to the target respiratory signal.

3. The method according to claim 2, characterized in that The method further comprises: Calculating the absolute value of the first-order derivative of the sample value; When the absolute value is greater than an abnormal threshold, the breathing signal corresponding to the sample value is determined to be an abnormal signal; the abnormal threshold is determined according to the sample value of the historical signal of the preset time length of the target detection object; The motion breathing signal of the current frame is removed.

4. The method according to claim 2, characterized in that: The step of determining the sample value of the breathing signal of the target detection object in the current frame according to the target feature point includes: Obtaining the cumulative sum of average displacements of the target feature point in the previous frame; Calculating an average vertical displacement of the target feature point in the current frame relative to the previous frame; The average displacement and the average displacement are accumulated and summed to obtain the sample value.

5. The method according to claim 2, characterized in that: Determining the respiratory frequency according to the target respiratory signal includes: filtering the target breathing signal; Performing a window transformation on the filtered target respiratory signal through a window function, and extracting the signal main frequency of the target respiratory signal through Fourier transformation; The breathing frequency is determined according to the main frequency of the signal.

6. The method according to claim 1, characterized in that The step of using the trained face detection model to identify the target area of ​​the respiratory motion image includes: Using a preset face recognition frame to recognize a face area in the respiratory motion image; The target area is determined based on the preset position of the face area and the preset position of the respiratory motion image.

7. The method according to claim 1, characterized in that The step of selecting target feature points from the target area includes: In the target area, a feature point set of the respiratory motion image is determined by using a detection algorithm; the feature point set is a point set suitable for optical flow calculation; the feature point set is a set of points of a previous frame of the respiratory motion image; For any feature point in the feature point set, performing optical flow calculation on the feature point to obtain a displacement change of the feature point in the respiratory motion image; The target feature point in the feature point set is determined according to the displacement change.

8. The method according to claim 7, characterized in that The performing optical flow calculation on the feature points to obtain displacement changes of the feature points in the respiratory motion image includes: Acquire first position information of the feature point in the previous frame; Determine, according to the first position information, second position information of the first position information relative to a current frame; Determine, according to the second position information, third position information of the first position information relative to the previous frame; The displacement change of the feature point is determined according to the first position information and the third position information.

9. The method according to claim 7, characterized in that: The step of determining the target feature point in the feature point set according to the displacement change includes: When the displacement change is less than a preset displacement change, the feature point is determined to be the target feature point.

10. A respiratory rate detection device, characterized in that: Applied to a smart terminal, the device comprises: An acquisition module, used for acquiring continuous frames of respiratory motion images of a target detection object; A recognition module, used to recognize the target area of ​​the respiratory motion image by using the trained face detection model; the respiratory motion image is the respiratory motion image of the target detection object in the previous frame; A screening module, used for screening out target feature points from the target area; the target feature points are determined based on a respiratory motion image of a previous frame of the target detection object; The determination module is used to determine the breathing frequency of the target detection object using the target feature points.

11. An intelligent terminal, characterized in that: The intelligent terminal includes a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the respiratory rate detection method according to any one of claims 1 to 9.