Human factor intelligence-based vital sign signal measurement method, device and apparatus

Abstract

The embodiments of the present specification provide a vital sign signal measurement method, device and equipment based on human factor intelligence. The method comprises: acquiring individual characteristic representation data of a measured object, environment characteristic representation data of an environment in which the measured object is located, and vital sign spectrum data of the measured object; wherein the individual characteristic representation data and the environment characteristic representation data have different influences on the vital sign spectrum data; performing signal value prediction based on the individual characteristic representation data, the environment characteristic representation data and the vital sign spectrum data to obtain a vital sign signal value removed from the different influences, so that the accuracy of vital sign signal measurement can be improved.

Description

TECHNICAL FIELD

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular, to a vital sign signal measurement method, device and equipment based on human factor intelligence. BACKGROUND

[0002] A vital sign signal is a set of medical parameters that can describe the health status and body function of an individual, which can include heart rate, respiratory rate, body temperature, blood pressure, etc.

[0003] In related technologies, when measuring a vital sign signal, a contact sensor such as an electrocardiogram monitor or a millimeter wave radar can be used for measurement, but using a contact sensor for measurement can interfere with the vital sign signal of the measured person, and needs to be calibrated and maintained regularly, which has certain limitations. Vital sign signal measurement based on millimeter wave radar is easily disturbed, and the accuracy of the measurement needs to be improved. SUMMARY

[0004] Therefore, the embodiments of the present specification are committed to providing a vital sign signal measurement method, device and equipment based on human factor intelligence to improve the accuracy of vital sign signal measurement.

[0005] The embodiments of the present specification provide a vital sign signal measurement method based on human factor intelligence, the method comprising: obtaining individual feature representation data of a measured object, environment feature representation data of an environment in which the measured object is located, and vital sign spectrum data of the measured object; wherein the individual feature representation data and the environment feature representation data have different influences on the vital sign spectrum data; performing signal value prediction based on the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain a vital sign signal value removed from the different influences.

[0006] In some embodiments, the individual feature representation data of the measured object is obtained by: obtaining biological feature data of the measured object based on a video signal; wherein the video signal is obtained by photographing an environment in which the measured object is located; and performing individual identification on the measured object based on the biological feature data in the video signal to obtain the individual feature representation data.

[0007] In some embodiments, the biological feature data of the measured object is any one of face data, iris data, retina data, and eyeprint data; and the individual feature representation data includes at least one of gender features, age features, and skin type features of the measured object.

[0008] In some embodiments, the environment feature representation data of the environment where the measured object is located is obtained by: obtaining environment data of the environment where the measured object is located based on a video signal; wherein the video signal is obtained by photographing the environment where the measured object is located; and performing feature extraction on the environment data in the video signal to obtain the environment feature representation data; wherein the environment feature representation data comprises at least one of humidity features, temperature features, weather features, and wind speed features.

[0009] In some embodiments, the vital sign spectrum data of the measured object is obtained by: obtaining a video signal and a digital mixing signal of a first measurement device; wherein the video signal is obtained by photographing the environment where the measured object is located; the digital mixing signal is determined based on the first measurement device transmitting and receiving a frequency-modulated continuous wave radar signal through the millimeter wave radar; determining an initial distance unit where the measured object is located relative to the first measurement device based on the digital mixing signal; correcting the initial distance unit based on the video signal to obtain a target distance unit; and determining the vital sign spectrum data based on the target distance unit.

[0010] In some embodiments, the target distance unit is obtained by correcting the initial distance unit based on the video signal, comprising: detecting a distance between the first measurement device and the measured object based on the video signal to obtain a video detection distance; and correcting the initial distance unit based on the video detection distance to obtain the target distance unit.

[0011] In some embodiments, the vital sign signal value is obtained by performing signal value prediction based on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data, comprising: performing feature jointing on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain a feature jointing result; and performing signal value prediction based on the feature jointing result to obtain the vital sign signal value; wherein the vital sign signal value comprises at least one of a heart rate and a respiration rate.

[0012] In some embodiments, the feature jointing result is obtained by performing feature jointing on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data, comprising: performing splicing processing on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain the feature jointing result.

[0013] In some embodiments, the acquiring the individual feature representation data of the measured object and the environment feature representation data of the environment in which the measured object is located comprises: acquiring a video signal obtained by photographing the environment in which the measured object is located; determining environment data and individual recognition results of the measured object based on the video signal; performing mapping processing on the environment data to obtain the environment feature representation data of a specified dimension; and performing mapping processing on the individual recognition results to obtain the individual feature representation data of the specified dimension.

[0014] In some embodiments, the vital sign signal value is output by a target classification model; the target classification model is obtained by the following training method: a training sample set is constructed; the training sample set includes a plurality of training samples, and each training sample includes historical vital sign spectrum data, historical individual feature representation data and historical environment feature representation data; a label of the training sample is a historical vital sign signal true value; an initial classification model is trained according to the training sample and the label to obtain the target classification model; and the initial classification model is built based on any one of VGG, EfficientNet and ResNet model structures.

[0015] In some embodiments, the training sample is constructed by the following method: historical vital sign spectrum data is determined based on a historical digital mixing signal collected by a first measuring device at a historical time for the measured object or a non-measured object; a historical video signal collected at the historical time is acquired; historical individual feature representation data and historical environment feature representation data are determined according to the historical video signal; and a historical vital sign signal true value collected by a second measuring device at the historical time for the measured object or the non-measured object is taken as a label.

[0016] In some embodiments, the second measuring device is different from the first measuring device; and the second measuring device is any one of a mechanical measuring device and a biological signal measuring device.

[0017] The embodiments of the present specification provide a vital sign signal measuring device based on human factor intelligence, the device comprising: a feature data acquisition module, configured to acquire individual feature representation data of a measured object, environment feature representation data of an environment in which the measured object is located, and vital sign spectrum data of the measured object; wherein the individual feature representation data and the environment feature representation data have different influences on the vital sign spectrum data; and a vital sign determination module, configured to perform signal value prediction based on the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain a vital sign signal value removed from the different influences.

[0018] The embodiment of the present specification provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the measurement method of any one of the above embodiments when executing the computer program.

[0019] The embodiment of the present specification provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the measurement method of any one of the above embodiments.

[0020] The plurality of embodiments provided by the present specification can improve the accuracy of the measurement of the vital sign signal by obtaining the individual feature representation data of the measured object, the environment feature representation data of the environment where the measured object is located, and the vital sign spectrum data of the measured object, and performing signal value prediction based on the individual feature representation data, the environment feature representation data and the vital sign spectrum data, so as to obtain the vital sign signal value which removes the influence of the individual feature representation data on the vital sign spectrum data and the influence of the environment feature representation data on the vital sign spectrum data. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1a A schematic diagram of a vital sign signal measurement system provided by the embodiment of the present specification is shown in the figure;

[0022] Figure 1b A schematic diagram of a vital sign signal measurement system provided by the embodiment of the present specification is shown in the figure;

[0023] Figure 2 A flowchart of a vital sign signal measurement method provided by the embodiment of the present specification is shown in the figure;

[0024] Figure 3 A flowchart of an individual feature representation data acquisition method provided by the embodiment of the present specification is shown in the figure;

[0025] Figure 4 A flowchart of an environment feature representation data acquisition method provided by the embodiment of the present specification is shown in the figure;

[0026] Figure 5 A flowchart of a vital sign spectrum data acquisition method provided by the embodiment of the present specification is shown in the figure;

[0027] Figure 6 A flowchart of a target distance unit determination method provided by the embodiment of the present specification is shown in the figure;

[0028] Figure 7a A flowchart of a vital sign signal measurement method provided by the embodiment of the present specification is shown in the figure;

[0029] Figure 7bAn environmental feature representation data acquisition method provided by the embodiment of the present specification is shown in the flowchart;

[0030] Figure 8a A target classification model training method provided by the embodiment of the present specification is shown in the flowchart;

[0031] Figure 8b A training sample construction method provided by the embodiment of the present specification is shown in the flowchart;

[0032] Figure 9 A schematic diagram of a vital sign signal measurement device provided by the embodiment of the present specification is shown in the flowchart;

[0033] Figure 10 A schematic diagram of a computer device provided by the embodiment of the present specification is shown in the flowchart. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the present specification, the technical solutions in the present specification will be described clearly and completely in conjunction with the accompanying drawings in the present specification. Obviously, the described embodiments are only part of the embodiments of the present specification, not all embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present specification.

[0035] In some related technologies, when measuring vital sign signals, contact sensors such as electrocardiogram monitors or millimeter wave radars are usually used for measurement. However, the use of contact sensors for measurement may interfere with the vital sign signals of the measured object, and requires regular calibration and maintenance, which has certain limitations. Vital sign signal measurement based on millimeter wave radar is easily disturbed, and its measurement accuracy needs to be improved.

[0036] Therefore, when measuring vital sign signals based on millimeter wave radar, it is necessary to provide a vital sign signal measurement method based on human factors, that is, to provide a vital sign signal measurement method that is different for different people, in order to measure the vital sign signals of different measured objects. First, individual feature representation data of the measured object, environmental feature representation data of the environment where the measured object is located, and vital sign spectrum data of the measured object are obtained. Then, signal value prediction is performed based on the individual feature representation data, the environmental feature representation data, and the vital sign spectrum data to obtain vital sign signal values that remove the influence of individual feature representation data and environmental feature representation data on vital sign spectrum data. In this way, the accuracy of measuring the vital sign signals of the measured object can be improved.

[0037] Please refer to Figure 1a ,Figure 1a A schematic diagram of a scenario example of a human factor intelligence based vital sign signal measurement system is provided for an embodiment of the present specification. The vital sign signal measurement system 100 can include a first measurement device 110, a video signal acquisition device 120, and a second measurement device 130.

[0038] Specifically, the first measurement device 110 can perform transmission and reception of a frequency-modulated continuous wave radar signal, such that a digital mixed signal can be determined based on the transmission and reception of the frequency-modulated continuous wave radar signal, and in turn, such that vital sign spectrum data of the measured object 140 can be obtained based on the digital mixed signal.

[0039] Specifically, the video signal acquisition device 120 can have a depth camera or a multi-view matching camera. For example, the multi-view matching camera can be a binocular matching camera. The video signal acquisition device 120 can capture an environment in which the measured object 140 is located through the depth camera or the multi-view matching camera to obtain a video signal, such that biological feature data of the measured object and environmental data of the environment in which the measured object is located can be obtained based on the video signal, and in turn, such that individual feature representation data of the measured object can be obtained based on the biological feature data, and such that environmental feature representation data of the environment in which the measured object is located can be obtained based on the environmental data.

[0040] Specifically, the second measurement device 130 can be a measurement device different from the first measurement device 110. Exemplarily, the second measurement device 130 can be a contact type measurement device, or in other words, a contact type sensor. As an example, the second measurement device 130 can be any one of a mechanical measurement device and a biological signal measurement device. The mechanical measurement device can be a respiratory belt sensor, for example, the respiratory belt sensor can be fixed on the thorax by an elastic belt or a chest belt, and the respiratory rate can be measured by detecting the expansion and contraction changes of the elastic belt and the chest belt. The mechanical measurement device can also be a respiratory quality sensor, for example, the respiratory quality sensor can measure the flow rate and flow volume of respiratory gas by using a pressure sensor or a mass sensor, and calculate the respiratory rate by analyzing the characteristics of the respiratory gas. The biological signal measurement device can be an electrocardiogram (ECG) monitoring device or a blood oxygen saturation (SpO2) monitoring device, the ECG monitoring device can collect electrocardiogram signals by using electrode patches attached to the skin to calculate the heart rate, and the SpO2 monitoring device can measure the blood oxygen saturation in the blood by using a photoelectric sensor to calculate the heart rate and the respiratory rate. In this way, the true value for training the target classification model can be obtained through the second measurement device.

[0041] As an example, the first measurement device 110 can be connected with the video signal acquisition device 120 and the second measurement device 130. The first measurement device 110 can be deployed with a target classification model to determine the vital sign signal value through the target classification model. The target classification model deployed in the first measurement device 110 is obtained by model training based on the vital sign spectrum data, the individual feature representation data, the environment feature representation data and the true value at the same historical moment.

[0042] As another example, please refer to Figure 1b , Figure 1b is a schematic diagram of a scene example of a vital sign signal measurement system provided by the embodiments of the present specification. The vital sign signal measurement system 100 can further include a data processing device 150, which can be connected with the first measurement device 110, the video signal acquisition device 120 and the second measurement device 130. The data processing device 150 can be deployed with a target classification model to determine the vital sign signal value through the target classification model. The target classification model deployed in the data processing device 150 is obtained by model training based on the vital sign spectrum data, the individual feature representation data, the environment feature representation data and the true value at the same historical moment. Exemplarily, the data processing device 150 can be a server or a terminal.

[0043] The embodiments of the present specification provide a vital sign signal measurement method based on human factor intelligence. Please refer to Figure 2 , Figure 2 is a flowchart of a vital sign signal measurement method based on human factor intelligence provided by the embodiments of the present specification. The embodiments of the present specification provide method operation steps as shown in the flowchart, but more or fewer operation steps can be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is only one of the many execution orders of the steps, and does not represent the only execution order. In actual system or server product execution, the method order shown in the embodiments can be executed in sequence or in parallel (for example, in a parallel processor or multi-thread processing environment). The vital sign signal measurement method can be applied to the first measurement device or the data processing device in the vital sign signal measurement system, as shown in Figure 2 , the vital sign signal measurement method can include the following steps.

[0044] Step S210: obtaining individual feature representation data of a measured object, environment feature representation data of an environment where the measured object is located, and vital sign spectrum data of the measured object; wherein the individual feature representation data and the environment feature representation data have different influences on the vital sign spectrum data.

[0045] In some cases, vital sign spectrum data corresponding to the subject under test can be determined. Also, individual feature representation data of the subject under test can be determined, and environment feature representation data of an environment in which the subject under test is located can be determined, so that a vital sign signal value of the subject under test can be predicted based on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data.

[0046] The vital sign spectrum data can be determined based on a millimeter wave radar. For example, the first measurement device can be configured with a millimeter wave radar, so that the vital sign spectrum data can be determined based on the millimeter wave radar of the first measurement device.

[0047] Specifically, a frequency-modulated continuous wave radar signal transmitted and received by the millimeter wave radar of the first measurement device can be mixed to obtain a digital mixed signal, and the vital sign spectrum data of the subject under test can be determined based on the digital mixed signal. For example, the frequency-modulated continuous wave radar signal can be generated by a millimeter wave radar transmitter of the first measurement device, the frequency of the frequency-modulated continuous wave radar signal can vary over time, for example, increase or decrease over time within a specified time period, the transmitted frequency-modulated continuous wave radar signal can be reflected back to the first measurement device by the subject under test, the first measurement device can receive the reflected frequency-modulated continuous wave radar signal through a millimeter wave radar receiver, and the transmitted frequency-modulated continuous wave radar signal and the received frequency-modulated continuous wave radar signal can be mixed to obtain a mixed signal. The mixed signal, i.e., an intermediate frequency signal or a beat signal, can facilitate subsequent determination of the vital sign spectrum data of the subject under test.

[0048] For example, the frequency-modulated continuous wave radar signal can be a chirp signal.

[0049] For example, the mixed signal can also be subjected to analog-to-digital conversion to obtain a digital mixed signal, so that subsequent spectrum estimation related operations can be facilitated based on the digital mixed signal.

[0050] For example, after the digital mixed signal is determined, spectrum estimation related operations can be performed based on the digital mixed signal to determine the vital sign spectrum data. The spectrum estimation related operations can include at least one of distance dimension determination, processing unit determination, phase extraction, phase unwrapping, phase differentiation, bandpass filtering, and spectrum estimation.

[0051] The Range FFT is distance information determined based on a Fast Fourier Transform (FFT) of the digital mixed frequency signal, or a Range curve determined based on the FFT of the digital mixed frequency signal, or a spectrum of the distance dimension determined based on the FFT of the digital mixed frequency signal, so as to determine a plurality of distance units. The Range bin tracking is to determine a distance unit corresponding to the measured object from the plurality of distance units. The Extract Phase is to extract a phase of the distance unit corresponding to the measured object. The Phase Unwrapping is to perform phase unwrapping to obtain a phase signal. The Phase Difference is to enhance the unwrapped phase signal and reduce the existing phase drift. The Bandpass Filtering is to filter the corresponding phase in the phase signal by using a bandpass filter to distinguish different vital sign signals, or to filter the frequency formed by the change of the corresponding phase in the phase signal to distinguish different vital sign signals, so as to facilitate the determination of different vital sign signals subsequently. The spectrum estimation is to perform an FFT transform on the obtained phase signal, so as to obtain vital sign spectrum data of the measured object, so that the vital sign signal of the measured object can be determined based on the vital sign spectrum data of the measured object.

[0052] The individual characteristic representation data can be data capable of describing the measured object, or data used for identifying the measured object.

[0053] The environmental characteristic representation data can be data capable of describing the environment in which the measured object is located, or data used for identifying the environment in which the measured object is located.

[0054] The difference influence of the individual characteristic representation data on the vital sign spectrum data can be the influence of the individual difference of the measured object on the vital sign spectrum data. For example, the individual difference of the measured object can be an identity, an age, a gender, a skin type, etc. For example, the difference between the frequency hopping continuous wave radar signals reflected by the skin type of oily skin and the skin type of dry skin can be relatively large, and the difference between the vital sign spectrum data obtained thereby can also be relatively large. Different skin types have different influences on the vital sign spectrum data, that is, the individual characteristic representation data of the measured object has a difference influence on the vital sign spectrum data.

[0055] The environmental feature representation data can represent the influence of the environmental difference of the environment where the measured object is located on the vital sign spectrum data. For example, the environmental difference of the environment where the measured object is located can refer to the difference in illumination, brightness, temperature, humidity, etc. of the environment where the measured object is located. For example, the difference between the reflected frequency hopping continuous wave radar signal in a high humidity environment and the reflected frequency hopping continuous wave radar signal in a low humidity environment is relatively large. In a high humidity environment, the performance of the frequency hopping continuous wave radar signal can be attenuated, which can affect the determination of the vital sign signal of the measured object based on the vital sign spectrum data.

[0056] Step S220: performing signal value prediction based on the individual feature representation data, the environmental feature representation data, and the vital sign spectrum data to obtain the vital sign signal value removed from the influence of the difference.

[0057] In the above embodiments, the individual feature representation data of the measured object, the environmental feature representation data of the environment where the measured object is located, and the vital sign spectrum data of the measured object are obtained, and signal value prediction is performed based on the individual feature representation data, the environmental feature representation data, and the vital sign spectrum data to remove the influence of the difference of the individual feature representation data and the environmental feature representation data on the vital sign spectrum data and obtain the corresponding vital sign signal value. In this way, the accuracy of measuring the vital sign signal of the measured object can be improved.

[0058] In some embodiments, referring to Figure 3 , the individual feature representation data of the measured object can include the following steps.

[0059] Step S310: obtaining biological feature data of the measured object based on the video signal; wherein the video signal is obtained by photographing the environment where the measured object is located.

[0060] In some cases, the environment where the measured object is located can be photographed and video signals can be collected by a video signal collection device.

[0061] Specifically, the biological feature data of the measured object can be obtained based on the video signal photographed by the video signal collection device. For example, the biological feature data of the measured object can be any one of face data, iris data, retinal data, and eyeprint data of the measured object.

[0062] As an example, when the face data is acquired based on the video signal captured by the video signal acquisition device, the video signal can be pre-processed, for example, noise removal, contrast enhancement and other pre-processing operations can be performed on the video signal to enhance the quality of the video signal. After pre-processing the video signal, the face region of the measured object can be recognized based on the pre-processed video signal, and when the face region is detected, the face features can be further extracted, for example, the geometric features of the face such as the face contour, eyes, mouth, nose, and the texture features of the face or the face data can be extracted, so that the individual recognition of the measured object can be performed based on the face data subsequently.

[0063] As an example, the video signal acquisition device can be a designated device capable of any one of iris recognition, retina recognition, and eyeprint recognition, so that the iris data, retina data, or eyeprint data of the measured object can be acquired based on the video signal captured by the designated video signal acquisition device.

[0064] Step S320: performing individual recognition on the measured object based on the biometric feature data in the video signal to obtain individual feature representation data.

[0065] Specifically, the biometric feature comparison data can be pre-stored, and the biometric feature data can be compared with the pre-stored biometric feature comparison data to obtain an individual recognition result, and the individual feature representation data can be determined based on the individual recognition result.

[0066] The individual recognition result can include at least one of an identity of the measured object, a gender feature, an age feature, and a skin type feature. The identity can be an identification information for uniquely identifying the measured object.

[0067] In some cases, the individual recognition result can be a high-dimensional sparse feature vector. In order to facilitate subsequent prediction of the vital sign signal value of the measured object, the individual recognition result can be embedded into a low-dimensional dense feature vector of a specified dimension to obtain the individual feature representation data, so that the calculation amount and storage space can be reduced and the prediction efficiency can be improved in the process of predicting the vital sign signal value based on the individual feature representation data.

[0068] The high-dimensional sparse feature vector can be used to process some unstructured data. For example, the individual recognition result of the measured object can be represented as a high-dimensional sparse feature vector. The low-dimensional dense feature vector, i.e., the individual feature representation data, can be the individual feature representation data obtained after the individual recognition result is embedded, which can retain the important features and information of the individual recognition result as a high-dimensional sparse feature vector, while reducing the dimension and complexity of the individual recognition result.

[0069] In the above embodiments, the biological feature data of the measured object is obtained based on the video signal, and the individual feature representation data is obtained by individual recognition of the measured object based on the biological feature data. In this way, the individual feature representation data for identifying the measured object can be quickly obtained, which facilitates subsequent removal of the influence of the individual feature representation data on the vital sign spectrum data, and thus the accuracy of the vital sign signal value can be improved.

[0070] In some embodiments, referring to Figure 4 The environment feature representation data of the environment in which the measured object is located can be obtained by the following steps.

[0071] Step S410: obtaining environment data of an environment in which the measured object is located based on a video signal; wherein the video signal is obtained by photographing the environment in which the measured object is located.

[0072] Specifically, the environment data of the environment in which the measured object is located can be obtained based on the video signal captured by the video signal acquisition device. Exemplarily, the video signal can be a video image. The video image can be preprocessed, for example, denoising, contrast enhancement, and other preprocessing that can enhance the image quality. The environment in which the measured object is located in the preprocessed video image can be processed and analyzed to extract the environment data of the environment in which the measured object is located. Exemplarily, the environment data can include at least one of temperature, humidity, weather, wind speed, and light intensity.

[0073] Exemplarily, the video signal can carry at least one of the information of the photographing time, the photographing place, the latitude and longitude, and the altitude. The environment data of the environment in which the measured object is located can be extracted based on the video image to estimate and determine at least one of the photographing time, the photographing place, the latitude and longitude, and the altitude. The photographing time can include any of spring, summer, autumn, and winter, and can also include any of daytime and night, and can also include any hour or any minute in 24 hours in a day. The photographing place can be indoor, outdoor, or semi-open place, and the semi-open place can be balcony or terrace, corridor or passageway, roof or roof, greenhouse or flower room, etc.

[0074] Step S420: performing feature extraction based on the environment data in the video signal to obtain environment feature representation data; wherein the environment feature representation data includes at least one of humidity feature, temperature feature, weather feature, wind speed feature, and light intensity.

[0075] In some cases, the environmental data can be a high-dimensional sparse feature vector. In order to facilitate subsequent prediction of the vital sign signal value of the measured object, the environmental data can be embedded into a low-dimensional dense feature vector of a specified dimension to obtain environmental feature representation data, so that the calculation amount and storage space can be reduced and the prediction efficiency can be improved in the process of predicting the vital sign signal value based on the environmental feature representation data.

[0076] The environmental data can be represented as a high-dimensional sparse feature vector. The environmental feature representation data can refer to the environmental data obtained after embedding, which can retain important features and information of the environmental data as a high-dimensional sparse feature vector, while reducing the dimension and complexity of the environmental data.

[0077] In the above embodiments, the environmental data of the environment in which the measured object is located is obtained based on the video signal, and the environmental feature representation data is obtained based on the environmental data. In this way, the environmental feature representation data of the environment in which the measured object is located can be quickly obtained, which facilitates subsequent removal of the difference influence of the environmental feature representation data on the vital sign frequency spectrum data, and thus the accuracy of the vital sign signal value can be improved.

[0078] In some embodiments, referring to Figure 5 The vital sign frequency spectrum data of the measured object can be obtained by the following steps.

[0079] Step S510: Obtain a video signal and a digital mixing signal of a first measurement device; wherein the video signal is obtained by photographing the environment in which the measured object is located; and the digital mixing signal is determined based on the emission and reception of the frequency-modulated continuous wave radar signal by the first measurement device.

[0080] In some cases, the vital sign frequency spectrum data of the measured object can be determined based on the digital mixing signal and the video signal. Determining the vital sign frequency spectrum data based on the digital mixing signal in combination with the video signal can improve the accuracy of the vital sign frequency spectrum data.

[0081] Specifically, the environment in which the measured object is located can be photographed by a video signal acquisition device to obtain a video signal, and a digital mixing signal determined by the emission and reception of the frequency-modulated continuous wave radar signal by the first measurement device through the millimeter wave radar can be obtained, so as to determine the vital sign frequency spectrum data of the measured object by the video signal and the digital mixing signal.

[0082] Exemplarily, the video signal can have a photographing time, and the digital mixing signal can have corresponding time information. The photographing time of the video signal and the corresponding time information of the digital mixing signal are consistent, or in other words, the video signal and the digital mixing signal are aligned in time.

[0083] Step S520: determining an initial distance unit in which the measured object is located relative to the first measuring device based on the digital mixed signal.

[0084] Specifically, the initial distance unit can be determined based on a range dimension determination (Range FFT) operation and a range bin tracking operation. For example, a fast Fourier transform (FFT) can be performed on the digital mixed signal to determine a spectrum diagram of the range dimension, thereby obtaining a plurality of distance units. Then, a search is performed among the plurality of distance units to determine a distance unit corresponding to the measured object as the initial distance unit.

[0085] Step S530: correcting the initial distance unit based on the video signal to obtain a target distance unit.

[0086] Specifically, after determining the initial distance unit corresponding to the measured object, the initial distance unit can be corrected based on the video signal to obtain a target distance unit corresponding to the measured object, so that the phase signal can be determined based on the target distance unit to further determine the vital sign spectrum data of the measured object.

[0087] Step S540: determining the vital sign spectrum data based on the target distance unit.

[0088] For example, after the initial distance unit is corrected to the target distance unit based on the video signal, a phase extraction operation can be performed on the target distance unit to obtain a phase corresponding to the measured object.

[0089] For example, the digital mixed signal can be determined by transmitting and receiving a Chirp signal by a millimeter wave radar. As an example, in a frame, or in a frame period, a plurality of Chirp signals can be continuously transmitted by the millimeter wave radar, and the frame period can refer to a complete transmission and reception period of a Chirp signal. For example, when a frame period is 50 milliseconds and a duration of a Chirp signal is 50 microseconds, the number of Chirp signals transmitted in a frame is 1000.

[0090] For example, the transmitted Chirp signal and the received reflected Chirp signal can be mixed to obtain a mixed signal, and then the mixed signal can be analog-to-digital converted. When analog-to-digital conversion is performed, the mixed signal can be sampled based on a specified sampling number to obtain a digital mixed signal. The specified sampling number is the sampling number within a Chirp signal. The digital mixed signal can have a corresponding bandwidth.

[0091] As an example, for a digital mixed signal, an initial distance bin is determined based on a Range FFT operation and a Range bin tracking operation, and then the initial distance bin is corrected based on a video signal to obtain a target distance bin, and then an Extract Phase operation is performed on the target distance bin to determine the phase of the target distance bin corresponding to the measured object, which can be performed in a loop, so that the change of the target distance bin of the measured object with the frame number can be determined, that is, the change of the phase of the measured object with time can be determined, and then a Phase Unwrapping operation can be performed to unwrap the phase to obtain a phase signal of the measured object, a Phase Difference operation can be performed to enhance the unwrapped phase signal and reduce the existing phase drift, a Bandpass Filtering operation can be performed to filter the corresponding phase in the phase signal to distinguish, or the frequency formed by the change of the corresponding phase in the phase signal is filtered to distinguish, which is convenient for subsequent determination of different vital sign signals, and a spectrum estimation operation can be performed on the obtained phase signal to perform FFT transformation to obtain vital sign spectrum data of the measured object.

[0092] By way of example, the vital sign spectrum data can be obtained after Embedding based on the result of the spectrum estimation operation, that is, the vital sign spectrum data is a low-dimensional dense feature vector with a specified dimension. By way of example, the vital sign spectrum data can include at least transmission wave quantity information, transmission wave sampling quantity information, and bandwidth information. The transmission wave quantity information can be the number of Chirp signals transmitted in a frame, the transmission wave sampling quantity information can be a specified sampling number in a Chirp signal, and the bandwidth can be the bandwidth after the Bandpass Filtering operation.

[0093] In the above embodiments, the video signal is obtained by photographing the environment where the measured object is located, at the same time, the digital mixed signal is determined based on the transmission and reception of the frequency-modulated continuous wave radar signal by the first measurement device, then the initial distance bin of the measured object relative to the first measurement device is determined based on the digital mixed signal, the target distance bin is obtained by correcting the initial distance bin based on the video signal, and the vital sign spectrum data is determined based on the target distance bin. In this way, the initial distance bin in the determination process of the vital sign spectrum data of the measured object based on the video signal can be corrected, so that the accuracy of the measured object in the vital sign signal measurement can be improved.

[0094] In some embodiments, please refer to Figure 6The initial distance unit is corrected based on the video signal to obtain a target distance unit, which can include the following steps.

[0095] Step S610: detecting the distance between the first measuring device and the measured object based on the video signal to obtain a video detection distance.

[0096] Step S620: correcting the initial distance unit based on the video detection distance to obtain a target distance unit.

[0097] Specifically, the initial distance unit can be corrected based on the video detection distance. If the difference between the video detection distance and the initial distance unit is greater than a specified difference threshold, the initial distance unit can be corrected based on the video detection distance. As an example, correcting the initial distance unit based on the video detection distance can mean that, if the video detection distance is greater than the specified difference threshold, the plurality of distance units obtained based on the distance dimension of the spectrogram are searched again in the plurality of distance units to determine the distance unit corresponding to the measured object as the target distance unit. As another example, correcting the initial distance unit based on the video detection distance can mean that, if the video detection distance is greater than the specified difference threshold, the initial distance unit is adjusted based on the video detection distance to obtain the target distance unit.

[0098] In the above embodiments, the video detection distance between the first measuring device and the measured object can be determined based on the video signal, and the initial distance unit in the determination process of the vital sign spectrum data of the measured object can be corrected based on the video detection distance, thereby improving the accuracy of the measured object when measuring the vital sign signal.

[0099] In some embodiments, referring to Figure 7a The signal value prediction based on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain the vital sign signal value removed from the difference influence can include the following steps.

[0100] Step S710a: jointly processing the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain a feature joint result.

[0101] Specifically, the individual feature representation data, the environment feature representation data, and the vital sign spectrum data can be jointly characterized, so that the individual feature representation data, the environment feature representation data, and the vital sign spectrum data can be converted into a joint feature result suitable for model training. For example, if the individual feature representation data, the environment feature representation data, and the vital sign spectrum data are all low-dimensional dense feature vectors of a specified dimension, the individual feature representation data, the environment feature representation data, and the vital sign spectrum data are spliced into a spliced vector, and the spliced vector is taken as the joint feature result.

[0102] Step S720a: signal value prediction is performed according to the joint feature result, and vital sign signal values are obtained; wherein the vital sign signal values include at least one of heart rate and respiration rate.

[0103] For example, the heart rate can be obtained by performing signal value prediction according to the joint feature result. The respiration rate can be obtained by performing signal value prediction according to the joint feature result. The heart rate and the respiration rate can also be obtained by performing signal value prediction according to the joint feature result.

[0104] As an example, the phases in the phase signals corresponding to the heart rate and the respiration rate are usually different, or in other words, the frequencies formed by the phase changes in the phase signals corresponding to the heart rate and the respiration rate are usually different, or in other words, the bandwidth information in the vital sign spectrum data corresponding to the heart rate and the respiration rate is different. Therefore, signal value prediction based on the vital sign spectrum data obtained after bandpass filtering operation can obtain the heart rate or the respiration rate.

[0105] For example, signal value prediction can be performed based on different confidence indicators.

[0106] In the above embodiments, by jointly characterizing the individual feature representation data, the environment feature representation data, and the vital sign spectrum data, and performing signal value prediction according to the joint feature result, vital sign signal values including at least one of heart rate and respiration rate are obtained. Therefore, the influence of the individual feature representation data and the environment feature representation data on the vital sign spectrum data can be removed, and the accuracy of vital sign signal measurement of the measured object can be improved.

[0107] In some embodiments, the joint feature result obtained by jointly characterizing the individual feature representation data, the environment feature representation data, and the vital sign spectrum data can include: splicing the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain the joint feature result.

[0108] Specifically, the individual feature representation data, the environment feature representation data, and the vital sign spectrum data can each be of a specified dimension, and when the individual feature representation data, the environment feature representation data, and the vital sign spectrum data are jointly characterized, the individual feature representation data, the environment feature representation data, and the vital sign spectrum data can be concatenated based on the specified dimension to obtain a joint characterization result, so that the vital sign signal value can be predicted based on the joint characterization result.

[0109] In the above embodiments, by concatenating the individual feature representation data and the environment feature representation data to the vital sign spectrum data, the influence of the individual feature representation data and the environment feature representation data on the vital sign spectrum data is considered and removed when measuring the vital sign signal of the measured object, thereby improving the accuracy of vital sign signal measurement.

[0110] In some embodiments, referring to Figure 7b , the individual feature representation data of the measured object and the environment feature representation data of the environment in which the measured object is located can include the following steps.

[0111] S710b: Obtain a video signal obtained by photographing the environment in which the measured object is located.

[0112] S720b: Determine the environment data and the individual identification result of the measured object based on the video signal.

[0113] S730b: Perform mapping processing on the environment data to obtain environment feature representation data of a specified dimension.

[0114] S740b: Perform mapping processing on the individual identification result to obtain individual feature representation data of a specified dimension.

[0115] Specifically, the video signal obtained by photographing the environment in which the measured object is located by the video signal acquisition device can be obtained, and the biological feature data of the measured object can be obtained based on the video signal. For example, the obtained biological feature data can be face data of the measured object. For example, the face region of the measured object can be identified based on the video signal, and when the face region is detected, the face features such as the face contour, eyes, mouth, and nose are further extracted to obtain the face data. The individual identification result of the measured object including at least one of the identity, gender feature, age feature, and skin type feature can be obtained based on the face data. Then, the individual identification result can be mapped to obtain individual feature representation data of a specified dimension.

[0116] Specifically, a video signal captured by the video signal acquisition device for the environment where the measured object is located can be acquired, and environment data of the environment where the measured object is located can be acquired based on the video signal. For example, the acquired environment data can include at least one of temperature, humidity, weather, wind speed, and light intensity. For example, the video signal can have at least one of shooting time, shooting place, latitude and longitude, and altitude information. For example, at least one of the shooting time, the shooting place, the latitude and longitude, and the altitude information is used to determine the environment data of the environment where the measured object is located. Then, the environment data can be mapped to obtain environment feature representation data of a specified dimension.

[0117] The specified dimension of the environment feature representation data and the individual feature representation data is the same.

[0118] For example, the vital sign spectrum data can have a specified dimension. When the environment data and the individual identification result are mapped to the environment feature representation data and the individual feature representation data of the specified dimension respectively, the specified dimension can be determined according to the specified dimension of the vital sign spectrum data.

[0119] In some embodiments, the vital sign signal value can be output by the target classification model. Please refer to Figure 8a The target classification model can be obtained by the following training method.

[0120] S810a: Construct a training sample set; wherein the training sample set includes a plurality of training samples, and each training sample includes historical vital sign spectrum data, historical individual feature representation data, and historical environment feature representation data; and the label of the training sample is a historical vital sign signal true value.

[0121] S820a: Model training is performed on the initial classification model according to the training sample and the label to obtain a target classification model; wherein the initial classification model is built based on any one of VGG, EfficientNet, and ResNet model structures.

[0122] Specifically, after the training sample set is constructed, the historical individual feature representation data, the historical environment feature representation data and the historical vital sign spectrum data in the training sample set can be input into an initial classification model for model training, and the historical individual feature representation data, the historical environment feature representation data and the historical vital sign spectrum data are jointly characterized by the initial classification model to obtain a historical feature joint result. Based on the historical feature joint result, a predicted vital sign signal is obtained. Further, the initial classification model corresponds to a loss function, and the input of the initial classification model corresponds to a label. The label and the predicted vital sign signal are input into the loss function to determine a model loss value, and the initial classification model is updated based on the determined model loss value. In this way, the target classification model is obtained until the model training stopping condition is met. The model training stopping condition can be that the model loss value converges, or the number of training rounds reaches a preset number of rounds.

[0123] In the above embodiments, the historical vital sign spectrum data, the historical individual feature representation data and the historical environment feature representation data are used to construct the training sample, and in the process of training the initial classification model using the training sample and the historical vital sign true value, the initial classification model gradually learns the ability to remove the influence of the individual feature representation data and the environment feature representation data on the vital sign spectrum data, and the target classification model is obtained, so that the accurate vital sign signal value can be obtained.

[0124] In some embodiments, referring to Figure 8b The training sample can be constructed in the following manner.

[0125] S810b: Based on the historical digital mixing frequency signal collected by the first measuring device at the historical time for the measured object or the non-measured object, historical vital sign spectrum data is determined.

[0126] S820b: Historical video signals collected at the historical time are obtained.

[0127] S830b: Based on the historical video signals, historical individual feature representation data and historical environment feature representation data are determined.

[0128] Specifically, since model training requires training samples, at multiple historical time points before model training, a historical digital mixing signal determined by the first measurement device through the transmission and reception of the frequency-modulated continuous wave radar signal of the millimeter wave radar is acquired, and a historical initial distance unit of the measured object or the non-measured object relative to the first measurement device is determined based on the historical digital mixing signal. Then, the distance between the first measurement device and the measured object or the non-measured object is detected based on the historical video signal to obtain a historical video detection distance, the historical initial distance unit is corrected based on the historical video detection distance to obtain a historical target distance unit, and historical vital sign spectrum data is determined based on the historical target distance unit.

[0129] Specifically, biological feature data of the measured object or the non-measured object can be acquired based on the historical video signal, and individual identification of the measured object or the non-measured object can be performed based on the biological feature data in the historical video signal to obtain historical individual feature representation data.

[0130] Specifically, environmental data of an environment in which the measured object or the non-measured object is located can be acquired based on the historical video signal, and feature extraction can be performed based on the environmental data in the historical video signal to obtain historical environmental feature representation data.

[0131] Specifically, a historical vital sign signal true value can also be determined based on the second measurement device for collecting the measured object or the non-measured object, so as to use the historical vital sign signal true value as a label for model training.

[0132] At this point, training samples input to the initial classification model can be constructed based on the historical vital sign spectrum data, the historical individual feature representation data, and the historical environmental feature representation data.

[0133] It should be noted that the historical vital sign spectrum data, the historical individual feature representation data, the historical environmental feature representation data, and the historical vital sign signal true value are time-aligned. In other words, the historical vital sign spectrum data, the historical individual feature representation data, the historical environmental feature representation data, and the historical vital sign signal true value are all determined at the same historical time point in the multiple historical time points.

[0134] In the above embodiment, the historical vital sign spectrum data, the historical individual feature representation data, and the historical environmental feature representation data are used to construct training samples, thereby providing a data basis for training the initial classification model.

[0135] The embodiment of the present specification provides a vital sign signal measurement method based on human factors intelligence, which can be applied to a first measurement device or a data processing device in a vital sign signal measurement system. The vital sign signal measurement method can include the following steps.

[0136] Step S901: Obtain historical vital sign spectrum data; wherein the historical vital sign spectrum data is determined by the first measuring device at a historical time for a measured object or a non-measured object.

[0137] Step S903: Obtain a historical video signal collected at the historical time.

[0138] Step S905: Obtain a historical vital sign signal true value; wherein the historical vital sign signal true value is determined by the second measuring device at the historical time for the measured object or the non-measured object.

[0139] Step S907: Determine historical individual feature representation data and historical environment feature representation data according to the historical video signal.

[0140] Step S909: Construct a training sample according to the historical individual feature representation data, the historical environment feature representation data, and the historical vital sign spectrum data.

[0141] It should be noted that a plurality of training samples constitute a training sample set. The training sample includes historical vital sign spectrum data, historical individual feature representation data, and historical environment feature representation data; and the label of the training sample adopts the historical vital sign signal true value.

[0142] Step S911: Model training is performed on the initial classification model according to the training sample and the label to obtain a target classification model.

[0143] Step S913: Obtain individual feature representation data of the measured object, environment feature representation data of an environment in which the measured object is located, and vital sign spectrum data of the measured object; wherein the individual feature representation data and the environment feature representation data have differential effects on the vital sign spectrum data.

[0144] Specifically, the individual feature representation data of the measured object is obtained by: obtaining biological feature data of the measured object based on a video signal; wherein the video signal is obtained by photographing the environment in which the measured object is located; and performing individual identification on the measured object based on the biological feature data in the video signal to obtain the individual feature representation data.

[0145] Specifically, the biological feature data of the measured object is any one of face data, iris data, retina data, and eyeprint data.

[0146] Specifically, the individual feature representation data includes at least one of gender features, age features, and skin type features of the measured object.

[0147] Specifically, the environment feature representation data of the environment where the measured object is located is obtained by the following method: obtaining environment data of the environment where the measured object is located based on a video signal; wherein the video signal is obtained by shooting the environment where the measured object is located; performing feature extraction based on the environment data in the video signal to obtain the environment feature representation data; wherein the environment feature representation data comprises at least one of humidity features, temperature features, weather features, and wind speed features.

[0148] Specifically, the vital sign spectrum data of the measured object is obtained by the following method: obtaining a video signal and a digital mixing signal of a first measuring device; wherein the video signal is obtained by shooting the environment where the measured object is located; the digital mixing signal is determined based on the first measuring device through the transmission and reception of the frequency-modulated continuous wave radar signal; determining an initial distance unit where the measured object is located relative to the first measuring device based on the digital mixing signal; correcting the initial distance unit based on the video signal to obtain a target distance unit; determining the vital sign spectrum data based on the target distance unit.

[0149] Specifically, the target distance unit is obtained by correcting the initial distance unit based on the video signal, comprising: detecting the distance between the first measuring device and the measured object based on the video signal to obtain a video detection distance; correcting the initial distance unit based on the video detection distance to obtain the target distance unit.

[0150] Step S915: signal value prediction based on individual feature representation data, environment feature representation data, and vital sign spectrum data to obtain vital sign signal values removed from the influence of differences.

[0151] Specifically, the vital sign signal value is output by the target classification model.

[0152] Specifically, the individual feature representation data, the environment feature representation data, and the vital sign spectrum data can be jointly characterized to obtain a feature joint result; the signal value prediction is performed according to the feature joint result to obtain the vital sign signal value; wherein the vital sign signal value comprises at least one of heart rate and respiration rate.

[0153] It can be understood that in various embodiments of the present specification, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present specification.

[0154] The embodiments of the present specification provide a vital sign signal measurement device based on human factor intelligence. The vital sign signal measurement device can be applied to a first measuring device or a data processing device in a vital sign signal measurement system. Please refer to Figure 9The measurement device can include a feature data acquisition module 910 and a vital sign determination module 920.

[0155] The feature data acquisition module 910 is configured to acquire individual feature representation data of the measured object, environmental feature representation data of an environment in which the measured object is located, and vital sign spectrum data of the measured object; wherein the individual feature representation data and the environmental feature representation data have different influences on the vital sign spectrum data.

[0156] The vital sign determination module 920 is configured to perform signal value prediction based on the individual feature representation data, the environmental feature representation data, and the vital sign spectrum data, to obtain a vital sign signal value removed from the different influences.

[0157] In some embodiments, the feature data acquisition module 910 is further configured to acquire biological feature data of the measured object based on a video signal; wherein the video signal is obtained by photographing the environment in which the measured object is located; and perform individual identification on the measured object based on the biological feature data in the video signal, to obtain the individual feature representation data. The biological feature data of the measured object is any one of face data, iris data, retina data, and eyeprint data; and the individual feature representation data includes at least one of gender features, age features, and skin type features of the measured object.

[0158] In some embodiments, the feature data acquisition module 910 is further configured to acquire environmental data of the environment in which the measured object is located based on a video signal; wherein the video signal is obtained by photographing the environment in which the measured object is located; and perform feature extraction based on the environmental data in the video signal, to obtain the environmental feature representation data; wherein the environmental feature representation data includes at least one of humidity features, temperature features, weather features, and wind speed features.

[0159] In some embodiments, the feature data acquisition module 910 is further configured to acquire a video signal and a digital mixing signal of the first measurement device; wherein the video signal is obtained by photographing the environment in which the measured object is located; the digital mixing signal is determined based on the first measurement device through frequency-modulated continuous wave radar signal transmission and reception by a millimeter wave radar; determine an initial distance unit of the measured object relative to the first measurement device based on the digital mixing signal; correct the initial distance unit based on the video signal, to obtain a target distance unit; and determine the vital sign spectrum data based on the target distance unit.

[0160] In some embodiments, the feature data acquisition module 910 is further configured to detect a distance between the first measurement device and the measured object based on the video signal, to obtain a video detection distance; and correct the initial distance unit based on the video detection distance, to obtain the target distance unit.

[0161] In some embodiments, the vital sign determination module 920 is further configured to: perform feature jointing on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain a feature jointing result; and perform signal value prediction based on the feature jointing result to obtain the vital sign signal value, wherein the vital sign signal value comprises at least one of a heart rate and a respiration rate.

[0162] In some embodiments, the vital sign determination module 920 is further configured to: perform splicing processing on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain the feature jointing result.

[0163] In some embodiments, the feature data acquisition module 910 is further configured to: acquire a video signal obtained by photographing an environment in which the measured object is located; determine environment data and an individual identification result of the measured object based on the video signal; perform mapping processing on the environment data to obtain environment feature representation data of a specified dimension; and perform mapping processing on the individual identification result to obtain individual feature representation data of a specified dimension.

[0164] In some embodiments, the vital sign signal value is output by a target classification model. The measurement device can further comprise a model training module. The model training module is configured to: construct a training sample set, wherein the training sample set comprises a plurality of training samples, and each training sample comprises historical vital sign spectrum data, historical individual feature representation data, and historical environment feature representation data; train an initial classification model based on the training samples and labels to obtain a target classification model, wherein the labels of the training samples are true values of historical vital sign signals; and the initial classification model is built based on any one of VGG, EfficientNet, and ResNet model structures.

[0165] In some embodiments, the model training module is further configured to: construct the training sample set based on the following: historical vital sign spectrum data is determined based on historical digital mixing signals collected by a first measurement device at a historical time for a measured object or a non-measured object; historical video signals collected at the historical time are acquired; and historical individual feature representation data and historical environment feature representation data are determined based on the historical video signals.

[0166] For specific functions and effects achieved by the measurement device, reference can be made to the explanations of other embodiments of the present specification, which will not be repeated here. Each module in the measurement device can be implemented in whole or in part by software, hardware, and combinations thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform operations corresponding to each module.

[0167] The embodiment of the present specification also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a computer to enable the computer to execute the vital sign signal measurement method in any of the above embodiments.

[0168] The embodiment of the present specification also provides a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the vital sign signal measurement method in any of the above embodiments.

[0169] The embodiment of the present specification also provides a computer device comprising a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement the vital sign signal measurement method in the above embodiments.

[0170] In the embodiment, referring to Figure 10 , the computer device can be a terminal, and its internal structure diagram can be as shown in Figure 10 . The computer device comprises a processor, a memory and a communication interface connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in wired or wireless mode. The wireless mode can be achieved by WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement the vital sign signal measurement method.

[0171] It can be understood that the specific examples herein are only to help those skilled in the art better understand the embodiments of the present specification, and do not limit the scope of the present application.

[0172] It can be understood that in various embodiments of the present specification, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present specification.

[0173] It can be understood that the various embodiments described in the present specification can be implemented alone or in combination, and the embodiments of the present specification do not limit this.

[0174] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this specification belongs. The terminology used in the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. As used in this specification, the terms "may" and "can" include any one of, or a combination of, the corresponding inexcitables. As used in this specification and the appended claims, the singular forms "a," "an" and "the" include plural referents unless the context clearly dictates otherwise.

[0175] It can be understood that the processor in the embodiments of the present specification can be an integrated circuit chip with processing capability of signals. In the implementation process, each step of the method embodiments described above can be completed by integrated logic circuits or instructions in the form of software in the processor. The processor described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present specification can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present specification can be directly embodied as a hardware coding processor to execute, or a combination of hardware and software modules in the coding processor. The software module can be located in a storage medium in the art such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the storage, and the processor reads the information in the storage, and combines the hardware to complete the steps of the above method.

[0176] It can be understood that the memory in the embodiments of the present specification can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM). It should be noted that the memory of the system and method described herein is intended to include but not limited to these and any other suitable type of memory.

[0177] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present specification.

[0178] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0179] In several embodiments provided in the present specification, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0181] In addition, each functional unit in each embodiment of the present specification can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0182] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present specification or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present specification. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0183] The above is only a specific embodiment of the present specification, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present specification, which should be covered within the protection scope of the present specification. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A human factor intelligence based vital sign signal measurement method, characterized by, The method comprises: obtaining individual feature representation data of a measured object, environment feature representation data of an environment in which the measured object is located, and vital sign spectrum data of the measured object; wherein the individual feature representation data and the environment feature representation data have different influences on the vital sign spectrum data; performing signal value prediction based on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data by a target classification model to obtain vital sign signal values removed from the different influences; wherein the vital sign spectrum data is a low-dimensional dense feature vector obtained by performing Embedding embedding processing on a digital mixed frequency signal after a spectrum estimation related operation; the spectrum estimation related operation can utilize a video signal to correct an initial distance unit determined based on the digital mixed frequency signal to obtain a target distance unit, and perform phase extraction, phase unwrapping, phase difference, bandpass filtering, and spectrum estimation operations based on the target distance unit and then perform Embedding embedding processing; the video signal is obtained by shooting the environment in which the measured object is located; the digital mixed frequency signal is determined by transmitting and receiving a Chirp signal by a millimeter wave radar, and the vital sign spectrum data at least includes transmission wave quantity information, transmission wave sampling quantity information, and bandwidth information, the transmission wave quantity information is the number of Chirp signals transmitted in any frame of transmission wave, the transmission wave sampling quantity information is a specified sampling number in any Chirp signal, and the bandwidth information is the bandwidth after bandpass filtering; wherein the correction method for the initial distance unit comprises: if the difference between the initial distance unit and the video detection distance obtained based on the video signal is greater than a specified difference threshold, correcting the initial distance unit based on the video detection distance; wherein the obtaining of the individual feature representation data of the measured object and the environment feature representation data of the environment in which the measured object is located comprises: obtaining a video signal obtained by shooting the environment in which the measured object is located; determining environment data and individual identification results of the measured object based on the video signal; performing mapping processing on the environment data to obtain the environment feature representation data of a specified dimension; performing mapping processing on the individual identification results to obtain the individual feature representation data of the specified dimension; wherein the individual identification results and the environment data are high-dimensional sparse feature vectors, the environment feature representation data is a low-dimensional dense feature vector obtained by performing Embedding embedding processing on the environment data, and the individual feature representation data is a low-dimensional dense feature vector obtained by performing Embedding embedding processing on the individual identification results; The signal value prediction based on the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain the vital sign signal value removed from the difference influence comprises: performing feature combination on the individual feature representation data, the environment feature representation data and the vital sign spectrum data based on the specified dimension to obtain a feature combination result; and performing signal value prediction according to the feature combination result to obtain the vital sign signal value; wherein the vital sign signal value comprises at least one of a heart rate and a respiration rate. The feature combination on the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain the feature combination result comprises: performing vector splicing on the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain the feature combination result.

2. The measurement method according to claim 1, characterized in that, The individual feature representation data of the measured object is obtained by the following way, comprising: The biological feature data of the measured object is obtained based on a video signal; wherein the video signal is obtained by photographing the environment where the measured object is located; The individual feature representation data is obtained by individual identification of the measured object based on the biological feature data in the video signal.

3. The measurement method according to claim 2, characterized in that, The biological feature data of the measured object is any one of face data, iris data, retina data and eyeprint data; The individual feature representation data comprises at least one of gender feature, age feature and skin type feature of the measured object.

4. The measurement method according to claim 1, characterized by, The environment feature representation data of the environment where the measured object is located is obtained by the following way, comprising: The environment data of the environment where the measured object is located is obtained based on a video signal; wherein the video signal is obtained by photographing the environment where the measured object is located; The environment feature representation data is obtained by feature extraction based on the environment data in the video signal; wherein the environment feature representation data comprises at least one of humidity feature, temperature feature, weather feature and wind speed feature.

5. The method of claim 1, wherein, The vital sign signal value is output by the target classification model; and the target classification model is obtained by the following training way: A training sample set is constructed; wherein the training sample set comprises a plurality of training samples, and each training sample comprises historical vital sign spectrum data, historical individual feature representation data and historical environment feature representation data; and a label of each training sample adopts a historical vital sign signal true value; An initial classification model is trained according to the training sample and the label to obtain the target classification model; wherein the initial classification model is constructed based on any one of VGG, EfficientNet and ResNet model structures.

6. The measurement method according to claim 5, characterized in that, The training sample is constructed by the following way: The historical vital sign spectrum data is determined based on a historical digital mixing signal collected by a first measuring device at a historical time for the measured object or a non-measured object; A historical video signal collected at the historical time is obtained; determining the historical individual feature representation data and the historical environment feature representation data according to the historical video signal; a historical vital sign signal true value collected by a second measurement device at the historical time for the measured object or a non-measured object as a label.

7. The measurement method according to claim 6, characterized in that, The second measurement device is a measurement device different from the first measurement device; wherein the second measurement device is any one of a mechanical measurement device and a biological signal measurement device.

8. A human factor intelligence based vital sign signal measurement device, characterized by, The device comprises: a feature data acquisition module for acquiring individual feature representation data of a measured object, environment feature representation data of an environment where the measured object is located, and vital sign spectrum data of the measured object; wherein the individual feature representation data and the environment feature representation data have different influences on the vital sign spectrum data; a vital sign determination module for performing signal value prediction based on the individual feature representation data, the environment feature representation data, and the vital sign spectrum data to obtain a vital sign signal value removed from the different influences; wherein the vital sign spectrum data is a low-dimensional dense feature vector obtained by performing Embedding embedding processing on a digital mixed frequency signal after a spectrum estimation related operation; the spectrum estimation related operation can utilize a video signal to correct an initial distance unit determined based on the digital mixed frequency signal to obtain a target distance unit, and perform phase extraction, phase unwrapping, phase difference, bandpass filtering, and spectrum estimation operations based on the target distance unit and then perform Embedding embedding processing; the video signal is obtained by shooting the environment where the measured object is located; the digital mixed frequency signal is determined by transmitting and receiving Chirp signals through a millimeter wave radar, and the vital sign spectrum data at least includes transmission wave quantity information, transmission wave sampling quantity information, and bandwidth information; the transmission wave quantity information is the number of Chirp signals transmitted in any frame of transmission wave; the transmission wave sampling quantity information is a specified number of samples within any Chirp signal; and the bandwidth information is the bandwidth after bandpass filtering; wherein the correction method for the initial distance unit includes: if the difference between the initial distance unit and the video detection distance obtained based on the video signal is greater than a specified difference threshold, correcting the initial distance unit based on the video detection distance; The individual feature representation data of the measured object and the environment feature representation data of the environment in which the measured object is located are obtained by: obtaining a video signal obtained by photographing the environment in which the measured object is located; determining environment data and individual identification results of the measured object based on the video signal; performing mapping processing on the environment data to obtain the environment feature representation data of a specified dimension; and performing mapping processing on the individual identification results to obtain the individual feature representation data of the specified dimension; wherein the individual identification results and the environment data are high-dimensional sparse feature vectors, the environment feature representation data is a low-dimensional dense feature vector obtained by performing Embedding embedding processing on the environment data, and the individual feature representation data is a low-dimensional dense feature vector obtained by performing Embedding embedding processing on the individual identification results. The signal value prediction based on the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain the vital sign signal value removed from the influence of the difference includes: performing feature combination on the individual feature representation data, the environment feature representation data and the vital sign spectrum data based on the specified dimension to obtain a feature combination result; and performing signal value prediction according to the feature combination result to obtain the vital sign signal value; wherein the vital sign signal value includes at least one of a heart rate and a respiration rate. The feature combination of the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain a feature combination result includes: performing vector splicing on the individual feature representation data, the environment feature representation data and the vital sign spectrum data to obtain the feature combination result.

9. The measuring device of claim 8, wherein, The feature data acquisition module is further configured to: obtain biological feature data of the measured object based on a video signal; wherein the video signal is obtained by photographing the environment in which the measured object is located; and perform individual identification on the measured object based on the biological feature data in the video signal to obtain the individual feature representation data. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the vital sign signal measurement method in any one of claims 1 to 7.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the vital sign signal measurement method in any one of claims 1 to 7.

Images