Vehicle-mounted electrocardiogram detection method and device, storage medium, system and vehicle

Abstract

The invention discloses a vehicle-mounted electrocardiogram detection method and device, a storage medium, a system and a vehicle. The vehicle-mounted electrocardio detection method comprises the steps that in response to a vehicle state and / or a user instruction, the number and positions of target to-be-detected passengers are determined; acquiring a first video containing the face of the target passenger to be detected; carrying out image analysis based on the first video to obtain electrocardiogram detection data of the member to be detected; according to the number and the positions of the target to-be-detected passengers, the electrocardiogram detection data of the target to-be-detected passengers are displayed in a preset area of display equipment. The intelligent degree and individuation degree of vehicle-mounted electrocardiogram detection can be effectively improved, and meanwhile the efficiency and accuracy of vehicle-mounted electrocardiogram detection can be improved.

Description

Technical Field

[0001] The present application relates to a vehicle-mounted electrocardiogram detection method, device, storage medium, system and vehicle. Background Art

[0002] In the vehicle-mounted scenario, the physical condition of the occupants is crucial for driving safety. Therefore, the importance of vehicle-mounted electrocardiogram detection function for vehicle driving safety is self-evident.

[0003] Currently, the vehicle-mounted electrocardiogram detection function mainly adopts non-contact methods. It uses a UWB millimeter-wave radar fixed on the seat to perform non-contact electrocardiogram detection on the occupants, or installs a camera to take videos of the vehicle occupants and perform non-contact electrocardiogram detection through image analysis. However, these non-contact electrocardiogram detection solutions can only perform electrocardiogram detection on the occupants within specific areas such as the area where the seat is located and the camera shooting range, and only support single-person electrocardiogram detection. They cannot perform multi-person electrocardiogram detection based on actual needs. At the same time, the degree of human-computer interaction and result matching during the electrocardiogram detection process is also relatively low.

[0004] Therefore, there is an urgent need to propose a new vehicle-mounted electrocardiogram detection solution to improve the intelligence of vehicle-mounted electrocardiogram detection and meet the personalized needs in practical applications. Summary of the Invention

[0005] To solve or at least partially solve the above technical problems, the present application provides a vehicle-mounted electrocardiogram detection method, device, storage medium, system and vehicle, which can effectively improve the intelligence and personalization of vehicle-mounted electrocardiogram detection.

[0006] One aspect of the present application provides a vehicle-mounted electrocardiogram detection method, including:

[0007] Responding to the vehicle state and / or user instruction, determining the number and position of the target occupants to be detected;

[0008] Obtaining a first video of the face of the target occupants to be detected;

[0009] Obtaining electrocardiogram detection data of the target occupants to be detected based on image analysis of the first video;

[0010] According to the number and position of the target occupants to be detected, displaying the electrocardiogram detection data of the target occupants to be detected in a predetermined area of the display device.

[0011] Preferably, responding to the vehicle state and / or user instruction, determining the number and position of the target occupants to be detected includes:

[0012] Detecting the vehicle state and / or receiving a user instruction;

[0013] If the vehicle state meets the first predetermined condition and / or the user instruction indicates that multi-person electrocardiogram detection is required, trigger the multi-person electrocardiogram detection function and determine the seats in the vehicle cockpit where there are occupants sitting.

[0014] Determine the number of seats with occupants sitting as the number of target occupants to be detected, and record the information of the seats as the positions of the corresponding target occupants to be detected.

[0015] Preferably, the user instruction is a voice instruction; in response to the vehicle state and / or the user instruction, determining the number and position of the target occupants to be detected includes: performing voice recognition on the voice instruction to determine the user's intention. When the user's intention indicates that single-person electrocardiogram detection is required, determine that the number of target occupants to be detected is 1, and perform sound localization analysis based on the voice instruction to obtain the position of the target occupant to be detected.

[0016] Preferably, in response to the vehicle state and / or the user instruction to determine the number and position of the target occupants to be detected, it further includes: after performing sound localization analysis based on the voice instruction to obtain the position of the target occupant to be detected, acquiring a second image at the position of the target occupant to be detected, and performing target detection on the second image to verify whether the position of the target occupant to be detected is correct.

[0017] Preferably, before acquiring the first video of the face of the target occupant to be detected, it further includes: acquiring a second image at the position of the target occupant to be detected; using the second image to determine whether the positions of the target occupants to be detected meet the requirements; if the positions of any one or more target occupants to be detected do not meet the requirements, then adjust the positions of the corresponding target occupants to be detected and / or control the corresponding camera to adjust the shooting angle so that the positions of all target occupants to be detected meet the requirements.

[0018] Preferably, adjusting the positions of the corresponding target occupants to be detected and / or controlling the camera to adjust the shooting angle includes: determining whether the target occupants to be detected with non-compliant positions include the driver, and detecting the vehicle state; if the target occupants to be detected with non-compliant positions include the driver and it is detected that the vehicle is in a driving state, then control the first camera to adjust its own shooting angle so that the first camera can capture the face of the driver and the face is within the region of interest. The first camera is at least used to collect the first video of the driver's face; if the target occupants to be detected with non-compliant positions do not include the driver or it is detected that the vehicle is in a stationary state, then issue a position adjustment prompt, and the position adjustment prompt is used to instruct the target occupants to be detected to adjust their own positions so that the first camera and / or the second camera can capture the faces of the target occupants to be detected and the faces are within the region of interest. The second camera is at least used to collect the first video of the occupants other than the driver.

[0019] Preferably, determining whether the positions of the target occupants to be detected meet the requirements by using the second image includes: if there are multiple target occupants to be detected, determining the display areas of the target occupants to be detected on the display screen according to the positions of the target occupants to be detected, and controlling the display device to display the faces of the corresponding target occupants to be detected in the second image in each display area; if the facial features displayed in any one or more display areas do not meet the requirements or the faces of the corresponding target occupants to be detected cannot be displayed normally, determining that the positions of the corresponding one or more target occupants to be detected do not meet the requirements.

[0020] Preferably, performing image analysis on the first video to obtain the electrocardiogram detection data of the members to be detected includes: performing image segmentation on the first video according to the positions and quantities of the target occupants to be detected to generate an image set corresponding to each target occupant to be detected, each image set including multiple frames of region-of-interest images of the same target occupant to be detected, and each frame of region-of-interest image including the face of the corresponding target occupant to be detected; respectively performing image analysis on the image sets of the target occupants to be detected to obtain the electrocardiogram detection data of the target occupants to be detected.

[0021] Preferably, controlling the display device to display the electrocardiogram detection data of the target occupants to be detected in a predetermined area of the display screen according to the quantity and position of the target occupants to be detected includes: if there are multiple target occupants to be detected, determining the display areas matched by the target occupants to be detected according to the positions of the target occupants to be detected, and controlling the display device to display the electrocardiogram detection data of the target occupants to be detected matched in each display area of the display screen.

[0022] Preferably, controlling the display device to display the electrocardiogram detection data of the target occupants to be detected in a predetermined area of the display screen according to the quantity and position of the target occupants to be detected includes: if there is one target occupant to be detected, controlling the display device to display the electrocardiogram detection data of the target occupant to be detected in a predetermined area corresponding to the position of the target occupant to be detected on the display screen.

[0023] In one aspect of the present application, there is provided a vehicle-mounted electrocardiogram detection device, which is characterized by including:

[0024] A determination unit, configured to determine the quantity and position of the target occupants to be detected in response to the vehicle state and / or user instruction;

[0025] A video acquisition unit, configured to acquire a first video of the faces of the target occupants to be detected;

[0026] An image analysis unit, configured to obtain the electrocardiogram detection data of the target members to be detected based on the image analysis of the first video;

[0027] A display unit, configured to display electrocardiogram detection data of a target occupant to be detected in a predetermined area of a display device according to the number and position of the target occupant to be detected.

[0028] In one aspect of the present application, a computing device is provided, including a processor and a memory. The memory stores program instructions, and when the program instructions are executed by the processor, the processor is caused to execute the above-mentioned vehicle electrocardiogram detection method.

[0029] In one aspect of the present application, a computer-readable storage medium is provided, on which program instructions are stored. It is characterized in that when the program instructions are executed by a computer, the computer is caused to execute the above-mentioned vehicle electrocardiogram detection method.

[0030] In one aspect of the present application, a vehicle electrocardiogram detection system is provided, including a computing device, a camera group, and a display device;

[0031] The computing device includes at least one processor and at least one memory. The memory stores program instructions, and when the program instructions are executed by the at least one processor, the at least one processor is caused to execute the above-mentioned vehicle electrocardiogram detection method;

[0032] The camera group includes one or more cameras installed in the vehicle cockpit, and is configured to collect a first video of the face of the target occupant to be detected.

[0033] The display device includes a display screen, and is configured to display electrocardiogram detection data of the target occupant to be detected in a predetermined area corresponding to the position of the target occupant to be detected on the display screen under the control of the computing device.

[0034] Preferably, the camera group includes a first camera and a second camera. The first camera is configured to collect a first video of the driver, and the second camera is configured to collect a first video of all occupants except the driver.

[0035] Preferably, the display device is specifically configured to: when the number of target occupants to be detected is multiple, divide the display screen into multiple display areas, and display electrocardiogram detection data of the matched target occupants to be detected in each display area of the display screen under the control of the computing device; when the number of target occupants to be detected is one, display electrocardiogram detection data of the target occupant to be detected in a predetermined area on the display screen under the control of the display device.

[0036] Preferably, the user instruction is a voice instruction; the vehicle electrocardiogram detection system further includes: an array microphone, configured to collect the user instruction and perform sound localization analysis based on the user instruction to obtain the position of the target occupant to be detected.

[0037] Preferably, the vehicle electrocardiogram detection system further includes: a pick-up microphone, configured to perform sound localization analysis based on the user instruction to obtain the position of the target occupant to be detected.

[0038] In one aspect of the present application, a vehicle is provided, including the on-vehicle electrocardiogram detection device, the computing device or the on-vehicle electrocardiogram detection system described above.

[0039] Compared with the prior art, the present application has the following beneficial effects:

[0040] The on-vehicle electrocardiogram detection method provided by the embodiments of the present application can determine the number of occupants and the positions of the occupants in real time in response to the vehicle state or user instructions, and perform multi-person electrocardiogram detection or single-person electrocardiogram detection according to the number of occupants and the positions of the occupants, effectively improving the intelligence and personalization of on-vehicle electrocardiogram detection, and at the same time, the efficiency and accuracy of on-vehicle electrocardiogram detection can also be improved. Brief Description of the Drawings

[0041] In order to more clearly illustrate the embodiments of the present application, the relevant drawings will be briefly introduced below. It can be understood that the drawings in the following description are only used to illustrate some embodiments of the present application, and those of ordinary skill in the art can also obtain many other technical features and connection relationships not mentioned in this article based on these drawings.

[0042] Figure 1 is a schematic flow chart of the on-vehicle electrocardiogram detection method provided by the embodiments of the present application;

[0043] Figure 2 is an example diagram of the division of the display area in a preferred embodiment of the present application;

[0044] Figure 3 is an example diagram of the partition display of the electrocardiogram detection data of three occupants in a preferred embodiment of the present application;

[0045] Figure 4 is an example diagram of the display of single-person electrocardiogram detection data in a preferred embodiment of the present application;

[0046] Figure 5 is an example diagram of the display interface when the occupant position is adjusted during multi-person electrocardiogram detection in a preferred embodiment of the present application;

[0047] Figure 6 is an example diagram of the display interface when all the facial features of the occupants meet the requirements during multi-person electrocardiogram detection in a preferred embodiment of the present application;

[0048] Figure 7 is an example diagram of the display interface when all the facial features of the occupant in the driver's seat meet the requirements during single-person electrocardiogram detection in a preferred embodiment of the present application;

[0049] Figure 8 is a schematic structural diagram of the on-vehicle electrocardiogram detection device provided by the embodiments of the present application;

[0050] Figure 9It is a schematic structural diagram of a computing device provided by an embodiment of the present application;

[0051] Figure 10 It is a schematic structural diagram of an in-vehicle electrocardiogram detection system provided by an embodiment of the present application;

[0052] Figure 11 It is an example diagram of a vehicle provided by an embodiment of the present application. Detailed implementation manners

[0053] Next, the present application will be described in detail with reference to the accompanying drawings.

[0054] As mentioned above, in-vehicle electrocardiogram detection solutions in related technologies mostly use sensors such as UWB millimeter-wave radars and cameras to implement. They can only perform electrocardiogram detection on occupants within a specific area, and cannot perform electrocardiogram detection on specific personnel in the vehicle. Moreover, due to the non-intrusive detection state, there are also problems such as poor interactivity and low matching degree.

[0055] In view of this, the embodiments of the present application propose an in-vehicle electrocardiogram detection method, device, equipment, storage medium and system, which can adaptively perform multi-person electrocardiogram detection or single-person electrocardiogram detection by determining the number and positions of occupants in real time in response to vehicle states or user instructions, effectively improving the intelligence and personalization of in-vehicle electrocardiogram detection, and at the same time, can also improve the efficiency and accuracy of in-vehicle electrocardiogram detection.

[0056] The embodiments of the present application can be applied to various in-vehicle scenarios. Exemplarily, the embodiments of the present application can be applied to scenarios with a large number of occupants and complex occupants in vehicles such as buses, coaches, and passenger cars, and can also be applied to scenarios with a small number of occupants in vehicles such as cars. Of course, the embodiments of the present application can also be applied to other transportation means such as ships and airplanes. The specific application scenarios of the embodiments of the present application are not limited herein.

[0057] Next, the detailed implementation manners of the present application will be elaborated.

[0058] Exemplary In-Vehicle Electrocardiogram Detection Method

[0059] Figure 1 shows a schematic flowchart of an in-vehicle electrocardiogram detection method provided by an embodiment of the present application. Refer to Figure 1 As shown, the in-vehicle electrocardiogram detection method of the embodiments of the present application includes:

[0060] Step S110, in response to vehicle states and / or user instructions, determine the number and positions of target occupants to be detected;

[0061] The vehicle state may include, but is not limited to, the vehicle driving state, the states of various on-vehicle devices loaded on the vehicle, the working states of specific components of the vehicle, etc. In one example, the vehicle driving state may include states such as vehicle starting, vehicle shutting down, vehicle driving, and vehicle stationary. States such as vehicle starting and vehicle shutting down may be determined according to the working state of the vehicle engine, and states such as vehicle driving and vehicle stationary may be indicated according to the real-time speed of the vehicle. In one example, the states of various on-vehicle devices loaded on the vehicle may include, but are not limited to, the working state of the camera group installed in the vehicle cockpit. In one example, the working states of specific components of the vehicle may include, but are not limited to, the door state, the headlight state, etc. The door state may include states such as the door being tightly locked, the door being opened, and the door being closed but not locked.

[0062] In specific applications, the vehicle state can be sensed by sensors installed on the vehicle or detected in real time by on-vehicle devices on the vehicle. Exemplarily, the door state can be sensed by sensors installed on the door and reported to the computing device.

[0063] The user instruction may include, but is not limited to, voice instructions, button instructions, gesture instructions, touch instructions, and / or any other type of human-computer interaction instructions input by any one or more occupants in the vehicle. In one example, the user instruction may be a voice instruction input by any one occupant (e.g., the driver) in the vehicle and collected by the array microphone installed in the vehicle cockpit. In one example, the user instruction may be an instruction input by any occupant in the vehicle by touching the virtual buttons on the in-vehicle central control screen. The specific form of the user instruction is not limited in the embodiments of the present application.

[0064] In step S110, the number of target occupants to be detected may be one, two, or more. If it is determined that the number of target occupants to be detected exceeds 1 person, multi-person electrocardiogram detection will be triggered to perform electrocardiogram detection on these target occupants to be detected simultaneously. If it is determined that the number of target occupants to be detected is only 1 person, single-person electrocardiogram detection can be started to perform electrocardiogram detection on the target occupant. In this way, multi-person electrocardiogram detection or single-person electrocardiogram detection can be performed according to the number of occupants in the vehicle, the in-vehicle scenario, and the personalized needs of the user.

[0065] In some embodiments, step S110 may include: detecting the vehicle state and / or the currently received user instruction. If the vehicle state meets the first predetermined condition and / or the user instruction indicates that multi-person electrocardiogram detection is required, trigger the multi-person electrocardiogram detection function, determine the seats in the vehicle cockpit where there are occupants sitting, determine the number of seats with occupants sitting as the number of target occupants to be detected, and record the seat information as the position of the corresponding target occupant to be detected.

[0066] The first predetermined condition can be flexibly configured according to actual requirements. In one example, the first predetermined condition can be set as follows: the camera captures at least one occupant, and at the same time, it is detected that the vehicle door is locked or the vehicle has started. In other examples, the first predetermined condition can also be set to other contents. The embodiments of the present application do not limit the setting method and specific content of the first predetermined condition.

[0067] For example, if the first predetermined condition can be set as: the camera captures at least one occupant, and at the same time, it is detected that the vehicle door is locked or the vehicle has started. At this time, if the vehicle camera captures an occupant and at the same time detects that the vehicle door is locked or the vehicle has started, a single electrocardiogram (ECG) detection for all current occupants in the vehicle can be automatically started. If there are four occupants in the current vehicle, located in the driver's seat, the co-driver's seat, the left rear seat, and the right rear seat respectively, at this time, the number of target occupants to be detected is determined to be 4, and the positions of the target occupants to be detected can be marked as: the driver's seat, the co-driver's seat, the left rear seat, and the right rear seat. If there is one occupant in the current vehicle, located in the driver's seat, at this time, the number of target occupants to be detected is determined to be 1, and the position of the target occupant to be detected can be marked as: the driver's seat.

[0068] In some embodiments, step S110 may include: receiving a user instruction; if the user instruction indicates that single-person ECG detection is required, triggering the single-person ECG detection function, determining the seats in the vehicle cockpit where there are occupants, determining that the number of target occupants to be detected is 1, and recording the seat information as the position of the corresponding target occupant to be detected.

[0069] In one example, the user instruction can be a voice instruction, and the user intention can be obtained through voice recognition. If the user intention indicates that multi-person ECG detection is required, trigger the multi-person ECG detection function; for example, after recognizing the voice instruction of "full-body ECG detection" through natural language, a single ECG detection for all current occupants in the vehicle can be automatically started, determine the seats in the vehicle cockpit where there are occupants, determine the number of seats where each occupant is sitting as the number of target occupants to be detected, and record the seat information as the position of the corresponding target occupant to be detected. If the user intention indicates that single-person ECG detection is required, trigger the single-person ECG detection function. For example, after recognizing "driver's seat ECG detection", "co-pilot ECG detection", "left rear ECG detection", etc. through natural language, a single ECG detection for the occupant who issued the user instruction can be automatically started, the number of target occupants to be detected is determined to be 1, and voice localization analysis is performed based on the voice instruction to obtain the position of the target occupant to be detected.

[0070] In specific applications, multiple methods can be used to determine the seats in the vehicle cockpit where there are occupants. In one example, a panoramic image of the interior scene of the vehicle cockpit can be captured by a camera, and the seats in the vehicle cockpit where there are occupants can be determined by performing object detection on the panoramic image of the interior scene of the vehicle cockpit. In one example, seat information indicating whether there is someone sitting on the corresponding seat can be collected by sensors (such as millimeter-wave radars, etc.) installed on each seat in the vehicle cockpit, and by counting the seat information collected by the sensors on all seats, it is possible to know the seats in the vehicle cockpit where there are occupants. Of course, other methods applicable to the vehicle cockpit environment can also be used to determine the seats in the vehicle cockpit where there are occupants, and the embodiments of this application do not limit this.

[0071] Specifically, the user intention can be represented by text information. If the text information representing the user intention contains pre-set multi-person electrocardiogram detection keywords (such as "multiple people", "electrocardiogram detection", "health detection"), it can be regarded that the user intention indicates the need for multi-person electrocardiogram detection. If the text information representing the user intention contains pre-set single-person electrocardiogram detection keywords (such as "one person", "electrocardiogram detection", "health detection", "single person"), it can be regarded that the user intention indicates the need for single-person electrocardiogram detection, and at this time, a single electrocardiogram detection can be automatically started for the occupant who issues a voice command in the current vehicle. If the text information of the user intention does not match the pre-set electrocardiogram detection keywords, the user command can be regarded as an invalid command and no action is taken. In some embodiments, if no target occupant to be detected is found at the corresponding position according to the user command, an error report or voice prompt can be given.

[0072] Specifically, when the user intention obtained based on the user command indicates that a single-person detection is required, the user command can be saved, and a pick-up or sound localization analysis module is called to read the user command and perform sound localization analysis on it to determine the orientation of the user command, and the position of the target occupant to be detected is obtained according to the pronunciation orientation of the user command. For example, if it is found through sound localization analysis that the occupant who issues the user command is the occupant in the rear left position, the position of the target occupant to be detected can be marked as: rear left position.

[0073] In one example, an array microphone can be installed in the vehicle, and the voice commands issued by the vehicle occupants are collected through the array microphone. Since the time, intensity, etc. of the sounds received by different microphones in the array microphone are different for sounds emitted from different positions, the sound localization analysis of the voice commands can be realized by collecting the differences in the sound information collected by different microphones in the array microphone. The array microphone can adopt circular or rectangular array microphones.

[0074] In one example, the sound localization analysis of the voice commands can be realized by installing a pick-up in the vehicle.

[0075] In some embodiments, considering that there may be errors in sound localization analysis, in the single-person ECG detection mode, the position of the target occupant to be detected can be verified secondarily by taking images. Specifically, after performing sound localization analysis based on a voice command to obtain the position of the target occupant to be detected, a second image at the position of the target occupant to be detected is acquired, and object detection is performed on the second image to verify whether the position of the target occupant to be detected is correct. If it is correct, single-person ECG detection continues for this target occupant to be detected; if it is incorrect, the current ECG detection can be terminated. Specifically, for object detection of the second image, if a person or any human body part is detected in the second image, the position of the target occupant to be detected obtained by performing sound localization analysis based on the voice command is correct; if no person or any human body part is detected in the second image, the position of the target occupant to be detected obtained by performing sound localization analysis based on the voice command is incorrect.

[0076] For example, if sound localization determines that the co-pilot's seat needs to be detected for ECG alone, the camera captures an image of the co-pilot's seat. After performing object detection on the image of the co-pilot's seat and determining that there is indeed an occupant in the co-pilot's seat, it indicates that the sound localization is accurate, and single-person ECG detection can continue for the occupant in the co-pilot's seat. If, after performing object detection on the image of the co-pilot's seat, it is determined that there is no occupant in the co-pilot's seat, it indicates that the sound localization is incorrect or the occupant's position has changed, and the current ECG detection can be terminated, and a prompt is sent through a voice or display device to prompt the occupant to re-enter the voice command. This prompt can be a prompt sound with the content "I didn't hear clearly. Please re-enter the command". Thus, abnormal ECG detection caused by incorrect sound localization can be effectively avoided.

[0077] Of course, different ECG detection modes can be switched according to the voice command. For example, in the single-person ECG detection mode described above, when multi-person detection is required, the multi-person ECG detection mode is entered.

[0078] Step S120: Obtain a first video containing the face of the target occupant to be detected;

[0079] The first video refers to a sequence of consecutive image frames of a predetermined length. The length of the first video depends on the requirements of image analysis in step S130 and can be flexibly configured according to actual needs. For example, the first video can be a video with a length of approximately 30 seconds to 60 seconds collected by a camera group installed in the vehicle cockpit, and each frame image in this video contains a face. Of course, in some embodiments, ECG detection can also be performed as long as part of the face is captured.

[0080] The first video can be collected by a camera group installed in the vehicle cockpit. The camera group can include one or more cameras, and these cameras can be installed at any position inside the vehicle cockpit. For example, the cameras in the camera group can be installed at positions such as the sun visor inside the vehicle cockpit, the main in-vehicle display screen, and the ceiling-mounted TV, and can be flexibly adjusted according to actual needs and the performance of the cameras. Taking a five-seat vehicle as an example, the camera group can include two cameras, namely the first camera and the second camera. The first camera is used to collect the first video of the driver's seat, and the second camera can be used to collect the first video of the co-driver's seat, the left rear seat, and the right rear seat. That is, the first video collected by the first camera contains the face of the driver, and the first video collected by the second camera contains the faces of all other occupants except the driver.

[0081] In some embodiments, before step S120, it may further include: obtaining a second image at the position of the target occupant to be detected, using the second image to determine whether the positions of the target occupants to be detected meet the requirements. If the positions of any one or more target occupants to be detected do not meet the requirements, then adjust the positions of the corresponding target occupants to be detected and / or control the corresponding camera to adjust the shooting angle so that the positions of all target occupants to be detected meet the requirements. Thus, it is possible to determine whether the shooting angle of the camera and the position of the occupant are appropriate by taking an image of the position of the target occupant to be detected, prompt the occupant to adjust the position in time or automatically adjust the shooting angle of the camera to ensure that at least a part of the face of the occupant is included in the first video collected. In some embodiments, the collected facial area can also be a preset area, such as the forehead, cheeks, complete face, etc., to avoid errors in electrocardiogram detection, thereby effectively improving the accuracy of in-vehicle electrocardiogram detection.

[0082] In specific applications, the requirements for the positions of the target occupants to be detected can be flexibly configured according to actual needs. For example, the requirements for the positions of the target occupants to be detected can be configured such that the second image contains a face and can be displayed on the display screen. In other examples, the requirements for the positions of the target occupants to be detected can also be configured such that a specified area in the second image contains at least a face and the face can be displayed on the display screen. Specifically, the steps of using the second image to determine whether the positions of the target occupants to be detected meet the requirements may include: if the number of target occupants to be detected is multiple, determining the display areas of the target occupants to be detected on the display screen according to the positions of the target occupants to be detected and controlling the display device to display the faces of the corresponding target occupants to be detected in the second image in the respective display areas; if the facial features displayed in any one or more display areas do not meet the requirements or the faces of the corresponding target occupants to be detected cannot be normally displayed, determining that the positions of the corresponding one or more target occupants to be detected do not meet the requirements. Thus, it is possible to confirm whether the shooting angle or the occupant position is appropriate by directly showing the user the image of the occupant captured by the camera in real time, facilitating the user to intuitively understand the personnel situation of the electrocardiogram detection, and adjusting their own position or the shooting angle of the camera at any time when necessary, thereby effectively improving the accuracy and detection efficiency of the electrocardiogram detection.

[0083] In some examples, before step S120, it may further include: determining whether the target occupants to be detected with non - compliant positions include the driver, and detecting the vehicle state; if the target occupants to be detected with non - compliant positions include the driver and the vehicle is detected to be in a driving state, controlling the first camera to adjust its own shooting angle so that the first camera can capture the face of the driver and the face is within the region of interest (the face here can be a partial face or the entire face), and the first camera is at least used to collect the first video of the driver; if the target occupants to be detected with non - compliant positions do not include the driver or the vehicle is detected to be in a stationary state, issuing a position adjustment prompt, where the position adjustment prompt is used to instruct the target occupants to be detected to adjust their own positions so that the first camera and / or the second camera can capture the faces of the target occupants to be detected and the faces are within the region of interest, and the second camera is at least used to collect the first video of the occupants other than the driver. Thus, it is possible to prevent the driver from being distracted by actively adjusting their own position for the need of electrocardiogram detection, thereby ensuring driving safety during the electrocardiogram detection process. At the same time, it is also possible to prompt other occupants to actively adjust their positions through human - machine interaction to improve the accuracy of their electrocardiogram detection.

[0084] In step S120, after determining that the positions of the target occupants to be detected all meet the requirements, the vehicle occupants can be prompted to maintain their positions through visual or voice prompts. The camera group is controlled to record a first video for a predetermined duration, and after the camera group finishes recording the first video, a prompt such as "Video acquisition completed" is issued. This can not only improve the recording quality of the first video, effectively improve the accuracy and detection efficiency of electrocardiogram detection, but also provide a good human-computer interaction experience and facilitate the user to understand the specific situation of electrocardiogram detection in real time.

[0085] Step S130: Obtain the electrocardiogram detection data of the member to be detected based on the image analysis of the first video.

[0086] In some embodiments, step S130 may include: performing image segmentation on the first video according to the positions and quantities of the target occupants to be detected to generate an image set corresponding to each target occupant to be detected. Each image set contains multiple frames of region-of-interest images of the same target occupant to be detected, and each frame of region-of-interest image contains the face of the corresponding target occupant to be detected. Image analysis is respectively performed on the image sets of each target occupant to be detected to obtain the electrocardiogram detection data of each target occupant to be detected.

[0087] Specifically, the first video is segmented frame by frame to obtain a continuous frame sequence that only contains the facial region of the target occupant to be detected, and then frame-by-frame image analysis is performed on the continuous frame sequence to obtain the electrocardiogram detection data of the target occupant to be detected. The electrocardiogram detection data may include, but is not limited to, heart rate, blood pressure, and other physiological indicators.

[0088] In one example, if each frame of an image of a certain first video contains the faces of three occupants, image recognition and segmentation are performed on each frame of the image to obtain three region-of-interest images. Each region-of-interest image is the facial image of an occupant, and the region-of-interest image is marked by the occupant's position. Thus, three continuous frame sequences are generated. Each continuous frame sequence contains continuous multiple frames of facial images of the same occupant and is marked by the occupant's position. At the same time, spatial averaging is performed frame by frame on the three continuous frame sequences to obtain pixels, and three IPPG signals are calculated. The three IPPG signals are marked by the occupant's position. After denoising and signal conversion of the three IPPG signals, the heart rates of the three occupants are obtained simultaneously, and the heart rates of the three occupants are marked by the occupant's position.

[0089] In one example, if each frame of an image of a certain first video contains the faces of three occupants, it can be pre-divided into three groups of images and these groups are marked by the occupant's position (for example, the co-pilot position group of images, the rear left position group of images, the rear right position group of images). Each frame of the image at the corresponding position in the first video is intercepted and assigned to the corresponding group of images, and then image analysis is sequentially performed on each group of images to obtain the heart rates of the occupants in the three positions. In this way, the electrocardiogram data of the occupants in the three positions can be presented sequentially.

[0090] Taking a four-seater vehicle as an example, if the first video captured by the second camera of the vehicle includes the faces of the co-pilot seat, the left rear seat, and the right rear seat, the heart rates of the occupants in the co-pilot seat, the left rear seat, and the right rear seat can be obtained through frame-by-frame image segmentation and image analysis of the first video.

[0091] In step S130, methods such as Image Photoplethysmography (IPPG) or Remote Photoplethysmography (rPPG) can be used for image analysis to obtain the electrocardiogram detection data of the occupants. The embodiments of the present application do not limit this.

[0092] Step S140: According to the number and positions of the target occupants to be detected, display the electrocardiogram detection data of the target occupants to be detected in a predetermined area of the display device.

[0093] In some embodiments, if the number of target occupants to be detected is multiple, the display areas matched to each target occupant to be detected can be determined according to the positions of the target occupants to be detected, and the display device is controlled to display the electrocardiogram detection data of the target occupants to be detected in the corresponding display areas on the display screen. If the number of target occupants to be detected is one, the display device is controlled to display the electrocardiogram detection data of the target occupant to be detected in a predetermined area, that is, a preset area, corresponding to the position of the target occupant to be detected on the display screen. In this way, the electrocardiogram detection data of each occupant can be accurately and clearly displayed on the display screen.

[0094] Figure 2 Fig. shows an example diagram of the division of the display area of the display screen. Taking a four-seater vehicle as an example, the display screen 200 can be defaultly divided into four display areas, namely the upper left area 201, the upper right area 202, the lower left area 203, and the lower right area 204. To facilitate the occupants in each position to clearly see the display situation in the display area, the upper left area 201 can be set as the display area for the left rear seat, the upper right area 202 can be set as the display area for the right rear seat, the lower left area 203 can be set as the display area for the driver's seat, and the lower right area 204 can be set as the display area for the co-pilot seat.

[0095] Assume that there are three occupants in the vehicle cockpit, in the driver's seat, the co-pilot seat, and the left rear seat respectively. After obtaining the heart rates of the three occupants, referring to Figure 3 , the heart rates of the three occupants can be provided to the display device, and the display device can display the heart rate of the driver in the lower left area 203, the heart rate of the occupant in the left rear seat in the upper left area 201, and the heart rate of the occupant in the co-pilot seat in the lower right area 204.

[0096] For example, if you only need to perform ECG testing on the driver, see Figure 4 After obtaining the driver's heart rate, the display device can be controlled to display the driver's heart rate in the lower left corner area 203 of the display screen at the driver's seat marked in the electrocardiogram detection data. In this way, the electrocardiogram detection data of a specific occupant can be accurately and clearly displayed in the designated area of ​​the display screen. In some embodiments, after starting the single-person detection mode, the entire display screen can display only one heart rate display frame area, or it can display the display area corresponding to the driver in multiple preset display areas. In other embodiments, if there are multiple in-vehicle screens, the electrocardiogram detection data of the target occupant can be displayed on the display device closest to the target occupant.

[0097] In the embodiment of the present application, the display device may be, but is not limited to, a vehicle-mounted central control screen, a vehicle-mounted television, or a display screen on a vehicle seat.

[0098] The following is a detailed description of an exemplary implementation of the vehicle-mounted ECG detection method provided by the embodiment of the present application, taking a four-seater vehicle as an example. In the following specific implementation, the full face is used as the requirement for facial features. It should be understood by those skilled in the art that the requirement for facial features may include specific partial facial features, such as the forehead area, the bridge of the nose area, etc.

[0099] Example 1: ECG detection of multiple people

[0100] The specific implementation process of triggering ECG detection of all occupants in the vehicle cabin based on the vehicle status may include the following steps a1 to a9:

[0101] Step a1, detecting vehicle status such as whether the doors are locked and whether the vehicle is started, and collecting images of the interior environment of the vehicle cabin;

[0102] Here, the image of the interior environment of the vehicle cabin may also be the second image captured by the camera group as described above.

[0103] Step a2: if the image of the interior environment of the vehicle cabin contains human features and it is detected that the door is locked or the vehicle is started, actively triggering electrocardiogram detection of all occupants in the vehicle cabin;

[0104] Step a3, identifying the number of passengers in the vehicle cabin and the position of each passenger;

[0105] For example, by performing image recognition or target detection on the image of the interior environment of the vehicle cabin collected in step a1, the number of people contained in the image and the position of each occupant can be obtained. Alternatively, the position of each occupant can be sensed and the number of occupants in the vehicle cabin can be determined by sensors installed on each seat in the vehicle cabin.

[0106] Step a4: Determine the display areas of each occupant on the in-vehicle central control screen according to the number of occupants in the vehicle cockpit and the positions of each occupant.

[0107] Step a5: The camera group takes a second image of the interior of the vehicle cockpit in real time, performs image segmentation on the second image to obtain the occupant avatars of each occupant, and displays the real-time captured occupant avatars in the corresponding display areas of each occupant.

[0108] Taking the camera group including the first camera and the second camera as an example, perform image recognition and image segmentation on the second image captured by the first camera in real time to obtain the occupant avatar in the driver's seat, and control the display device to display the occupant avatar in the driver's seat in the lower left corner area of the in-vehicle central control screen. Perform image recognition and image segmentation on the second image captured by the second camera in real time to obtain the occupant avatars in the co-driver's seat, the left rear seat, and the right rear seat, and control the central control device to display the occupant avatar in the left rear seat in the upper left corner area of the in-vehicle central control screen, display the occupant avatar in the right rear seat in the upper right corner area of the in-vehicle central control screen, and display the occupant avatar in the co-driver's seat in the lower right corner area of the in-vehicle central control screen.

[0109] Step a6: Determine whether the occupant avatars displayed in each display area of the in-vehicle central control screen are all complete. If so, jump to step a8; otherwise, continue with step a7.

[0110] Step a7: If the occupant avatar displayed in a display area is incomplete, issue a prompt to prompt the occupant to actively move their body or adjust their sitting posture, or automatically adjust the shooting angle of the camera, and return to step a6 until the occupant avatars displayed in each display area are all complete.

[0111] For example, if the occupant avatar in the driver's seat is incomplete, the shooting angle of the first camera can be automatically adjusted until the occupant avatar displayed in the lower left corner area is complete. If the occupant avatars in positions other than the driver's seat are incomplete, a prompt can be issued, and the occupant can actively move their body or adjust their sitting posture until the occupant avatars displayed in the display area are complete.

[0112] See Figure 5 As shown, assuming there are four occupants in the current four-seater vehicle, in the driver's seat, the co-driver's seat, the left rear seat, and the right rear seat respectively, divide the in-vehicle central control screen 200 into four display areas, namely the upper left corner area corresponding to the left rear seat, the lower left corner area corresponding to the driver's seat, the upper right corner area corresponding to the right rear seat, and the lower right corner area corresponding to the co-driver's seat. Detect that the facial features display in the upper right corner area is incomplete, and a prompt can be issued to remind the occupant in the right rear seat to move their body or adjust their sitting posture until the display interface presents as Figure 6 As shown, that is, it can display the avatars of the four occupants and the avatars are complete.

[0113] Step a8, after the faces of all the occupants can be fully displayed, prompt the occupants to maintain their positions and record a first video with a predetermined duration. The first video includes the faces of the occupants.

[0114] Step a9, perform frame-by-frame image analysis on the first video to obtain the heart rates of all the occupants, and control the in-vehicle central control device to display the heart rates of the occupants in the display areas corresponding to the positions of the occupants on the in-vehicle central control screen.

[0115] In some embodiments, the heart rate may be the instantaneous heart rate of a certain frame image or the average heart rate of each frame image.

[0116] Embodiment 2: Single-person electrocardiogram detection

[0117] The specific implementation process for triggering the electrocardiogram detection of all the occupants inside the vehicle cockpit based on the vehicle state may include the following steps b1 to step b11:

[0118] Step b1, after the vehicle starts, the occupants inside the vehicle cockpit issue a voice command. The array microphone installed inside the vehicle cockpit receives the voice command and stores the voice command in advance.

[0119] Step b2, determine whether it is necessary to start single-person electrocardiogram detection based on the voice command. If so, continue to step b4; otherwise, continue to step b3.

[0120] Step b3, the voice command is invalid, terminate the electrocardiogram detection, delete the voice command, issue a prompt to remind the user to re-enter the voice command, and return to step b1.

[0121] Step b4, automatically start single-person electrocardiogram detection, read the voice command stored in advance, and perform sound localization analysis to obtain the position of the occupant.

[0122] If the driver issues the voice command, after starting single-person electrocardiogram detection, it can be determined that the number of target occupants to be detected is 1, and the position of the target occupant to be detected can be determined as the driver's seat through sound localization analysis.

[0123] Step b5, take a second image at the position of the occupant to double-check whether the position of the occupant is accurate. If so, continue to step b7; otherwise, the sound localization is incorrect or the position of the occupant has changed, and jump to step b3.

[0124] Step b6, determine the display area of the occupant on the in-vehicle central control screen according to the position of the occupant.

[0125] Steps b7 to step Sb11 are basically the same as steps a5 to a9, except that only the heart rate of the driver needs to be determined and the heart rate is displayed in the lower left corner area of the in-vehicle central control screen.

[0126] Assume that the occupant who issues the voice command is the driver in the driver's seat. It can be determined that the lower left corner area of the in-vehicle central control screen is the display area for this occupant. In steps b7 and b8, if the face of the occupant in the lower left corner area is incomplete, the shooting angle of the first camera can be automatically adjusted until the display interface presents as Figure 7 the situation shown, that is, it can clearly show the complete head portrait of the driver.

[0127] The in-vehicle electrocardiogram detection method provided by the embodiments of the present application can respond to the vehicle state or user instructions to determine the number and positions of occupants in real time, and perform multi-person electrocardiogram detection or single-person electrocardiogram detection according to the number and positions of occupants, effectively improving the intelligence and personalization of in-vehicle electrocardiogram detection, and at the same time can also improve the efficiency and accuracy of in-vehicle electrocardiogram detection.

[0128] In-Vehicle Electrocardiogram Detection Device

[0129] Figure 8 shows a schematic structural diagram of an in-vehicle electrocardiogram detection device 800 provided by an embodiment of the present application. Refer to Figure 8 and this in-vehicle electrocardiogram detection device 800 may include:

[0130] A determination unit 810, configured to determine the number and positions of target occupants to be detected in response to the vehicle state and / or user instructions;

[0131] A video acquisition unit 820, configured to acquire a first video of the face of the target occupant to be detected;

[0132] An image analysis unit 830, configured to obtain electrocardiogram detection data of the member to be detected based on image analysis of the first video;

[0133] A display unit 840, configured to display the electrocardiogram detection data of the target occupant to be detected in a predetermined area of the display device according to the number and positions of the target occupants to be detected.

[0134] In some embodiments, the determination unit 810 may be configured to: detect the vehicle state and / or receive user instructions; if the vehicle state meets the first predetermined condition and / or the user instruction indicates that multi-person electrocardiogram detection needs to be performed, trigger the multi-person electrocardiogram detection function, and determine the seats in the vehicle cockpit where there are occupants sitting; determine the number of seats where each occupant is sitting as the number of target occupants to be detected, and record the seat information as the corresponding positions of the target occupants to be detected.

[0135] In some embodiments, the user instruction is a voice instruction, and the determination unit 810 can be used to: in response to the vehicle state and / or the user instruction, determine the number and position of the target occupant to be detected, including: performing speech recognition on the voice instruction to determine the user intention, triggering the single-person electrocardiogram detection function when the user intention indicates that single-person electrocardiogram detection is required, and performing sound localization analysis based on the voice instruction to obtain the position of the target occupant to be detected.

[0136] In some embodiments, the determination unit 810 can also be used to: after performing sound localization analysis based on the voice instruction to obtain the position of the target occupant to be detected, acquire a second image at the position of the target occupant to be detected, and perform target detection on the second image to verify whether the position of the target occupant to be detected is correct.

[0137] In some embodiments, the video acquisition unit 820 can also be used to: before acquiring the first video including the face of the target occupant to be detected, acquire a second image at the position of the target occupant to be detected; use the second image to determine whether the positions of the target occupants to be detected meet the requirements; if the positions of any one or more target occupants to be detected do not meet the requirements, adjust the positions of the corresponding target occupants to be detected and / or control the corresponding camera to adjust the shooting angle so that the positions of all target occupants to be detected meet the requirements.

[0138] In some embodiments, the video acquisition unit 820 can also be used to: determine whether the target occupant with an unqualified position includes the driver and detect the vehicle state; if the target occupant with an unqualified position includes the driver and the vehicle is detected to be in a driving state, control the first camera to adjust its own shooting angle so that the first camera can capture the face of the driver and the face is within the region of interest, and the first camera is at least used to collect the first video of the driver; if the target occupant with an unqualified position does not include the driver or the vehicle is detected to be in a stationary state, issue a position adjustment prompt, and the position adjustment prompt is used to instruct the target occupant to be detected to adjust its own position so that the first camera and / or the second camera can capture the face of the target occupant to be detected and the face is within the region of interest, and the second camera is at least used to collect the first video of the occupants other than the driver.

[0139] In some embodiments, the video acquisition unit 820 can also be used to: if the number of target occupants to be detected is multiple, determine the display areas of the target occupants to be detected on the display screen according to the positions of the target occupants to be detected and control the display device to display the faces of the corresponding target occupants to be detected in the second image in the respective display areas; if the facial features displayed in any one or more display areas do not meet the requirements or the faces of the corresponding target occupants to be detected cannot be displayed normally, determine that the positions of the corresponding one or more target occupants to be detected do not meet the requirements.

[0140] In some embodiments, the image analysis unit 830 is configured to: segment the first video according to the positions and quantities of the target occupants to be detected, so as to generate an image set corresponding to each target occupant to be detected, where each image set includes multiple frames of region-of-interest images of the same target occupant to be detected, and each frame of region-of-interest image includes the face of the corresponding target occupant to be detected; perform image analysis on the image sets of each target occupant to be detected to obtain the electrocardiogram detection data of each target occupant to be detected.

[0141] In some embodiments, the display unit 840 is configured to: if the number of target occupants to be detected is multiple, determine the display areas matched with each target occupant to be detected according to the positions of each target occupant to be detected, and control the display device to display the electrocardiogram detection data of the target occupants to be detected matched in each display area on the display screen; and, if the number of target occupants to be detected is one, control the display device to display the electrocardiogram detection data of the target occupant to be detected in a predetermined area corresponding to the position of the target occupant to be detected on the display screen.

[0142] In practical applications, the vehicle-mounted electrocardiogram detection device 800 can be implemented by software, hardware, or a combination of both.

[0143] Computing Device

[0144] Figure 9 It is a structural schematic diagram of a computing device 900 provided by an embodiment of the present application. The computing device 900 includes: a processor 910 and a memory 920.

[0145] Wherein, the processor 910 can be connected to the memory 920. The memory 920 can be used to store the program code and data. Therefore, the memory 920 can be an internal storage unit of the processor 910, an external storage unit independent of the processor 910, or a component including an internal storage unit of the processor 910 and an external storage unit independent of the processor 910.

[0146] The memory 920 can include a read-only memory and a random access memory, and provide instructions and data to the processor 910. A part of the processor 910 can also include a non-volatile random access memory. For example, the processor 910 can also store information about the device type.

[0147] The processor 910 may be a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a CPLD, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. When the computing device 900 is running, the processor 910 executes the computer-executable instructions in the memory 920 to perform the operations / calculation steps of the vehicle electrocardiogram detection method in any of the above embodiments.

[0148] Optionally, the computing device 900 may further include components such as a communication interface and a bus.

[0149] It should be understood that the computing device 900 according to the embodiments of the present application may correspond to the corresponding subject executing the methods in the embodiments of the present application, and the above and other operations and / or functions of each module in the computing device 900 respectively implement the corresponding processes of the methods in this embodiment. For the sake of brevity, they will not be described in detail here.

[0150] In a specific application, the computing device 900 may be implemented as a microcontroller in a vehicle central control system, a vehicle cockpit controller, or other similar devices.

[0151] Computer-Readable Storage Medium

[0152] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the program is run by a processor, the processor is caused to execute the above-mentioned vehicle electrocardiogram detection method. Here, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, semiconductor system, device, or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0153] Computer Program Product

[0154] An embodiment of the present application also provides a computer program product, which includes a computer program that, when run by a processor, causes the processor to execute the above-mentioned vehicle-mounted electrocardiogram detection method. Here, the programming language of the computer program product can be one or more, and the programming language can include, but is not limited to, object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as the "C" language.

[0155] In-Vehicle Electrocardiogram Detection System

[0156] An embodiment of the present application also provides a vehicle-mounted electrocardiogram detection system 1000, which may include the aforementioned computing device 900, camera group 1100, and display device 1200. Among them, the camera group 1100 may include one or more cameras installed in the vehicle cockpit for collecting a first video containing the face of the target occupant to be detected; the display device 1200 may include a display screen 200 for displaying the electrocardiogram detection data of the target occupant to be detected in a predetermined area corresponding to the position of the target occupant to be detected on the display screen 200 under the control of the computing device 900.

[0157] In a specific application, each camera in the camera group 1100 may be connected to the computing device 900 in a wired or wireless manner respectively, and transmit the collected first video to the computing device 900.

[0158] In one example, the camera group 1100 includes a first camera 1101 and a second camera 1102. The first camera is used to collect the first video of the driver, and the second camera is used to collect the first video of all occupants except the driver. The cameras in the camera group 1100 can be installed at any set position in the vehicle cockpit, such as on the sun visor, the main in-vehicle display, the ceiling-mounted TV, etc.

[0159] In one example, the user instruction is a voice instruction; the vehicle-mounted electrocardiogram detection system 1000 may further include: an array microphone 1300 for collecting the user instruction and performing sound localization analysis based on the user instruction to obtain the position of the target occupant to be detected. Of course, in addition to the array microphone, any other sound collection device applicable to the vehicle-mounted environment can be used to collect the voice instruction.

[0160] In one example, the vehicle-mounted electrocardiogram detection system 1000 may further include: a pickup 1400 for performing sound localization analysis based on the user instruction to obtain the position of the target occupant to be detected.

[0161] Vehicle

[0162] An embodiment of the present application also provides a vehicle 1500, in which a computing device 900, an in-vehicle electrocardiogram detection system 1000 or an in-vehicle electrocardiogram detection device 800 is installed. Figure 11 An example of the vehicle 1500 is shown.

[0163] Finally, it should be noted that those of ordinary skill in the art can understand that in order to enable readers to better understand the present application, many technical details are proposed in the embodiments of the present application. However, even without these technical details and various changes and modifications based on the above embodiments, the technical solutions required to be protected by the various claims of the present application can be basically implemented. Therefore, in practical applications, various changes can be made in form and details to the above embodiments without departing from the spirit and scope of the present application.

Claims

1. A vehicle-mounted electrocardiogram detection method, characterized in that, Including: Determine the number and position of the target occupants to be detected in response to the vehicle state and / or user instructions; Obtain a first video of the faces of the target occupants to be detected; Obtain electrocardiogram detection data of the target occupants to be detected based on image analysis of the first video; According to the number and position of the target occupants to be detected, display the electrocardiogram detection data of the target occupants to be detected in a predetermined area of the display device.

2. The vehicle-mounted electrocardiogram detection method according to claim 1, wherein Determine the number and position of the target occupants to be detected in response to the vehicle state and / or user instructions, including: Detect the vehicle state and / or receive user instructions; If the vehicle state meets the first predetermined condition and / or the user instruction indicates that multi-person electrocardiogram detection is required, trigger the multi-person electrocardiogram detection function and determine the seats in the vehicle cockpit where there are occupants; Determine the number of seats with occupants as the number of target occupants to be detected, and record the seat information as the positions of the corresponding target occupants to be detected.

3. The vehicle-mounted electrocardiogram detection method according to claim 2, wherein, Determine the number and position of the target occupants to be detected in response to the vehicle state and / or user instructions, further including: after performing sound localization analysis based on a voice instruction to obtain the position of the target occupants to be detected, obtain a second image at the position of the target occupants to be detected, and perform object detection on the second image to verify whether the position of the target occupants to be detected is correct.

4. The vehicle-mounted electrocardiogram detection method according to claim 1, characterized in that, Before obtaining the first video of the faces of the target occupants to be detected, further including: Obtain a second image at the position of the target occupants to be detected; Use the second image to determine whether the positions of the target occupants to be detected meet the requirements; If the position of any one or more of the target occupants to be detected does not meet the requirements, adjust the positions of the corresponding target occupants to be detected and / or control the corresponding camera to adjust the shooting angle so that the positions of all the target occupants to be detected meet the requirements.

5. The vehicle-mounted electrocardiogram detection method according to claim 4, wherein, Adjust the positions of the corresponding target occupants to be detected and / or control the camera to adjust the shooting angle, including: Determine whether the driver is included among the target occupants to be detected with non-compliant positions, and detect the vehicle state; If the driver is included among the target occupants to be detected with non-compliant positions and the vehicle is detected to be in a driving state, control the first camera to adjust its own shooting angle so that the first camera can capture the driver's face and the face is within the region of interest. The first camera is at least used to collect the first video of the driver's face; If the driver is not included among the target occupants to be detected with non-compliant positions or the vehicle is detected to be in a stationary state, issue a position adjustment prompt, which is used to instruct the target occupants to be detected to adjust their own positions so that the first camera and / or the second camera can capture the faces of the target occupants to be detected and the faces are within the region of interest. The second camera is at least used to collect the first video of the occupants other than the driver.

6. The vehicle-mounted electrocardiogram detection method according to claim 4, characterized in that, Use the second image to determine whether the positions of the target occupants to be detected meet the requirements, including: If the number of target occupants to be detected is multiple, determine the display areas of the target occupants to be detected on the display screen according to the positions of the target occupants to be detected, and control the display device to display the faces of the corresponding target occupants to be detected in the second image in each display area; If the facial features displayed in any one or more display areas do not meet the requirements or the face of the corresponding target occupant to be detected cannot be normally displayed, it is determined that the positions of the corresponding one or more target occupants to be detected do not meet the requirements.

7. The vehicle-mounted electrocardiogram detection method according to any one of claims 1-6, characterized in that Perform image analysis on the first video to obtain the electrocardiogram detection data of the target member to be detected, including: Perform image segmentation on the first video according to the positions and quantities of the target occupants to be detected to generate an image set corresponding to each target occupant to be detected. Each image set contains multiple frames of region-of-interest images of the same target occupant to be detected, and each frame of region-of-interest image contains the face of the corresponding target occupant to be detected; Perform image analysis on the image sets of each target occupant to be detected respectively to obtain the electrocardiogram detection data of each target occupant to be detected.

8. A vehicle-mounted electrocardiogram detection device, characterized in that, Including: A determination unit, configured to determine the quantity and position of the target occupants to be detected in response to the vehicle state and / or user instructions; A video acquisition unit, configured to acquire a first video of the faces of the target occupants to be detected; An image analysis unit, configured to obtain the electrocardiogram detection data of the target member to be detected based on the image analysis of the first video; A display unit, configured to display the electrocardiogram detection data of the target occupants to be detected in a predetermined area of the display device according to the quantity and position of the target occupants to be detected.

9. A computer-readable storage medium having program instructions stored thereon, characterized in that, The program instructions, when executed by a computer, cause the computer to execute the vehicle-mounted electrocardiogram detection method according to any one of claims 1-8.

10. A vehicle-mounted electrocardiogram detection system, characterized in that, Including a computing device, a camera group, and a display device; The computing device includes a processor and a memory. The memory stores program instructions, and the program instructions, when executed by the processor, implement the vehicle-mounted electrocardiogram detection method according to any one of claims 1-8; The camera group includes one or more cameras installed in the vehicle cockpit, and is configured to collect a first video of the faces of the target occupants to be detected; The display device includes a display screen, and is configured to display the electrocardiogram detection data of the target occupants to be detected in a predetermined area corresponding to the positions of the target occupants to be detected on the display screen under the control of the computing device.

11. The vehicle-mounted electrocardiogram detection system according to claim 10, wherein The camera group includes a first camera and a second camera. The first camera is configured to collect a first video of the driver, and the second camera is configured to collect a first video of all occupants except the driver.

12. The vehicle-mounted electrocardiogram detection system according to claim 10, characterized in that, The display device is specifically configured to: When the quantity of the target occupants to be detected is multiple, divide the display screen into multiple display areas, and display the electrocardiogram detection data of the matching target occupants to be detected in each display area of the display screen under the control of the computing device; When the quantity of the target occupants to be detected is one, display the electrocardiogram detection data of the target occupant to be detected in a predetermined area on the display screen under the control of the display device.

13. The vehicle-mounted electrocardiogram detection system according to claim 10, wherein The vehicle-mounted electrocardiogram detection system further includes: an array microphone and / or a pickup. The array microphone is configured to collect user instructions and perform sound localization analysis based on the user instructions to obtain the positions of the target occupants to be detected; The pickup is configured to perform sound localization analysis based on the user instructions to obtain the positions of the target occupants to be detected.

14. A vehicle, characterized in that, Including the vehicle-mounted electrocardiogram detection device according to claim 8 or the vehicle-mounted electrocardiogram detection system according to any one of claims 10 to 13.

Images