Passenger feature information extraction method and device, controller, vehicle and storage medium

Abstract

The application discloses a kind of passenger feature information extraction method and device, controller, vehicle and storage medium, method includes: obtaining the current video frame of passenger and reference feature point set;Wherein, reference feature point set includes the relative position relationship between the feature points of passenger;Current video frame is extracted to obtain the identifiable feature point set of passenger;Based on the relative position relationship between the feature points in identifiable feature point set and reference feature point set, predict unrecognizable feature point set;Based on identifiable feature point set and unrecognizable feature point set, obtain the feature information of passenger.The method of the application has the advantages of not needing to increase hardware, low cost, strong robustness, higher completeness and accuracy of passenger feature information extraction, important research significance for subsequent automobile safety components and system development, passenger restraint system parameter matching and optimization.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a method for extracting occupant feature information, a computer-readable storage medium, a controller, an occupant feature information extraction device, and a vehicle. Background Technology

[0002] Current vehicle cabin perception solutions often utilize cameras to acquire images of occupants. However, due to the varying postures of occupants and the complexity of the vehicle's interior environment, coupled with the fixed camera positions, the images captured by these cameras often only capture partial information, making it impossible to calculate complete key feature points for all occupants in this scenario. To address this challenge, a common approach is to train the system on a large occupant information database, comparing the available feature points with the database to predict and compensate for the remaining feature points.

[0003] However, this method firstly has high requirements for the dataset and training algorithm, and high requirements for hardware resources and cost. Secondly, it cannot make predictions based on the specific body characteristics of the current occupants, resulting in poor prediction accuracy. For example, some occupants have a longer upper body than ordinary people, and the camera only captures the feature points of the upper body. When predicting the feature points of the lower body, the deviation from reality will be too large. Finally, the method has poor interpretability, unstable prediction results, and poor robustness. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the first objective of this invention is to propose a method for extracting occupant feature information, which has the advantages of not requiring additional hardware, low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction. This method has significant research value for the subsequent development of automotive safety components and systems, as well as for the matching and optimization of occupant restraint system parameters.

[0005] A second objective of this invention is to provide a computer-readable storage medium.

[0006] The third objective of this invention is to provide a controller.

[0007] The fourth objective of this invention is to provide a device for extracting occupant characteristic information.

[0008] The fifth objective of this invention is to provide a vehicle.

[0009] To achieve the above objectives, a first aspect of the present invention proposes a method for extracting occupant feature information. The method includes: acquiring a current video frame of the occupant and a reference feature point set; wherein the reference feature point set includes the relative positional relationship between feature points of the occupant; extracting feature points from the current video frame to obtain a set of identifiable feature points of the occupant; predicting a set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the reference feature point set; and obtaining the occupant's feature information based on the set of identifiable feature points and the set of unidentified feature points.

[0010] The occupant feature information extraction method according to embodiments of the present invention first acquires the current video frame and a reference feature point set of the occupant, and then extracts feature points from the current video frame to obtain a set of identifiable feature points of the occupant. Next, based on the relative positional relationship between feature points in the identifiable feature point set and the reference feature point set, a set of unidentified feature points is predicted. Finally, based on the identifiable feature point set and the unidentified feature point set, the occupant's feature information is obtained. Therefore, this method has the advantages of not requiring additional hardware, low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction. It has significant research value for the subsequent development of automotive safety components and systems, and for the matching and optimization of occupant restraint system parameters.

[0011] In addition, the occupant feature information extraction method according to the above embodiments of the present invention may also have the following additional technical features:

[0012] According to one embodiment of the present invention, predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points includes: obtaining the relative positional relationship between the identifiable feature points and the unidentified feature points in the set of unidentified feature points from the set of reference feature points based on the identifiable feature points in the set of identifiable feature points; and predicting the unidentified feature points based on the relative positional relationship and the identifiable feature points.

[0013] According to an embodiment of the present invention, before predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points, the method further includes: obtaining the number of identifiable feature points in the set of identifiable feature points and the number of reference feature points in the set of reference feature points; predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points when the number of reference feature points is greater than or equal to the number of identifiable feature points; and using the set of identifiable feature points as the feature information of the occupant when the number of reference feature points is less than the number of identifiable feature points.

[0014] According to one embodiment of the present invention, when the number of reference feature points is less than the number of identifiable feature points, the method further includes: obtaining the relative positional relationship between identifiable feature points in the set of identifiable feature points; and updating the set of reference feature points based on the relative positional relationship.

[0015] According to one embodiment of the present invention, the method further includes: determining whether the occupant is a new occupant based on the reference feature point set; and registering the occupant as a user if the occupant is a new occupant.

[0016] According to one embodiment of the present invention, the identifiable feature points and the unidentified feature points respectively include one or more of the following: facial feature points, shoulder feature points, elbow feature points, wrist feature points, hip feature points, knee feature points, and ankle feature points of the occupant.

[0017] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the above-described method for extracting occupant feature information.

[0018] According to the computer-readable storage medium of the present invention, by executing the above-described occupant feature information extraction method, it has the advantages of low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, which is of great research significance for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0019] To achieve the above objectives, a third aspect of the present invention provides a controller, comprising: a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the above-described occupant feature information extraction method.

[0020] According to the controller of the present invention, by executing the above-described occupant feature information extraction method, it has the advantages of low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, which is of great research significance for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0021] To achieve the above objectives, a fourth aspect of the present invention provides an occupant feature information extraction device, the device comprising: a first acquisition module for acquiring a current video frame of the occupant; a second acquisition module for acquiring a reference feature point set of the occupant; wherein the reference feature point set includes the relative positional relationship between feature points of the occupant; a feature extraction module for extracting feature points from the current video frame to obtain a set of identifiable feature points of the occupant; and a feature recognition module for predicting a set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the reference feature point set, and obtaining feature information of the occupant based on the set of identifiable feature points and the set of unidentified feature points.

[0022] According to an embodiment of the present invention, an occupant feature information extraction device comprises: a first acquisition module acquiring the current video frame of the occupant; a second acquisition module acquiring a reference feature point set of the occupant; a feature extraction module extracting feature points from the current video frame to obtain a set of identifiable feature points of the occupant; and a feature recognition module predicting a set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the reference feature point set, and obtaining the occupant's feature information based on the set of identifiable feature points and the set of unidentified feature points. Therefore, this device has the advantages of not requiring additional hardware, low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, and is of significant research value for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0023] To achieve the above objectives, a fifth aspect of the present invention provides a vehicle including the controller described above, or the occupant feature information extraction device described above.

[0024] The vehicle according to the embodiments of the present invention, through the above-described controller or occupant feature information extraction device, has the advantages of low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, which is of great research significance for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0025] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] Figure 1 This is a flowchart of a method for extracting occupant feature information according to an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of feature points according to an embodiment of the present invention;

[0028] Figure 3This is a block diagram of a controller according to an embodiment of the present invention;

[0029] Figure 4 This is a block diagram of an occupant feature information extraction device according to an embodiment of the present invention;

[0030] Figure 5 A schematic block diagram of a vehicle according to an embodiment of the present invention;

[0031] Figure 6 This is a block diagram of a vehicle according to another embodiment of the present invention. Detailed Implementation

[0032] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0033] The following description, with reference to the accompanying drawings, outlines the occupant feature information extraction method, computer-readable storage medium, controller, occupant feature information extraction device, and vehicle according to embodiments of the present invention.

[0034] This provides crucial input for timely adjustments to occupant safety strategies. For example, if a camera detects that a user's seating position is too close to the steering wheel, the airbag control unit can adjust the airbag ignition timing based on this input information to protect the occupant and prevent secondary injury. Current vehicle cabin perception solutions often utilize cameras such as RGB cameras, 3D TOF cameras, and binocular cameras to acquire image information of occupants inside the vehicle. Image processing extracts occupant features, such as facial features and key body feature points. Common products include DMS (Discrete Monitoring System) to determine if an occupant is drowsy or distracted, and 3D TOF to recognize gestures for entertainment control functions.

[0035] The varying postures of vehicle occupants and the complex environment inside the vehicle, coupled with the fixed positions of cameras, mean that the information captured by cameras in the car cabin is often only partial. Current products and patents, when the camera is partially obstructed, occupants move outside the camera's field of view (FOV), or occupant posture obscures some information, result in incomplete image information. In such cases, it's impossible to fully extract occupant feature information, and most current products or patents simply cannot identify them. Some advanced products or patents can extract and deduce the approximate location of occupant feature points from a large occupant information database for compensation. This compensation method uses machine learning combined with training on a large occupant information database to extract occupant feature information and predict the remaining undetected feature points. However, this method cannot predict based on the specific body shape characteristics of each occupant, resulting in poor accuracy and interpretability. Consequently, it cannot provide effective input for the formulation of subsequent safety strategies, severely impacting the implementation of safety policies.

[0036] To address the current challenge of incomplete feature points in cockpit camera monitoring for occupant feature extraction due to incomplete image data, this invention proposes a time-series-based method for occupant feature extraction. This provides reliable input for the formulation of future occupant safety strategies and has significant implications for the development of in-vehicle safety technology.

[0037] Figure 1 This is a flowchart of a method for extracting occupant feature information according to an embodiment of the present invention.

[0038] like Figure 1 As shown, the occupant feature information extraction method of this invention may include the following steps:

[0039] S1, obtain the current video frame and reference feature point set of the occupant; wherein, the reference feature point set includes the relative positional relationship between the feature points of the occupant.

[0040] S2, extract feature points from the current video frame to obtain a set of identifiable feature points of the occupants.

[0041] S3 predicts the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points.

[0042] S4, based on the set of identifiable feature points and the set of unidentified feature points, obtain the occupant's feature information.

[0043] Specifically, after the vehicle is powered on, the camera inside the passenger compartment starts working, capturing a sequence of video frames of the occupants and transmitting the current video frame to the controller. The controller's ROM also stores a set of reference feature points. The controller uses deep learning algorithms such as CNN and OPENPOSE to identify and extract feature points from the current video frame, thus obtaining a set of identifiable feature points for the occupants. The controller compares this set of identifiable feature points with the reference set, comparing the relative positions of the feature points in the two sets. This identifies feature points that belong to the reference set but not to the set of identifiable feature points; these are the unidentified feature points. The controller can then use both the identifiable and unidentified feature point sets to obtain the occupants' characteristic information.

[0044] For example, current traditional safety product development, such as airbag deployment, is designed to meet standard regulatory conditions, using standard dummies and standard seating postures. This fails to reflect the actual driving conditions of occupants. The occupant feature information extraction method of this invention can obtain the occupant's real-world driving conditions. For instance, some female drivers tend to drive too close to the steering wheel. In the event of a crash, the airbag deployed from the steering wheel can cause secondary injuries to the driver (deployment speed exceeding 300 kph, far greater than the injuries caused by the accident itself). Using the occupant feature information extraction method of this invention, the user's actual spatial position can be perceived, such as the head being too close to the steering wheel. When designing safety strategies, the controller can identify this scenario and adjust the airbag deployment timing or even prevent it from deploying. Furthermore, the controller can also adjust actuators in other positions, such as seat position and seatbelt force limiters. The goal is to first obtain the current occupant's spatial position → make decisions → and then coordinate with other actuators to ensure personalized, real-time safety.

[0045] Therefore, the occupant feature information extraction method of the present invention is preferably based on the time series dimension. The postures of occupants in the car are different and the in-car environment is complex, but the camera position is fixed, which means that the information captured by the camera in the car cabin can often only obtain partial information. Therefore, the occupant feature information extraction method of the present invention is based on the time series of the user's car use process, and takes the occupant feature information that can be completely captured in the time series as a reference. It predicts the complete feature point information of the occupant by relying on the relative position between the occupant feature points at this time and the information of the partial feature points captured at the moment. Compared with the existing machine learning approach of automatically predicting by training and learning from sample sets, it has the characteristics of low resource consumption and high prediction accuracy.

[0046] According to one embodiment of the present invention, such as Figure 2As shown, identifiable and unidentifiable feature points include one or more of the following: facial feature points, shoulder feature points, elbow feature points, wrist feature points, hip feature points, knee feature points, and ankle feature points. Therefore, the feature point set contains the relative positional relationships between the various feature points, thus describing the occupant's body shape and posture.

[0047] According to one embodiment of the present invention, predicting a set of unidentified feature points based on the relative positional relationship between feature points in a set of identifiable feature points and a set of reference feature points includes: obtaining the relative positional relationship between identifiable feature points in the set of identifiable feature points and unidentified feature points in the set of unidentified feature points from the set of reference feature points based on identifiable feature points in the set of identifiable feature points; and predicting unidentified feature points based on the relative positional relationship and identifiable feature points.

[0048] Specifically, after the vehicle is powered on, the camera inside the passenger compartment starts working, capturing a sequence of video frames of the occupants. The controller then extracts feature points from the current video frames to obtain a set of identifiable feature points F for the occupants. ij Where i represents the occupant's sequence number and j represents the video frame sequence number. The controller retrieves the reference feature point set R corresponding to that occupant from the ROM. i Based on the reference feature point set R i The relative positional relationships of each feature point in the data are used to predict the set of identifiable feature points F. ij Unidentified feature points in the set of identifiable feature points, for example, the set of identifiable feature points F. ij If only the shoulder feature point coordinates are recognized, but the hip feature points are not recognized and therefore cannot be output, then the coordinates can be determined based on the reference feature point set R. i Based on the spatial relationship between the mid-shoulder and the hip, and combined with the current shoulder feature point coordinate information, the spatial coordinate position of the hip feature point is predicted.

[0049] According to one embodiment of the present invention, before predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points, the method further includes: obtaining the number of identifiable feature points in the set of identifiable feature points and the number of reference feature points in the set of reference feature points; predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points when the number of reference feature points is greater than or equal to the number of identifiable feature points; and using the set of identifiable feature points as the feature information of the occupants when the number of reference feature points is less than the number of identifiable feature points.

[0050] Specifically, the controller obtains the set of identifiable feature points F of the occupants. ij Then, obtain the set of identifiable feature points F. ij The number of identifiable feature points in the dataset is compared with the reference feature point set R.i The controller compares the number of reference feature points in the reference feature point set R. When the number of reference feature points is greater than or equal to the number of identifiable feature points, the controller determines the reference feature point set R. i The relative positional relationships of each feature point in the data are used to predict the set of identifiable feature points F. ij Unidentified feature points. When the number of reference feature points is less than the number of identifiable feature points, the controller will use the set of identifiable feature points as the occupant's feature information.

[0051] According to one embodiment of the present invention, when the number of reference feature points is less than the number of identifiable feature points, the method further includes: obtaining the relative positional relationship between identifiable feature points in the set of identifiable feature points; and updating the set of reference feature points based on the relative positional relationship.

[0052] Specifically, when the number of reference feature points is less than the number of identifiable feature points, the controller calculates the set of identifiable feature points F. ij The relative positional relationships of the feature points are determined and updated in the reference feature point set R. i Save to ROM for later retrieval. Due to the reference feature point set R i The information is stored in the controller's ROM and can be read even after the vehicle is powered off and then powered on again. Theoretically, over the vehicle's lifespan, as the user's usage time increases, the reference set R that records complete feature point information will grow. i The more complete the system, the more reasonable the spatial relationship between the occupant's characteristic points.

[0053] According to one embodiment of the present invention, the method further includes: determining whether the occupant is a new occupant based on a set of reference feature points; and registering the occupant as a user if the occupant is a new occupant.

[0054] Specifically, the controller uses the existing feature point set F ij To identify whether a passenger is a new passenger, if the passenger is a new passenger and their information is not saved in the database, refer to the feature point set R. i If the value is empty, the controller will directly extract the current occupant feature point information into the reference feature point set R. i It outputs the occupant's characteristic information and registers the occupant as a new user. If the occupant has previously entered the vehicle and their characteristic information is stored in the database, it refers to the feature point set R. i If not empty, the controller can use the existing feature point set F ij With reference feature point set R i Compare them.

[0055] In summary, in scenarios where cockpit cameras capture incomplete occupant information, the method of this invention has the advantages of low hardware resources, low cost, and high accuracy in predicting feature points. Furthermore, it can be automatically updated and optimized based on user body shape characteristics, resulting in higher completeness and accuracy of occupant feature information extraction. This has significant research value for the development of subsequent automotive safety components and systems, as well as for the matching and optimization of occupant restraint system parameters.

[0056] In summary, the occupant feature information extraction method according to embodiments of the present invention first obtains the current video frame and a reference feature point set of the occupant, and then extracts feature points from the current video frame to obtain a set of identifiable feature points of the occupant. Next, based on the relative positional relationship between feature points in the identifiable feature point set and the reference feature point set, a set of unidentified feature points is predicted. Finally, based on the identifiable feature point set and the unidentified feature point set, the occupant's feature information is obtained. Therefore, this method has the advantages of not requiring additional hardware, low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction. It has significant research value for the subsequent development of automotive safety components and systems, and for the matching and optimization of occupant restraint system parameters.

[0057] Corresponding to the above embodiments, the present invention also proposes a computer-readable storage medium.

[0058] The computer-readable storage medium of this invention stores a program that, when executed by a processor, implements the above-described method for extracting occupant feature information.

[0059] According to the computer-readable storage medium of the present invention, by executing the above-described occupant feature information extraction method, it has the advantages of low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, which is of great research significance for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0060] Corresponding to the above embodiments, the present invention also proposes a controller.

[0061] Figure 3 This is a block diagram of a controller according to an embodiment of the present invention.

[0062] like Figure 3 As shown, the controller 200 of this embodiment includes: a memory 210, a processor 220, and a program stored in the memory 210 and executable on the processor 220. When the processor 220 executes the program, it implements the above-mentioned occupant feature information extraction method.

[0063] According to the controller of the present invention, by executing the above-described occupant feature information extraction method, it has the advantages of low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, which is of great research significance for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0064] Figure 4 This is a block diagram of an occupant feature information extraction device according to an embodiment of the present invention.

[0065] like Figure 4 As shown, the occupant feature information extraction device 100 of this embodiment may include: a first acquisition module 110, a second acquisition module 120, a feature extraction module 130, and a feature recognition module 140.

[0066] The first acquisition module 110 is used to acquire the current video frame of the occupant. The second acquisition module 120 is used to acquire a reference feature point set of the occupant; wherein, the reference feature point set includes the relative positional relationship between the feature points of the occupant. The feature extraction module 130 is used to extract feature points from the current video frame to obtain a set of identifiable feature points of the occupant. The feature recognition module 140 is used to predict a set of unidentified feature points based on the relative positional relationship between the feature points in the set of identifiable feature points and the reference feature point set, and to obtain the feature information of the occupant based on the set of identifiable feature points and the set of unidentified feature points.

[0067] According to one embodiment of the present invention, the feature recognition module 140 predicts the set of unrecognized feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points. Specifically, it is used to: obtain the relative positional relationship between identifiable feature points in the set of identifiable feature points and unrecognized feature points in the set of unrecognized feature points from the set of reference feature points based on identifiable feature points in the set of identifiable feature points; and predict unrecognized feature points based on the relative positional relationship and identifiable feature points.

[0068] According to an embodiment of the present invention, before predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points, the feature identification module 140 is further configured to: obtain the number of identifiable feature points in the set of identifiable feature points and the number of reference feature points in the set of reference feature points; if the number of reference feature points is greater than or equal to the number of identifiable feature points, predict the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points; if the number of reference feature points is less than the number of identifiable feature points, use the set of identifiable feature points as the feature information of the occupants.

[0069] According to an embodiment of the present invention, when the number of reference feature points is less than the number of identifiable feature points, the feature recognition module 140 is further configured to obtain the relative positional relationship between identifiable feature points in the set of identifiable feature points; and update the set of reference feature points based on the relative positional relationship.

[0070] According to one embodiment of the present invention, the feature recognition module 140 is further configured to determine whether the passenger is a new passenger based on a set of reference feature points; and to register the passenger as a user if the passenger is a new passenger.

[0071] According to one embodiment of the present invention, the identifiable feature points and the unidentified feature points respectively include one or more of the following: facial feature points, shoulder feature points, elbow feature points, wrist feature points, hip feature points, knee feature points, and ankle feature points of the occupant.

[0072] It should be noted that for details not disclosed in the occupant feature information extraction device of this embodiment, please refer to the details disclosed in the occupant feature information extraction method of this embodiment, which will not be repeated here.

[0073] According to an embodiment of the present invention, an occupant feature information extraction device comprises: a first acquisition module acquiring the current video frame of the occupant; a second acquisition module acquiring a reference feature point set of the occupant; a feature extraction module extracting feature points from the current video frame to obtain a set of identifiable feature points of the occupant; and a feature recognition module predicting a set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the reference feature point set, and obtaining the occupant's feature information based on the set of identifiable feature points and the set of unidentified feature points. Therefore, this device has the advantages of not requiring additional hardware, low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, and is of significant research value for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0074] Corresponding to the above embodiments, the present invention also proposes a vehicle.

[0075] like Figure 5 As shown, the vehicle 300 of this embodiment includes the controller 200 described above; or, as... Figure 6 As shown, the vehicle 300 of this embodiment of the invention also includes the occupant feature information extraction device 100 described above.

[0076] The vehicle according to the embodiments of the present invention, through the above-described controller or occupant feature information extraction device, has the advantages of low cost, strong robustness, and higher completeness and accuracy of occupant feature information extraction, which is of great research significance for the subsequent development of automotive safety components and systems, and the matching and optimization of occupant restraint system parameters.

[0077] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0078] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0079] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0080] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0081] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0082] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for extracting occupant feature information, characterized in that, The method includes: Obtain the current video frame and reference feature point set of the occupant; wherein, the reference feature point set includes the relative positional relationship between the feature points of the occupant; Feature point extraction is performed on the current video frame to obtain a set of identifiable feature points of the occupant; Based on the relative positional relationship between feature points in the identifiable feature point set and the reference feature point set, predict the unidentified feature point set; The occupant's feature information is obtained based on the set of identifiable feature points and the set of unidentified feature points; The step of predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points includes: Based on the identifiable feature points in the identifiable feature point set, the relative positional relationship between the identifiable feature points and the unidentified feature points in the unidentified feature point set is obtained from the reference feature point set; The unidentified feature points are predicted based on the relative positional relationship and the identifiable feature points.

2. The method according to claim 1, characterized in that, Before predicting the set of unidentified feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points, the method further includes: Obtain the number of identifiable feature points in the identifiable feature point set and the number of reference feature points in the reference feature point set; When the number of reference feature points is greater than or equal to the number of identifiable feature points, the set of unidentified feature points is predicted based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points. If the number of reference feature points is less than the number of identifiable feature points, the set of identifiable feature points is used as the feature information of the occupant.

3. The method according to claim 2, characterized in that, When the number of reference feature points is less than the number of identifiable feature points, the method further includes: Obtain the relative positional relationships between identifiable feature points in the set of identifiable feature points; The reference feature point set is updated based on the relative positional relationship.

4. The method according to claim 1, characterized in that, The method further includes: Determine whether the occupant is a new occupant based on the set of reference feature points; If the passenger is a new passenger, user registration is performed on the passenger.

5. The method according to claim 1, characterized in that, The identifiable feature points and the unidentifiable feature points respectively include one or more of the following: facial feature points, shoulder feature points, elbow feature points, wrist feature points, hip feature points, knee feature points, and ankle feature points of the occupant.

6. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the occupant feature information extraction method according to any one of claims 1-5.

7. A controller, characterized in that, include: The method includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the occupant feature information extraction method according to any one of claims 1-5.

8. An occupant feature information extraction device, suitable for control using the occupant feature information extraction method according to any one of claims 1-5, characterized in that, The device includes: The first acquisition module is used to acquire the current video frame of the occupant; The second acquisition module is used to acquire a set of reference feature points of the occupant; wherein, the set of reference feature points includes the relative positional relationships between the feature points of the occupant; The feature extraction module is used to extract feature points from the current video frame to obtain a set of identifiable feature points of the occupant; The feature recognition module is used to predict the set of unrecognized feature points based on the relative positional relationship between feature points in the set of identifiable feature points and the set of reference feature points, and to obtain the feature information of the occupant based on the set of identifiable feature points and the set of unrecognized feature points.

9. A vehicle, characterized in that, Includes the controller according to claim 7, or the occupant feature information extraction device according to claim 8.

Images