Passenger position determination method and device, cabin equipment and vehicle

Abstract

The invention is suitable for the technical field of automobiles, and provides a passenger position determination method and device, cabin equipment and a vehicle, and the method comprises the steps: obtaining a to-be-detected image containing a passenger in the vehicle; inputting the to-be-detected image into the target detection model for processing to obtain a detection result corresponding to the to-be-detected image; the detection result comprises a detection frame of the passenger in the to-be-detected image; and determining a first position of the passenger based on the detection frame and the area range corresponding to each seat in the vehicle. Compared with the prior art, the passenger position can be positioned without depending on each seat pressure sensor, so that the cost is reduced; meanwhile, by combining the detection frame detected by the target detection model for the image in the vehicle and the area range corresponding to each seat in the vehicle, accurate positioning of the position of the passenger can be realized, and the positioning accuracy of the position of the passenger is improved.

Description

Technical Field

[0001] The present application belongs to the field of automobile technology, and in particular relates to a method and device for determining the position of an occupant, a cockpit device and a vehicle. Background Art

[0002] At present, with the development of cockpit equipment, the system is no longer limited to the observation and analysis of the driver's status, but has also begun to gradually expand to the entire cabin occupants, providing more customized experiences for occupants in different positions of the vehicle, such as occupant positioning, rear occupant monitoring, etc.

[0003] However, existing methods for determining the position of passengers in a vehicle usually rely on seat pressure sensors to locate the position of passengers, which is not comprehensive enough and not only increases costs, but also fails to accurately locate the position of passengers, thereby reducing the accuracy of locating the position of passengers. Summary of the invention

[0004] The embodiments of the present application provide a method, device, cabin equipment and vehicle for determining the position of an occupant to address the problem that the prior art is not comprehensive enough, which not only increases the cost but also fails to accurately locate the position of the occupant, thereby reducing the accuracy of locating the position of the occupant.

[0005] In a first aspect, an embodiment of the present application provides a method for determining an occupant position, comprising:

[0006] Acquire an image to be detected containing passengers in the vehicle;

[0007] Inputting the image to be detected into the target detection model for processing to obtain a detection result corresponding to the image to be detected; the detection result includes a detection frame of the occupant in the image to be detected;

[0008] Based on the detection frame and the area range corresponding to each seat in the vehicle, the first position of the occupant is determined.

[0009] Optionally, determining the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle includes:

[0010] Determine a position range of a set part of the occupant in the detection frame;

[0011] The position range is matched with each of the area ranges to obtain the first position.

[0012] Optionally, matching the position range with each of the area ranges to obtain the first position includes:

[0013] If the position range is within the target area range, the seat corresponding to the target area range is determined as the first position; the target area range is any one of the area ranges;

[0014] or,

[0015] If there is an overlapping portion between the position range and the target area range, and the overlapping portion is greater than a first threshold, the seat corresponding to the target area range is determined as the first position.

[0016] Optionally, the method further includes:

[0017] Acquiring voice information of the occupant in the vehicle;

[0018] Analyzing the voice information based on a sound source localization algorithm to obtain a second position of the occupant;

[0019] Accordingly, the method further includes:

[0020] Based on the first position and the second position, a target position of the occupant is determined.

[0021] Optionally, determining the target position of the occupant based on the first position and the second position includes:

[0022] Obtaining behavior information of the occupant;

[0023] Based on the behavior information, determining the scene mode in which the occupant is located;

[0024] Based on the scene mode, determining a first weight corresponding to the first position and a second weight corresponding to the second position;

[0025] The target position is determined based on the first weight and the second weight.

[0026] Optionally, determining the target position of the occupant based on the first position and the second position includes:

[0027] If the first position and the second position are different, the first position is determined as the target position.

[0028] Optionally, after determining the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle, the method further includes:

[0029] determining a seat area where the occupant is located according to the first position;

[0030] determining a target device corresponding to the seat area;

[0031] The remaining devices in the vehicle except the target device are turned off.

[0032] In a second aspect, an embodiment of the present application provides a device for determining an occupant position, comprising:

[0033] A first acquisition unit, used to acquire an image to be detected containing a passenger in the vehicle;

[0034] A processing unit, configured to input the image to be detected into a target detection model for processing, and obtain a detection result corresponding to the image to be detected; the detection result includes a detection frame of the occupant in the image to be detected;

[0035] The first position determining unit is used to determine the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle.

[0036] In a third aspect, an embodiment of the present application provides a cockpit device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a method for determining an occupant's position as described in any one of the first aspects above.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining the position of an occupant as described in any one of the first aspects above is implemented.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a cockpit device, enables the cockpit device to execute the occupant position determination method described in any one of the first aspects above.

[0039] In a sixth aspect, an embodiment of the present application provides a vehicle, comprising a cockpit device, wherein the cockpit device is used to execute the occupant position determination method as described in any one of the first aspects.

[0040] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0041] The embodiment of the present application provides a method for determining the position of an occupant, which includes obtaining an image to be detected containing an occupant in a vehicle; inputting the image to be detected into a target detection model for processing to obtain a detection result corresponding to the image to be detected; the detection result includes a detection frame of the occupant in the image to be detected; and determining the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle. The present application can determine the seat position of the occupant in the vehicle by shooting an image, without determining the seat position of the occupant in the vehicle through the seat pressure sensor of each seat, so there is no need to install a seat pressure sensor on each seat, thereby reducing costs. At the same time, since the image can clearly show the scene where the occupant is in the vehicle, and the detection frame output by the target detection model can accurately mark the position range of the occupant at this time on the image, therefore, by comparing the detection frame output by the target detection model with the area range corresponding to each seat in the vehicle, the occupant can be accurately located in which seat position range, so that the seat where the occupant is located can be accurately determined, thereby achieving accurate positioning of the occupant position, and improving the positioning accuracy of the occupant position. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 is a flowchart of an implementation method for determining an occupant position provided in an embodiment of the present application;

[0044] Figure 2 is a flowchart of an implementation method for determining an occupant position provided by another embodiment of the present application;

[0045] Figure 3 is a flowchart of an implementation method for determining an occupant position provided in yet another embodiment of the present application;

[0046] Figure 4 is a flowchart of an implementation method for determining an occupant position provided by another embodiment of the present application;

[0047] Figure 5 is a structural schematic diagram of an occupant position determination device provided in an embodiment of the present application;

[0048] Figure 6 It is a structural schematic diagram of a cockpit device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0049] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0050] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0051] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0052] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0053] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0054] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0055] In actual applications, with the development of cockpit equipment, the system is no longer limited to the observation and analysis of the driver's status, but has also gradually expanded to the entire cabin occupants, providing more customized experiences for occupants in different positions of the vehicle, such as occupant positioning, rear occupant monitoring, etc.

[0056] However, existing methods for determining the position of passengers in a vehicle usually rely on seat pressure sensors to locate the position of passengers, which is not comprehensive enough and not only increases costs, but also fails to accurately locate the position of passengers, thereby reducing the accuracy of locating the position of passengers.

[0057] Based on this, the present application proposes a method for determining the position of an occupant to achieve accurate determination of the position of an occupant in a vehicle.

[0058] See also Figure 1 , Figure 1 This is a flowchart of an implementation of a method for determining the position of an occupant provided in an embodiment of the present application. In the embodiment of the present application, the occupant position determination method is executed by a cockpit device.

[0059] In practical applications, cockpit equipment refers to electronic equipment that integrates intelligent and networked technologies, software and hardware, and can achieve intelligent perception and intelligent decision-making of the cockpit space through continuous learning and iteration. It is not only a human-computer interaction scene in the car cockpit, but also an important link and key node for the evolution of the human-vehicle relationship from a tool to a partner.

[0060] The main components of the cockpit equipment include the in-vehicle infotainment system, streaming media central rearview mirror, head-up display system HUD, full LCD instrument and car networking module, etc. These components together form a complete intelligent cockpit system, enabling the vehicle to interact intelligently with people, roads and the vehicle itself.

[0061] In the embodiment of the present application, the cockpit equipment may further include a vehicle-mounted camera. The vehicle-mounted camera is used to take pictures inside the vehicle.

[0062] In some possible embodiments, the vehicle-mounted cameras include but are not limited to occupancy monitoring system (OMS) cameras and driver monitoring system (DMS) cameras.

[0063] In actual applications, OMS monitors passengers and can monitor their age, status, emotions, etc.; DMS monitors drivers and can monitor their fatigue status, dangerous driving behavior, etc.

[0064] In other possible embodiments, in order to achieve accurate shooting of various positions in the vehicle, the above-mentioned DMS camera can be set at the A-pillar inside the vehicle; the above-mentioned OMS camera can be set at the front windshield inside the vehicle, and can also be set at the rear roof.

[0065] In practical applications, the A-pillar is a connecting pillar on the left and right front of the vehicle that connects the roof and the front cabin.

[0066] like Figure 1 As shown, the occupant position determination method provided in an embodiment of the present application may include S101 to S103, which are described in detail as follows:

[0067] In S101 , an image to be detected containing a passenger in a vehicle is acquired.

[0068] In one implementation of an embodiment of the present application, after the vehicle door is opened, the on-board camera in the vehicle can automatically enter a working state and take pictures of the interior of the vehicle in real time to obtain an image to be detected containing passengers in the vehicle, and send the image to be detected to the cockpit device.

[0069] In another implementation of the embodiment of the present application, in order to reduce energy consumption, the cockpit equipment can control the vehicle-mounted camera to enter a working state when detecting the presence of passengers in the vehicle, so as to obtain the image to be detected containing the passengers in the vehicle captured by the vehicle-mounted camera.

[0070] In the embodiment of the present application, the image to be detected may be one or more images.

[0071] In some possible embodiments, when there is only one vehicle-mounted camera and the vehicle-mounted camera is a camera for photographing a panoramic view inside the vehicle, the above-mentioned image to be detected may be one.

[0072] In some other possible embodiments, when there are two on-board cameras, and one camera is used to photograph the front area of ​​the vehicle, and one camera is used to photograph the rear area of ​​the vehicle, the above-mentioned images to be detected may be multiple.

[0073] It should be noted that the front area of ​​the vehicle may include the main driving area and the co-pilot area, and the rear area of ​​the vehicle refers to the area inside the vehicle except the above-mentioned front area of ​​the vehicle.

[0074] In S102, the image to be detected is input into a target detection model for processing to obtain a detection result corresponding to the image to be detected; the detection result includes a detection frame of the occupant in the image to be detected.

[0075] In the embodiment of the present application, after obtaining the image to be detected, the cockpit device can input the image to be detected into the target detection model for detection to obtain the detection result corresponding to the image to be detected, wherein the detection result includes the detection frame where the occupant is located in the image to be detected.

[0076] It should be noted that the target detection model can be obtained by training a pre-built deep learning model based on a preset sample set. Each sample data in the preset sample set includes a sample image and a sample result corresponding to the sample image. When training the pre-built deep learning model, the sample image in each sample data is used as the input of the deep learning model, and the sample result corresponding to the sample image in each sample data is used as the output of the deep learning model. Through training, the deep learning model can learn the correspondence between all possible sample images and sample results, and the trained deep learning model is used as the target detection model.

[0077] In some possible embodiments, the deep learning model may be a YOLO (You Only Look Once) network.

[0078] In S103, the first position of the occupant is determined based on the detection frame and the area range corresponding to each seat in the vehicle.

[0079] It should be noted that the cockpit device pre-stores the area range corresponding to each seat. Therefore, in the embodiment of the present application, after obtaining the detection frame where the occupant is located in the image to be detected, the cockpit device can match the detection frame with each area range to determine the first position of the occupant. The area range can be composed of a three-dimensional space formed by the seat cushion and the seat back.

[0080] The first position is used to describe the target seat where the occupant is located.

[0081] In actual applications, when a passenger sits on any seat, the passenger must be in the three-dimensional space formed by the seat cushion and the seat back. Therefore, in the embodiment of the present application, when the cockpit equipment detects that the detection frame is within a certain area range, it can determine that the detection frame matches the area range, thereby determining that the first position of the passenger is the seat corresponding to the area range.

[0082] In one embodiment of the present application, the area range corresponding to each seat may also be: when there are passengers sitting in each seat in the vehicle, the standard seating range of the passengers in each seat.

[0083] It should be noted that, since the sitting posture of the occupant is flexible and changeable, in this embodiment, after obtaining the detection frame, the cockpit device can obtain the overlapping part between the detection frame and each area range, and determine the maximum overlapping part. At this time, the cockpit device can preliminarily determine that the occupant may be in the area range corresponding to the maximum overlapping part. Afterwards, in order to clarify whether the occupant is indeed in the area range corresponding to the maximum overlapping part, the cockpit device can detect the proportion of the maximum overlapping part in the area range corresponding to it, and when it is detected that the proportion is greater than the set proportion, the cockpit device can determine that the occupant is indeed in the area range corresponding to the maximum overlapping part, so that the seat corresponding to the area range corresponding to the maximum overlapping part can be determined as the first position of the occupant. Among them, the set proportion is greater than or equal to one half.

[0084] In another embodiment of the present application, since the limbs or head of the occupant are not necessarily in the same area as the occupant's buttocks, in order to improve the accuracy of determining the first position of the occupant, the cockpit device can specifically be as follows: Figure 2 Steps S201 to S202 shown determine the first position of the occupant, as described in detail below:

[0085] In S201 , the position range of the set part of the occupant in the detection frame is determined.

[0086] In actual applications, the seat position where the occupant's buttocks are located is the occupant's position. Therefore, in this embodiment, the set position can be the buttocks.

[0087] In this embodiment, the cockpit device can input the image to be detected into the image detection model for processing to obtain a target frame corresponding to the set part of the occupant. The image detection model is used to detect the set part of the human body.

[0088] It should be noted that the image detection model can be obtained by training a pre-built neural network model based on a preset sample set. Each sample data in the preset sample set includes a sample image and a sample set part corresponding to the sample image. When training the pre-built neural network model, the sample image in each sample data is used as the input of the neural network model, and the sample set part corresponding to the sample image in each sample data is used as the output of the neural network model. Through training, the neural network model can learn the correspondence between all possible sample images and sample set parts, and the trained neural network model is used as the image detection model.

[0089] Afterwards, since it is the same image to be detected, the cockpit equipment can overlay the image to be detected containing the target frame onto the image to be detected containing the detection frame, thereby determining the position range of the target frame in the detection frame, and then obtaining the position range of the set part of the occupant in the detection frame.

[0090] In S202, the position range is matched with each of the area ranges to obtain the first position.

[0091] In this embodiment, after determining the position range of the set part of the occupant in the detection frame, the cockpit device may match the position range with each area range to determine the first position of the occupant.

[0092] In one embodiment of the present application, since each area range is composed of the three-dimensional space between the seat cushion and the seat back of each seat, and each seat can completely accommodate the body of an occupant, when the cockpit device detects that the above position range is within the target area range, that is, the above position range is completely within the target area range, it can be determined that the set part of the occupant is completely within the target area range, so the cockpit device can determine the seat corresponding to the target area range as the first position of the occupant. Among them, the target area range can be any one of the various area ranges.

[0093] In another embodiment of the present application, since in actual applications, there is no baffle between seats in the same row in the vehicle, that is, a small part of the occupant's set position may be on other seats. Therefore, in order to improve the accuracy of determining the first position of the occupant, the cockpit equipment can detect whether there is an overlapping part between the above-mentioned position range and the target area range, and when an overlapping part is detected, the overlapping part can be compared with a first threshold, wherein the first threshold is greater than or equal to one half of the target area range.

[0094] In this embodiment, when the cockpit equipment detects that the overlapping part of the above-mentioned position range and the target area range is greater than the first threshold, it means that most of the set part of the occupant is within the target area range. Therefore, the cockpit equipment can determine the seat corresponding to the target area range as the first position of the occupant.

[0095] As can be seen from the above, the embodiment of the present application provides a method for determining the position of an occupant, by acquiring an image to be detected containing an occupant in a vehicle; inputting the image to be detected into a target detection model for processing to obtain a detection result corresponding to the image to be detected; the detection result includes a detection frame of the occupant in the image to be detected; and based on the detection frame and the area range corresponding to each seat in the vehicle, the first position of the occupant is determined. The present application can determine the seat position of the occupant in the vehicle by shooting an image, without determining the seat position of the occupant in the vehicle through the seat pressure sensor of each seat, so there is no need to install a respective seat pressure sensor on each seat, thereby reducing the cost. At the same time, since the image can clearly show the scene where the occupant is located in the vehicle, and the detection frame output by the target detection model can accurately mark the position range of the occupant at this time on the image, therefore, by comparing the detection frame output by the target detection model with the area range corresponding to each seat in the vehicle, the occupant can be accurately located in which seat position range, so that the seat where the occupant is located can be accurately determined, thereby achieving accurate positioning of the occupant position, and improving the positioning accuracy of the occupant position.

[0096] In actual applications, since the occupants are not stationary in the vehicle, but may occasionally move or squat, if the terminal device takes a picture of the occupant when the occupant is moving or squatting, it is impossible to clearly determine in which seat range the occupant is when the occupant position is detected based on the image to be detected. Therefore, in order to improve the accuracy of determining the occupant position, the terminal device can also determine the final position of the occupant in combination with sound source positioning.

[0097] Based on this, see Figure 3 , Figure 3 FIG. 1 is a flowchart of an implementation method for determining the position of an occupant provided in another embodiment of the present application. Figure 1 Correspondingly, the occupant position determination method of this embodiment may further include S301 to S302. Accordingly, after step S103, step S303 may further be included, which is described in detail as follows:

[0098] In S301, voice information of the occupant in the vehicle is obtained.

[0099] In S302, the voice information is analyzed based on a sound source localization algorithm to obtain a second position of the occupant.

[0100] In S303 , a target position of the occupant is determined based on the first position and the second position.

[0101] In one implementation of this embodiment, the cockpit device can obtain the voice information of the occupant in the vehicle through a vehicle-mounted voice device wirelessly connected to the cockpit device, wherein the vehicle-mounted voice device can be a vehicle-mounted microphone array.

[0102] Afterwards, the cockpit equipment can analyze the voice information based on the sound source localization algorithm to obtain the second position of the occupant in the vehicle.

[0103] It should be noted that the sound source localization algorithm is to determine the position of the sound source in space by analyzing information such as the time difference, amplitude difference or frequency difference of the sound between different microphones.

[0104] In practical applications, sound source localization algorithms include but are not limited to: cross-correlation method, overtone delay measurement method, and sound intensity level difference method.

[0105] The cross-correlation method specifically refers to determining the distance difference of the sound source by calculating the cross-correlation function between different microphones. This method is based on the relationship that the time difference of sound waves propagating between different microphones is proportional to the distance between the sound source and the microphone1.

[0106] In this embodiment, the terminal device can receive the sound signals corresponding to the language information output by the occupant through microphones at multiple different positions of the vehicle, and then calculate the cross-correlation function between each sound signal. Afterwards, the terminal device can calculate the distance difference between the language information output by the occupant and each microphone through the time delay corresponding to the peak of each cross-correlation function. Finally, the terminal device can determine the sound source position through each distance difference, thereby obtaining the second position of the occupant.

[0107] It should be noted that the cross-correlation method has high accuracy and can use the correlation between signals to accurately find the time delay, which has a good guarantee for positioning accuracy, thereby improving the accuracy of determining the second position. At the same time, the cross-correlation method can adapt to various types of sound signals, including complex broadband signals and transient signals, and can achieve a high success rate in determining the second position.

[0108] The overtone delay measurement method specifically refers to the use of the overtone spectrum of the sound source and the time difference relationship between different microphones to determine the arrival time difference of the sound source through spectral analysis and time-frequency domain processing of the sound signal, and then determine the direction of the sound source1.

[0109] In this embodiment, the terminal device can receive sound signals corresponding to the language information output by the occupant through microphones at multiple different positions in the vehicle, and then use the time difference between the fundamental tone and overtones in the language information output by the occupant reaching the microphones at multiple different positions in the vehicle, as well as the speed of sound propagation, to calculate the distance between the output language information and the microphones at each different position. Afterwards, the terminal device can obtain the sound source position through each distance, and thus obtain the second position of the occupant.

[0110] It should be noted that, since the overtone structure is relatively stable, the terminal device can overcome the problem of mixing of direct sound and reflected sound through the overtone delay measurement method, thereby improving the accuracy of determining the second position.

[0111] The sound intensity level difference method specifically means that when the sound source is located on one side, a directional response will be generated. By calculating the sound pressure level difference of different microphones, the direction of the sound source can be determined1.

[0112] In this embodiment, the terminal device can receive the sound signal corresponding to the language information output by the occupant through the microphones at multiple different positions of the vehicle, and then measure the intensity of the language information output by the occupant at the microphones at multiple different positions. Afterwards, the terminal device can determine the sound source position according to the preset correspondence between each intensity and distance, thereby obtaining the second position of the occupant.

[0113] It should be noted that the sound intensity level difference method can quickly obtain the approximate direction of the sound source for the near-field and single sound source situation, and is more suitable for vehicles with a limited range, thereby improving the determination rate of the second position of the occupant.

[0114] In actual applications, since the occupants are not stationary in the vehicle, but may occasionally move or squat, if the terminal device takes a picture of the occupant at the moment of movement or squatting, it is impossible to clearly determine in which seat range the occupant is when the occupant position is detected based on the image to be detected. Therefore, in order to improve the accuracy of determining the occupant position, in this embodiment, the terminal device can perform auxiliary judgment on the second position determined by the image to be detected in combination with the first position obtained by the above-mentioned sound source positioning, that is, judge whether the second position is accurate in combination with the first position to obtain the final accurate position of the occupant.

[0115] Based on this, the cockpit equipment can combine the first position and the second position determined above to obtain the final target position of the occupant.

[0116] It should be noted that the first position and the second position are both used to describe the target seat where the occupant is located.

[0117] In this embodiment, the cockpit device can detect whether the first position and the second position are the same.

[0118] In one embodiment of the present application, when an occupant sits in a seat and leans forward or turns to speak in another seat, when the cockpit device locates the sound source through the voice information of the occupant acquired at this time, it is easy to determine the second position of the occupant as the other seat mentioned above. In other words, sound source positioning is not reliable, and the image to be detected can truly reflect the position of the occupant. Therefore, in this embodiment, when the cockpit device detects that the first position and the second position are different, it can directly determine the first position as the target position of the occupant.

[0119] In another embodiment of the present application, in different situations, sometimes visual positioning (i.e. positioning by image) is relatively more reliable, and sometimes sound source positioning is more reliable. Therefore, in order to further improve the accuracy of determining the target position of the occupant, the cockpit device can specifically determine the target position of the occupant according to the following steps, which are described in detail as follows:

[0120] Obtaining behavior information of the occupant;

[0121] Based on the behavior information, determining the scene mode in which the occupant is located;

[0122] Based on the scene mode, determining a first weight corresponding to the first position and a second weight corresponding to the second position;

[0123] The target position is determined based on the first weight and the second weight.

[0124] In this embodiment, the terminal device can input the behavior information into the trained mode determination model for processing to obtain the scene mode that the occupant is in. The mode determination model is trained by a pre-built first neural network model.

[0125] It should be noted that the pattern determination model can be obtained by training a pre-constructed first neural network model based on a preset sample set. Each sample data in the preset sample set includes sample behavior information and a sample scene pattern corresponding to the sample behavior information. When training the pre-constructed first neural network model, the sample behavior information in each sample data is used as the input of the first neural network model, and the sample scene pattern corresponding to the sample behavior information in each sample data is used as the output of the first neural network model. Through training, the first neural network model can learn the correspondence between all possible sample behavior information and sample scene patterns, and the trained first neural network model is used as the pattern determination model.

[0126] In this embodiment, the scene modes include but are not limited to: passenger probe mode and passenger squat mode. The passenger probe mode is used to describe that when a passenger is sitting in a seat, his head is within the area of ​​another seat. The passenger squat mode is used to describe that a passenger is not sitting in any seat, but is within the area of ​​the seat.

[0127] It should be noted that the sum of the first weight and the second weight is equal to 1, and the first weight and the second weight are not equal to one half.

[0128] In this embodiment, when the scene mode in which the occupant is located is the occupant probe mode, since the error probability of the second position obtained by analyzing the occupant's voice information through the sound source localization algorithm is relatively large at this time, the cockpit equipment can determine that the first weight corresponding to the first position is greater than one half, and the second weight corresponding to the second position is less than one half.

[0129] When the scene mode in which the occupant is located is the occupant squatting mode, since the error probability of the first position obtained through the image to be detected is relatively large at this time, the cockpit equipment can determine that the first weight corresponding to the first position is less than one half, and the second weight corresponding to the second position is greater than one half.

[0130] In some possible embodiments, the terminal device may obtain the first weight set and the second weight set corresponding to each of the scene modes in the historical time period. The first weight set includes the historical weights corresponding to the first position of the scene mode in the historical time period. The second weight set includes the historical weights corresponding to the second position of the scene mode in the historical time period.

[0131] The historical time period can be determined according to actual needs and is not limited here.

[0132] In this embodiment, the terminal device can count the number of occurrences of each historical weight corresponding to the first position in the first weight set of the scene mode, and determine the historical weight with the largest number of occurrences as the first weight of the first position in the scene mode.

[0133] The terminal device may count the number of occurrences of each historical weight corresponding to the second position in the second weight set of the scene mode, and determine the historical weight with the largest number of occurrences as the second weight of the second position in the scene mode.

[0134] Afterwards, the cockpit equipment may determine the position corresponding to the larger value of the first weight and the second weight as the target position of the occupant.

[0135] Specifically, when the cockpit device detects that the first weight is greater than the second weight, the cockpit device may determine the first position as the target position of the occupant; when the cockpit device detects that the second weight is greater than the first weight, the cockpit device may determine the second position as the target position of the occupant.

[0136] It can be seen from the above that the method for determining the position of the occupant provided in this embodiment can obtain the voice information of the occupant in the vehicle; analyze the voice information based on the sound source localization algorithm to obtain the second position of the occupant; and determine the target position of the occupant based on the first position and the second position. Since the occupant is not stationary in the vehicle, but may occasionally move or squat, if the terminal device takes a picture of the occupant at the moment of the above movement or squatting, it is impossible to clearly determine which seat range the occupant is in when the occupant position is detected based on the image to be detected. Therefore, this embodiment can determine whether the first position is accurate in combination with the second position determined by the sound source localization, thereby further improving the accuracy of determining the final position of the occupant.

[0137] See also Figure 4 , Figure 4 FIG. 1 is a flowchart of a method for determining the position of an occupant provided in another embodiment of the present application. Figure 1 Corresponding embodiment, this embodiment may further include S401 to S403 after S103, which are described in detail as follows:

[0138] In S401 , a seat area where the occupant is located is determined according to the first position.

[0139] In S402, a target device corresponding to the seat area is determined.

[0140] In S403, the remaining devices in the vehicle except the target device are turned off.

[0141] In this embodiment, since the first position is used to describe the target seat where the occupant is located, the cockpit equipment can determine the seat area where the occupant is located according to the first position.

[0142] In some possible embodiments, the terminal device may input the first position into a trained seat area determination model for processing, thereby determining the seat area where the occupant is located. The seat area determination model is trained by a pre-built second neural network model.

[0143] It should be noted that the seat area determination model can be obtained by training a pre-constructed second neural network model based on a preset sample set. Each sample data in the preset sample set includes a sample position and a sample area corresponding to the sample position. When training the pre-constructed second neural network model, the sample position in each sample data is used as the input of the second neural network model, and the sample area corresponding to the sample position in each sample data is used as the output of the second neural network model. Through training, the second neural network model can learn the correspondence between all possible sample positions and sample areas, and the trained second neural network model is used as the seat area determination model.

[0144] It should be noted that the seat area includes but is not limited to: the main driver area, the co-driver area and the rear seat area.

[0145] Among them, when the occupant is in the main driving seat, the cockpit equipment can determine that the seat area where the occupant is located is the main driving area; when the occupant is in the front passenger seat, the cockpit equipment can determine that the seat area where the occupant is located is the front passenger area; when the occupant is in any seat in the back row (such as the left rear seat and the right rear seat), the cockpit equipment can determine that the seat area where the occupant is located is the back seat area.

[0146] In this embodiment, in order to prevent the relevant equipment of the corresponding seat area from being turned on when there is no occupant in a seat, the cockpit equipment can determine the target equipment corresponding to the seat area according to the seat area where the occupant is located.

[0147] It should be noted that the target device specifically refers to a device used to provide services to the occupants in the above-mentioned seat area, such as a rear ceiling screen used to serve the occupants in the rear seat area, a co-pilot screen used to serve the occupants in the co-pilot area, etc.

[0148] The target device may be one or more than one target device.

[0149] Afterwards, the cockpit device can control the above target device to be in the turned-on state and turn off the remaining devices in the vehicle except the target device to reduce power consumption.

[0150] As can be seen from the above, the occupant position determination method provided in this embodiment can shut down the remaining devices except the target device corresponding to the seat where the occupant is located, thereby reducing some unnecessary vehicle service operations, saving service computing power in the background, and reducing power consumption. At the same time, it also improves the intelligence of the cockpit equipment.

[0151] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0152] Corresponding to the method for determining the position of an occupant described in the above embodiment, Figure 5 The structure diagram of a passenger position determination device provided by an embodiment of the present application is shown. For the convenience of explanation, only the part related to the embodiment of the present application is shown. Figure 5 The occupant position determination device 500 includes: a first acquisition unit 51, a processing unit 52 and a first position determination unit 53. Among them:

[0153] The first acquisition unit 51 is used to acquire an image to be detected containing passengers in the vehicle.

[0154] The processing unit 52 is used to input the image to be detected into the target detection model for processing to obtain a detection result corresponding to the image to be detected; the detection result includes a detection box of the occupant in the image to be detected.

[0155] The first position determining unit 53 is used to determine the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle.

[0156] In one embodiment of the present application, the first position determination unit 53 specifically includes: a range determination unit and a matching unit. Among them:

[0157] The range determination unit is used to determine the position range of the set part of the occupant in the detection frame.

[0158] The matching unit is used to match the position range with each of the area ranges to obtain the first position.

[0159] In one embodiment of the present application, the matching unit is used to specifically include: a second position determination unit.

[0160] The second position determination unit is used to determine the seat corresponding to the target area range as the first position if the position range is within the target area range; the target area range is any one of the area ranges.

[0161] In one embodiment of the present application, the matching unit is used to specifically include: a third position determination unit.

[0162] The third position determining unit is used to determine the seat corresponding to the target area range as the first position if there is an overlapping portion between the position range and the target area range, and the overlapping portion is greater than a first threshold.

[0163] In one embodiment of the present application, the occupant position determination device 500 further includes: a second acquisition unit, an analysis unit and a fourth position determination unit.

[0164] The second acquiring unit is used to acquire voice information of the occupant in the vehicle.

[0165] The analyzing unit is used to analyze the voice information based on a sound source localization algorithm to obtain the second position of the occupant.

[0166] The fourth position determination unit is used to determine a target position of the occupant based on the first position and the second position.

[0167] In one embodiment of the present application, the fourth position determination unit specifically includes: a third acquisition unit, a mode determination unit, a weight determination unit and a fifth position determination unit. Wherein:

[0168] The third acquisition unit is used to acquire the behavior information of the occupant.

[0169] The mode determination unit is used to determine the scene mode of the occupant based on the behavior information.

[0170] The weight determination unit is used to determine a first weight corresponding to the first position and a second weight corresponding to the second position based on the scene mode.

[0171] The fifth position determination unit is configured to determine the target position based on the first weight and the second weight.

[0172] In one embodiment of the present application, the fourth position determining unit specifically includes: a sixth position determining unit.

[0173] The sixth position determining unit is configured to determine the first position as the target position if the first position is different from the second position.

[0174] In one embodiment of the present application, the occupant position determination device 500 further includes: an area determination unit, a device determination unit, and a closing unit.

[0175] The area determination unit is used to determine the seat area where the occupant is located according to the first position.

[0176] The device determining unit is used to determine a target device corresponding to the seat area.

[0177] The shut down unit is used to shut down the remaining devices in the vehicle except the target device.

[0178] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0179] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0180] Figure 6 This is a schematic diagram of the structure of the cockpit equipment provided by one embodiment of the present application. Figure 6 As shown, the cockpit device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown), a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 implements the steps of any of the above-mentioned embodiments of the method for determining the position of the occupant when executing the computer program 62.

[0181] The cockpit device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the cockpit device 6 and does not constitute a limitation of the cockpit device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0182] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0183] In some embodiments, the memory 61 may be an internal storage unit of the cockpit device 6, such as the memory of the cockpit device 6. In other embodiments, the memory 61 may also be an external storage device of the cockpit device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the cockpit device 6. Further, the memory 61 may also include both the internal storage unit and the external storage device of the cockpit device 6. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0184] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0185] An embodiment of the present application provides a computer program product. When the computer program product runs on a cockpit device, the cockpit device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0186] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the cockpit equipment, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0187] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0188] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for determining an occupant position, characterized in that: include: Acquire an image to be detected containing passengers in the vehicle; Inputting the image to be detected into the target detection model for processing to obtain a detection result corresponding to the image to be detected; the detection result includes a detection frame of the occupant in the image to be detected; Based on the detection frame and the area range corresponding to each seat in the vehicle, the first position of the occupant is determined.

2. The method for determining the position of an occupant according to claim 1, wherein: The determining the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle includes: Determine a position range of a set part of the occupant in the detection frame; The position range is matched with each of the area ranges to obtain the first position.

3. The method for determining the position of an occupant according to claim 2, wherein: The matching process of the position range with each of the area ranges to obtain the first position includes: If the position range is within the target area range, the seat corresponding to the target area range is determined as the first position; the target area range is any one of the area ranges; or, If there is an overlapping portion between the position range and the target area range, and the overlapping portion is greater than a first threshold, the seat corresponding to the target area range is determined as the first position.

4. The method for determining the position of an occupant according to claim 1, wherein: The method further comprises: Acquiring voice information of the occupant in the vehicle; Analyzing the voice information based on a sound source localization algorithm to obtain a second position of the occupant; Accordingly, the method further includes: Based on the first position and the second position, a target position of the occupant is determined.

5. The method for determining the position of an occupant according to claim 4, wherein: The determining the target position of the occupant based on the first position and the second position includes: Obtaining behavior information of the occupant; Based on the behavior information, determining the scene mode in which the occupant is located; Based on the scene mode, determining a first weight corresponding to the first position and a second weight corresponding to the second position; The target position is determined based on the first weight and the second weight.

6. The method for determining the position of an occupant according to claim 4, wherein: The determining the target position of the occupant based on the first position and the second position includes: If the first position and the second position are different, the first position is determined as the target position.

7. The method for determining the position of an occupant according to any one of claims 1 to 6, characterized in that: After determining the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle, the method further includes: determining a seat area where the occupant is located according to the first position; determining a target device corresponding to the seat area; The remaining devices in the vehicle except the target device are turned off.

8. An occupant position determination device, characterized in that: include: A first acquisition unit, used to acquire an image to be detected containing a passenger in the vehicle; A processing unit, configured to input the image to be detected into a target detection model for processing, and obtain a detection result corresponding to the image to be detected; the detection result includes a detection frame of the occupant in the image to be detected; The first position determining unit is used to determine the first position of the occupant based on the detection frame and the area range corresponding to each seat in the vehicle.

9. A cockpit device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for determining the occupant position according to any one of claims 1 to 7 is implemented.

10. A vehicle, characterized in that: Comprising the cockpit equipment as claimed in claim 9.

Images