Method for determining passenger seating in a vehicle, vehicle control method, device, computer device, computer-readable storage medium, and computer program

Abstract

The present disclosure provides a method for determining occupant seats in a vehicle, a vehicle control method and an apparatus, the method for determining occupant seats in a vehicle includes acquiring a vehicle image within the vehicle, determining a plurality of first seat areas within the vehicle image, detecting occupants in the vehicle based on the vehicle image and determining a detection frame for each of the occupants in the vehicle image, for each occupant detection frame, determining a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas, determining a plurality of second seat areas within the vehicle image, and when a target occupant detection frame of a corresponding target first seat area is not determined based on the first matching degree, determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the second seat areas.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) Embodiments of the present disclosure are based on and claim priority from a Chinese patent application having application number 202111682157.3, filing date December 31, 2021, and title "Method for determining passenger seating in a vehicle and vehicle control method and device," the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates to the field of intelligent driving technology, and relates to a method for determining a passenger's seat in a vehicle, and a vehicle control method, device, electronic device, medium, and program product. [Background technology]

[0003] With the rapid development of the automobile industry, in addition to the repeated updates of engines and other key components required for automobiles, many emerging technologies related to the automobile industry have also emerged, including vehicle intelligent driving technology. In vehicle intelligent driving technology, in order to achieve the effect of having a vehicle provide corresponding feedback based on human behavior, it is necessary to bind a person to a vehicle seat in advance, and then have the intelligent system in the vehicle make corresponding decisions based on different human behaviors.

[0004] In the related art, when a person is bound to a vehicle seat, there are often situations where the binding between the person and the vehicle seat is incorrect due to objective factors such as the sitting posture or height, so how to correctly bind a person to a vehicle seat has become an issue that needs to be solved urgently. Summary of the Invention [Means for solving the problem]

[0005] SUMMARY OF THE INVENTION Embodiments of the present disclosure provide a method for determining at least the seating of an occupant in a vehicle, and a vehicle control method, apparatus, electronic device, medium, and program product.

[0006] In one aspect, an embodiment of the present disclosure provides a method for determining a seat of an occupant in a vehicle, the method comprising: acquiring vehicle images within the vehicle; determining a plurality of first seating areas within the vehicle image, each of the first seating areas corresponding to a seat; detecting occupants in the vehicle based on the vehicle image and determining a detection frame for each of the occupants in the vehicle image; determining, for each of the occupant detection frames, a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas; determining a plurality of second seating areas within the vehicle image, each second seating area corresponding to a row or column of seats; determining a target second seat area corresponding to the target occupant detection frame based on a second matching degree between the target occupant detection frame and each of the second seat areas, when a target occupant detection frame of the corresponding target first seat area is not determined based on the first matching degree; The method includes determining the seat corresponding to the detection frame of the target occupant based on the position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant.

[0007] In the solution provided by the embodiments of the present disclosure, when the detection frame of the target occupant of the seat cannot be determined by the first seat area, the area range of the first seat area is expanded, and the seat corresponding to the detection frame of the target occupant is further determined based on the second seat area of ​​the expanded area range, thereby reducing the probability of failure in matching or binding between the occupant and the seat and improving the success rate of matching the occupant to the seat.

[0008] In one possible embodiment, determining a plurality of first seating areas within the vehicle image comprises: The method includes determining a first seat area corresponding to each seat in the vehicle image based on seat distribution setting information of the vehicle.

[0009] Since the seat distribution setting information of different vehicle types may be different, the first seat area can be determined more accurately according to the seat distribution setting information of the vehicle.

[0010] In one possible embodiment, for each of the occupant detection frames, determining a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas includes: traversing each of the first seating areas in a predetermined order; and for each of the traversed first seating areas, Calculating an overlap degree between the detection frame and the first seat area as a first matching degree between the detection frame and the first seat area; When the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to another detection frame, binding the seat corresponding to the first seat area to the detection frame and determining that the seat corresponding to the detection frame is the seat corresponding to the first seat area.

[0011] When the first matching degree reaches a first threshold, by detecting whether other detection frames are bound to the first seat area, it is possible to avoid binding multiple detection frames to one seat area, and further to avoid affecting the running of the vehicle when performing vehicle control based on the binding result.

[0012] In one possible embodiment, the detection frame of each of the occupants in the vehicle image is at least one of a face detection frame and a human body detection frame.

[0013] In one possible embodiment, determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the second seat areas includes: traversing the second seating area in a predetermined order for each target occupant detection frame; and Calculating an overlap degree between the detection frame of the target occupant and the second seat area as the second matching degree; If the second matching degree reaches a second threshold, determining that the second seat area is a target second seat area corresponding to a detection frame of a target occupant.

[0014] In this way, it is possible to first detect which row or column the target occupant is in, and then perform further seat binding, thereby improving the accuracy of the binding result.

[0015] In one possible embodiment, determining a seat corresponding to the detection frame of the target occupant based on a position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant includes: determining, based on the detection frame of the target occupant and a position range of each seat within the target second seat area, that a seat corresponding to the position range in which a center point of the detection frame of the target occupant is located, as a candidate seat corresponding to the detection frame of the target occupant; If the candidate seat is not bound to another detection frame, determining that the candidate seat is a seat corresponding to the detection frame of the target occupant, and binding the seat corresponding to the candidate seat to the detection frame of the target occupant.

[0016] In one possible embodiment, determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the second seat areas includes: For each traversed second seat area, if the degree of overlap does not reach a second threshold, the method further includes determining that the second seat area having the largest overlap area with the detection frame of the target occupant is the target second seat area.

[0017] In one possible embodiment, the method comprises: If a target second seat area corresponding to the detection frame of the target occupant is not determined, the method further includes determining a seat whose corresponding first seat area is closest to the detection frame of the target occupant and is not bound to other detection frames as the seat corresponding to the detection frame of the target occupant.

[0018] In another aspect, an embodiment of the present disclosure further provides a vehicle control method, the vehicle control method comprising: determining a user detection window corresponding to each of the seating areas based on the method for determining occupant seats in a vehicle according to the first aspect or any possible embodiment of the first aspect; Detecting a target action performed by a target user within the target detection frame, and controlling the vehicle based on the target action and a seat area corresponding to the target detection frame.

[0019] Based on the method for determining the seat of a passenger in a vehicle described in the above embodiments, when used for vehicle control, precise feedback can be provided based on the user's actions and their seats in response to an action command issued by the user, thereby improving the user's interaction sensation when using the vehicle.

[0020] In another aspect, embodiments of the present disclosure further provide an apparatus for determining a seat of an occupant in a vehicle, the apparatus comprising: an acquisition unit configured to acquire vehicle images inside the vehicle; a first determination unit configured to determine a plurality of first seating areas in the vehicle image, each first seating area corresponding to one seat; a second determination unit configured to detect occupants in the vehicle based on the vehicle image and determine a detection frame for each of the occupants in the vehicle image; a third determination unit configured to determine, for each of the occupant detection frames, a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas; a fourth determination unit configured to determine a plurality of second seating areas within the vehicle image, each second seating area corresponding to a row or column of seats; a fifth determination unit configured to determine a target second seat area corresponding to the target occupant detection frame based on a second matching degree between the target occupant detection frame and each of the second seat areas when a target occupant detection frame of the corresponding target first seat area is not determined based on the first matching degree; and a sixth determination unit configured to determine the seat corresponding to the detection frame of the target occupant based on the position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant.

[0021] In one possible embodiment, the first determination unit is further configured to determine a first seat area corresponding to each seat in the vehicle image based on seat distribution setting information of the vehicle.

[0022] In one possible embodiment, the third determination unit is further configured to traverse each of the first seat areas according to a preset order, and for each traversed first seat area, calculate the overlap degree between the detection frame and the first seat area as a first matching degree between the detection frame and the first seat area, and when the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame and determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area.

[0023] In one possible embodiment, the detection frame of each of the occupants in the vehicle image is at least one of a face detection frame and a human body detection frame.

[0024] In one possible embodiment, the fifth determination unit is further configured to traverse the second seat area for each target occupant detection frame according to a predetermined order, calculate the degree of overlap between the target occupant detection frame and the second seat area for each traversed second seat area as the second matching degree, and determine that the second seat area is a target second seat area corresponding to the target occupant detection frame if the second matching degree reaches a second threshold.

[0025] In one possible embodiment, the sixth determination unit is further configured to determine, based on the detection frame of the target occupant and the position range of each seat within the target second seat area, that the seat corresponding to the position range in which the center point of the detection frame of the target occupant is located is a candidate seat corresponding to the detection frame of the target occupant, and if the candidate seat is not bound to another detection frame, determine that the candidate seat is the seat corresponding to the detection frame of the target occupant and bind the seat corresponding to the candidate seat to the detection frame of the target occupant.

[0026] In one possible embodiment, the fifth determination unit is further configured to determine, for each traversed second seat area, if the degree of overlap does not reach a second threshold, that second seat area having the largest overlap area with the detection frame of the target occupant is the target second seat area.

[0027] In one possible embodiment, the seventh determination unit is further configured to determine, when a target second seat area corresponding to the detection frame of the target occupant is not determined, a seat whose corresponding first seat area is closest to the detection frame of the target occupant and is not bound to other detection frames, as the seat corresponding to the detection frame of the target occupant.

[0028] In another aspect, an embodiment of the present disclosure further provides a vehicle control device, the device comprising: an eighth determination unit configured to determine a user detection window corresponding to each of the seat areas based on the method for determining a seat of an occupant in a vehicle according to the first aspect or any possible embodiment of the first aspect; The vehicle control system includes a detection unit configured to detect a target action performed by a target user within the target detection frame and to control the vehicle based on the target action and a seat area corresponding to the target detection frame.

[0029] In another aspect, an embodiment of the present disclosure further provides a computer device, the computer device comprising a processor, a memory, and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the computer device is operating, and the machine-readable instructions, when executed by the processor, perform the steps in any of the above possible embodiments.

[0030] In another aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored therein, the computer program performing the steps of any of the above possible embodiments when executed by a processor.

[0031] In another aspect, an embodiment of the present disclosure further provides a computer program product, the computer program product including computer readable code, which, when executed in an electronic device, causes a processor in the electronic device to perform the steps of any of the possible embodiments described above.

[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of illustrative embodiments taken in conjunction with the drawings. The present invention provides, for example, the following items. (Item 1) 1. A method for determining an occupant's seat in a vehicle performed by an electronic device, comprising: acquiring vehicle images within the vehicle; determining a plurality of first seating areas within the vehicle image, each of the first seating areas corresponding to a seat; detecting occupants in the vehicle based on the vehicle image and determining a detection frame for each of the occupants in the vehicle image; determining, for each of the occupant detection frames, a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas; determining a plurality of second seating areas within the vehicle image, each second seating area corresponding to a row or column of seats; determining a target second seat area corresponding to the target occupant detection frame based on a second matching degree between the target occupant detection frame and each of the second seat areas, when a target occupant detection frame of the corresponding target first seat area is not determined based on the first matching degree; A method for determining a seat of an occupant in a vehicle, the method comprising: determining a seat corresponding to the detection frame of the target occupant based on the position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant. (Item 2) Determining a plurality of first seating areas within the vehicle image includes: determining a first seat area corresponding to each seat in the vehicle image based on seat distribution setting information of the vehicle; Item 1. A method for determining the seating of an occupant in a vehicle. (Item 3) Determining a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas includes: traversing each of the first seating areas according to a predetermined order; and for each traversed first seating area, setting an overlap degree between the detection frame and the first seat area as a first matching degree between the detection frame and the first seat area; When the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to another detection frame, binding the seat corresponding to the first seat area to the detection frame, and determining that the seat corresponding to the detection frame is the seat corresponding to the first seat area. 3. A method for determining the seat of an occupant in a vehicle according to item 1 or 2. (Item 4) The detection frame for each of the occupants in the vehicle image is at least one of a face detection frame and a human body detection frame. A method for determining a seat for a passenger in a vehicle according to any one of items 1 to 3. (Item 5) determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the second seat areas; traversing the second seating areas according to a predetermined sequence; and for each of the second seating areas traversed, setting an overlapping degree between the detection frame of the target occupant and the second seat area as a second matching degree between the detection frame of the target occupant and the second seat area; If the second matching degree reaches a second threshold, determining that the second seat area is a target second seat area corresponding to a detection frame of the target occupant. A method for determining a seat of a passenger in a vehicle according to any one of items 1 to 4. (Item 6) determining a seat corresponding to the detection frame of the target occupant based on a position range of each seat in one row or one column of seats corresponding to the target second seat area and the detection frame of the target occupant, determining, based on the detection frame of the target occupant and a position range of each seat within the target second seat area, that a seat corresponding to the position range in which a center point of the detection frame of the target occupant is located, as a candidate seat corresponding to the detection frame of the target occupant; If the candidate seat is not bound to another detection frame, determining that the candidate seat is a seat corresponding to the detection frame of the target occupant, and binding the seat corresponding to the candidate seat to the detection frame of the target occupant. 6. A method for determining a seat of a passenger in a vehicle according to any one of items 1 to 5. (Item 7) determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the second seat areas; and determining, for each of the traversed second seating areas, if the degree of overlap does not reach the second threshold, that the second seating area having the largest overlap area with the detection frame of the target occupant is the target second seating area. Item 5. A method for determining the seating of an occupant in a vehicle. (Item 8) When a target second seat area corresponding to the detection frame of the target occupant is not determined, determining a seat whose corresponding first seat area is closest to the detection frame of the target occupant and is not bound to other detection frames as the seat corresponding to the detection frame of the target occupant. A method for determining a seat of an occupant in a vehicle according to any one of items 1 to 7. (Item 9) A vehicle control method executed by an electronic device, comprising: determining a detection frame corresponding to each of the seat areas based on the method for determining seats of occupants in a vehicle according to any one of items 1 to 8; Detecting a target action performed by a target occupant within a target detection frame, and controlling the vehicle based on the target action and a seat area corresponding to the target detection frame. (Item 10) 1. An apparatus for determining a seat of an occupant in a vehicle, comprising: an acquisition unit configured to acquire vehicle images inside the vehicle; a first determination unit configured to determine a plurality of first seating areas in the vehicle image, each first seating area corresponding to one seat; a second determination unit configured to detect occupants in the vehicle based on the vehicle image and determine a detection frame for each of the occupants in the vehicle image; a third determination unit configured to determine, for each of the occupant detection frames, a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas; a fourth determination unit configured to determine a plurality of second seating areas in the vehicle image, each second seating area corresponding to a row or column of seats; a fifth determination unit configured to determine a target second seat area corresponding to the target occupant detection frame based on a second matching degree between the target occupant detection frame and each of the second seat areas when a target occupant detection frame of the corresponding target first seat area is not determined based on the first matching degree; and a sixth determination unit configured to determine the seat corresponding to the detection frame of the target occupant based on the position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant. (Item 11) The first determination unit is further configured to determine a first seat area corresponding to each seat in the vehicle image based on seat distribution setting information of the vehicle. Item 11. A device for determining the seating of an occupant in a vehicle according to item 10. (Item 12) the third determination unit is further configured to traverse each of the first seat areas according to a preset order, calculate, for each traversed first seat area, an overlap degree between the detection frame and the first seat area as a first matching degree between the detection frame and the first seat area, and, when the first matching degree reaches a first threshold and a seat corresponding to the first seat area is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame, and determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area; 12. A device for determining the seat of an occupant in a vehicle according to item 10 or 11. (Item 13) The detection frame for each of the occupants in the vehicle image is at least one of a face detection frame and a human body detection frame. 13. A device for determining the seat of a passenger in a vehicle according to any one of items 10 to 12. (Item 14) the fifth determination unit is further configured to traverse the second seat area according to a predetermined order for each target occupant detection frame, calculate an overlap degree between the target occupant detection frame and the second seat area for each traversed second seat area as the second matching degree, and determine that the second seat area is a target second seat area corresponding to the target occupant detection frame if the second matching degree reaches a second threshold. 14. A device for determining the seat of a passenger in a vehicle according to any one of items 10 to 13. (Item 15) the sixth determination unit is further configured to determine, based on the detection frame of the target occupant and a position range of each seat in the target second seat area, a seat corresponding to the position range in which a center point of the detection frame of the target occupant is located, as a candidate seat corresponding to the detection frame of the target occupant; and, if the candidate seat is not bound to another detection frame, determine the candidate seat as the seat corresponding to the detection frame of the target occupant, and bind the seat corresponding to the candidate seat to the detection frame of the target occupant. 15. A device for determining the seat of a passenger in a vehicle according to any one of items 10 to 14. (Item 16) the fifth determination unit is further configured to, for each traversed second seat area, determine, when the degree of overlap does not reach a second threshold, that the second seat area having the largest overlap area with the detection frame of the target occupant is the target second seat area; Item 15. A device for determining the seating of an occupant in a vehicle according to item 14. (Item 17) the seventh determination unit is further configured, when a target second seat area corresponding to the detection frame of the target occupant is not determined, to determine a seat whose corresponding first seat area is closest to the detection frame of the target occupant and is not bound to other detection frames as the seat corresponding to the detection frame of the target occupant. 17. A device for determining the seat of a passenger in a vehicle according to any one of items 10 to 16. (Item 18) A vehicle control device, an eighth determination unit configured to determine a detection frame corresponding to each of the seat areas based on the method for determining seats of occupants in a vehicle according to any one of items 1 to 8; A vehicle control device comprising: a detection unit configured to detect a target action performed by a target occupant within a target detection frame, and to perform vehicle control based on the target action and a seat area corresponding to the target detection frame. (Item 19) A computer device comprising a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is operating, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform steps of the method for determining a seat of an occupant in a vehicle according to any one of items 1 to 8, or to perform steps of the vehicle control method according to item 9. (Item 20) A computer-readable storage medium having stored thereon a computer program that, when executed by a processor, causes the processor to perform steps of the method for determining a seat of an occupant in a vehicle according to any one of items 1 to 8, or to perform steps of the vehicle control method according to item 9. (Item 21) A computer program product including computer-readable code, which, when executed in an electronic device, causes a processor in the electronic device to perform steps of the method for determining a seat of an occupant in a vehicle according to any one of items 1 to 8, or to perform steps of the vehicle control method according to item 9. [Brief explanation of the drawings]

[0033] [Figure 1]1 illustrates a flowchart of a method for determining a seat of an occupant in a vehicle according to an embodiment of the present disclosure.

[0034] [Figure 2] 1 shows a schematic diagram of a division scheme for a first seating area of ​​a four-seater vehicle according to an embodiment of the present disclosure.

[0035] [Figure 3] 10 illustrates a flowchart for determining a seat corresponding to the detection window according to an embodiment of the present disclosure.

[0036] [Figure 4] 1 illustrates a schematic diagram of a calculation scheme for the degree of overlap between a detection window and a first seating area according to an embodiment of the present disclosure;

[0037] [Figure 5a] 1 shows a schematic diagram of a second seating area of ​​a four-seater vehicle divided by rows according to an embodiment of the present disclosure.

[0038] [Figure 5b] 1 shows a schematic diagram of a second seating area of ​​a four-seater vehicle divided by rows according to an embodiment of the present disclosure.

[0039] [Figure 6] 10 shows a flowchart for determining a target second seat area corresponding to the target occupant detection frame according to an embodiment of the present disclosure.

[0040] [Figure 7a] 10 shows a schematic diagram of determining a target second seat area corresponding to the target occupant detection frame according to an embodiment of the present disclosure.

[0041] [Figure 7b] 10 illustrates another schematic diagram of determining a target second seating area corresponding to the target occupant detection frame according to an embodiment of the present disclosure.

[0042] [Figure 8]10 illustrates a flowchart for determining a seat corresponding to the detection frame of the target occupant based on a target second seat area according to an embodiment of the present disclosure.

[0043] [Figure 9] 10 illustrates a schematic diagram of determining a seat corresponding to the detection frame of the target occupant based on a target second seat area according to an embodiment of the present disclosure.

[0044] [Figure 10] 10 shows a schematic diagram illustrating, in accordance with an embodiment of the present disclosure, when a target second seat area corresponding to the detection frame of the target occupant is not determined, a seat whose corresponding first seat area is closest to the detection frame of the target occupant and is not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.

[0045] [Figure 11] 1 shows a flowchart of a vehicle control method according to an embodiment of the present disclosure.

[0046] [Figure 12] 1 shows an architecture schematic diagram of an apparatus for determining a passenger's seat in a vehicle according to an embodiment of the present disclosure;

[0047] [Figure 13] 1 illustrates a schematic architecture diagram of a vehicle control device according to an embodiment of the present disclosure.

[0048] [Figure 14] 14 shows a structural schematic diagram of a computing device 1400 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0049] In order to more clearly explain the technical solutions of the embodiments of the present disclosure, the drawings required for the embodiments are briefly introduced above, and the drawings herein are incorporated into the specification and constitute a part of this specification, these drawings show embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. The following drawings only show some embodiments of the present disclosure, so they should not be considered as limiting the scope, and it should be understood that those skilled in the art can also obtain other related drawings based on these drawings without requiring creative efforts.

[0050] In order to allow those skilled in the art to better understand the solutions disclosed herein, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without any creative efforts fall within the protection scope of the present disclosure.

[0051] The terms "first," "second," etc. in the specification and claims of this disclosure and in the above drawings are used to distinguish different objects without describing a particular order. Furthermore, the terms "comprises" and "includes" and variations thereof are intended to cover a non-exclusive inclusion.

[0052] For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units already listed, but may include additional steps or units that are not listed or may include other steps or units inherent to such process, method, product, or apparatus.

[0053] It should be understood that in this application, "at least one" refers to one or more, "multiple" refers to two or more, "at least two" refers to two or more, and "and / or" is used to describe a relation between related objects, indicating that three types of relations can exist. For example, "A and / or B" can indicate three situations: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " can indicate that the related objects before and after it are in an "or" relation, and refers to any combination of these terms, including any combination of one term or multiple terms. For example, at least one term of a, b, or c can refer to a, b, c, "a and b," "a and c," "b and c," or "a, b, and c," where a, b, and c can be singular or plural. The character " / " can also represent a division symbol in mathematical operations, for example, a / b = a divided by b, 6 / 3 = 2. "At least one of the following" or similar expressions.

[0054] Research has shown that in the existing disclosed technologies for binding people to vehicle seats, there are often situations in the process of binding people to vehicle seats that machines cannot recognize due to objective factors such as the person's sitting posture or height. For example, children are small in size, not tall enough, and therefore easy to move around, making them unable to be monitored by machines.

[0055] Furthermore, in the existing disclosed technologies relating to binding between a person and a vehicle seat, only one binding method is generally adopted in the process of binding between a person and a vehicle seat, and the binding is performed only once, with no related basic strategy, which is likely to lead to a situation where the execution device makes a mistake, thereby leading to a situation where the binding between the person and the vehicle seat is incorrect.

[0056] Based on the above research, an embodiment of the present disclosure provides a method for determining an occupant's seat in a vehicle, and when the detection frame of the target occupant of the seat cannot be determined based on the first seat area, the area range of the first seat area is expanded, and the seat corresponding to the detection frame of the target occupant is further determined based on the second seat area of ​​the expanded area range, thereby reducing the probability of failing to match or bind the occupant with the seat and improving the success rate of occupant seat matching.

[0057] It should be noted that like symbols and letters represent like items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and interpreted in subsequent drawings.

[0058] To facilitate understanding of the embodiments of the present disclosure, the method for determining the seat of a passenger in a vehicle and the vehicle control method disclosed in the embodiments of the present disclosure will be first described in detail. The execution body of the method for determining the seat of a passenger in a vehicle and the vehicle control method provided in the embodiments of the present disclosure is a vehicle control device (e.g., a rail transit cockpit intelligent analysis box, a bus cockpit intelligent analysis box) or a server.

[0059] Referring to FIG. 1, there is shown a flowchart of a method for determining a seat of an occupant in a vehicle according to an embodiment of the present disclosure, the method including steps S101 to S107.

[0060] In S101, a vehicle image of the interior of the vehicle is acquired.

[0061] At S102, a plurality of first seating areas in the vehicle image are determined, each of the first seating areas corresponding to one seat.

[0062] In S103, occupants in the vehicle are detected based on the vehicle image, and a detection frame for each of the occupants in the vehicle image is determined.

[0063] In S104, for each of the occupant detection frames, a seat corresponding to the detection frame is determined based on a first matching degree between the detection frame and each of the first seat areas.

[0064] At S105, a plurality of second seating areas in the vehicle image are determined, each of the second seating areas corresponding to a row or column of seats.

[0065] In S106, the detection frame of the target occupant of the corresponding target first seat area is not determined based on the first matching degree, whereas the target second seat area corresponding to the detection frame of the target occupant is determined based on the second matching degree between the detection frame of the target occupant and each of the second seat areas.

[0066] In S107, a seat corresponding to the detection frame of the target occupant is determined based on the position range of each seat in one row or one column of seats corresponding to the target second seat area and the detection frame of the target occupant.

[0067] Steps 101 to 107 will be explained in detail below.

[0068] About the S101: In one possible embodiment, step S101 may include capturing a vehicle image within the vehicle in response to satisfying an image capture trigger condition.

[0069] Here, satisfying the image collection trigger condition and acquiring a vehicle image of the vehicle can be achieved in any one of a number of scenarios described below.

[0070] In scenario A, the driver opens the vehicle and starts it. Here, the vehicle can be started by ignition or power supply. After the vehicle starts, the vehicle circuit system is turned on, the vehicle accesses the network, and starts capturing vehicle images inside the vehicle.

[0071] In scenario B, the driver opens the car door or triggers the start switch of the car. Here, triggering the start switch of the car may be performed by at least one of a method of turning on a remote control switch of the car, a method of inserting a start key of the car, a method of contacting the car body using a wearable device, etc. Here, after it is detected that the driver has opened the car door or triggered the start switch of the car, a vehicle image of the inside of the vehicle is acquired.

[0072] In scenario C, while the vehicle is traveling, images of the interior of the vehicle are collected periodically according to a preset cycle, that is, vehicle images of the interior of the vehicle can be acquired at regular intervals.

[0073] In one possible embodiment, the vehicle image is an image including the interior environment of the vehicle, and illustratively the content of the vehicle image may include all seats in the vehicle, people in the vehicle, and other pets, etc.

[0074] In one possible embodiment, the vehicle images may be collected and acquired using a driver monitoring system (DMS), and the collection process may involve using a DMS camera. In the process of collecting vehicle images using a DMS, the vehicle images need to be collected in real time while the vehicle is traveling, because the situation inside the vehicle changes dynamically in real time.

[0075] In practical applications, the seating positions of passengers in the vehicle may change while the vehicle is being driven, so the detection frame and seat area can be determined for each collected frame of vehicle image based on steps S102 to S107.

[0076] In one possible embodiment, a method of collecting and determining passenger seats at time intervals can be used, and after one collection, vehicle images are collected again based on the current vehicle situation after a preset time interval. This method can reduce the collection pressure of the DMS system and reduce resource occupation. In actual use, the preset time interval can be set as short as possible, so that the DMS system can update vehicle images more timely based on the vehicle situation inside the vehicle, reducing resource occupation and minimizing the error between the collected vehicle images and the real-time vehicle situation inside the vehicle.

[0077] In one possible embodiment, in response to satisfying the image collection trigger condition, the server performs an operation of acquiring a vehicle image inside the vehicle. Here, after detecting that a vehicle accesses a preset system, a server corresponding to the preset system can detect whether the vehicle satisfies the image collection trigger condition. If it is detected that the vehicle satisfies the image collection trigger condition, the server can send a vehicle image acquisition request to an on-board camera or an on-board host connected to the on-board camera, thereby acquiring a vehicle image inside the vehicle.

[0078] Here, the vehicle's access to the preset system can, for example, refer to accessing the preset system via an in-vehicle system (e.g., a navigation system installed in the vehicle) or via a wearable device carried by the driver (e.g., a smart watch, etc.).

[0079] If the vehicle only accesses the preset system but does not meet the image collection trigger condition, the server may not acquire vehicle images inside the vehicle in order to reduce resource occupation.

[0080] In another possible embodiment, the vehicle does not need to access the preset system, and after the vehicle meets the image collection trigger condition, the vehicle host side directly acquires the vehicle images inside the vehicle, so that the DMS system on the vehicle side can directly monitor the vehicle in real time and collect the vehicle images inside the vehicle, thereby improving the safety of vehicle use.

[0081] The above two embodiments can be combined in certain scenarios, where the vehicle cannot access the server due to extreme conditions, such as when the car is wet and stops, or when the vehicle reaches a road section without traffic lights, the DMS system built into the vehicle can detect the vehicle conditions inside the vehicle and update the current vehicle image, and after the vehicle returns to normal conditions, the server can continue to obtain the vehicle images inside the vehicle.

[0082] Regarding S102, After capturing a vehicle image within a vehicle, a plurality of first seating areas within the vehicle image can be determined.

[0083] Here, each of the first seating areas is an independently accessible seating area within the vehicle. When dividing the first seating area, each independently accessible seat within the vehicle can be divided into one first seating area, and there are multiple first seating areas within one vehicle. For example, as shown in Figure 2, a division method of the first seating area of ​​a four-seater vehicle is shown in Figure 2.

[0084] Here, each four-seater vehicle includes four seats, with the upper left being a driver's seat 201, the upper right being a passenger seat 202, and the lower left and lower right being two passenger seats (including 203 and 204), each of which can accommodate at least one passenger. When dividing the first seating area, each seat can be divided to correspond to one first seating area, so that this four-seater vehicle includes four first seating areas, namely, the driver's seat area 201, the passenger seat area 202, the passenger A seat area 203, and the passenger B seat area 204.

[0085] In one possible embodiment, in the process of determining multiple first seat areas in the vehicle image, a first seat area corresponding to each seat in the vehicle image can be determined based on seat distribution setting information of the vehicle.

[0086] The seat distribution setting information of the vehicle may include, for example, first seat range information of each seat inside the vehicle, a seat position layout of the vehicle, and the number of seats.

[0087] In one possible embodiment, the method for determining the seat distribution setting information of the vehicle may include determining, based on vehicle model information, seat distribution setting information corresponding to the vehicle model information. Here, a correspondence relationship between vehicle model information and seat distribution setting information may be stored in advance, and when determining the seat distribution setting information of the vehicle, the seat distribution setting information corresponding to the vehicle model information may be found based on the correspondence relationship.

[0088] In the process of determining the vehicle model information, if the vehicle can execute a preset strategy for automatically reporting the vehicle model information, the vehicle model information can be automatically reported by the vehicle when the vehicle accesses the preset system. If the vehicle model used by the occupant is too old to execute the preset automatic reporting strategy, the occupant can report the relevant vehicle model information by himself.

[0089] Here, the first seat range information may include image coordinate range information corresponding to a first seat area in the vehicle interior of the vehicle model, and the image coordinate range information may be obtained by dividing the seats in the collected vehicle interior image after a camera for collecting the vehicle image is fixedly installed inside the vehicle and calculating boundary coordinates of each corresponding seat area. Each of the first seat areas can be divided from the vehicle image based on the corresponding coordinate range information of the first seat area.

[0090] Regarding S103, The detection frame of each of the occupants in the vehicle image is at least one of a face detection frame and a human body detection frame. In one possible embodiment, the process of determining the detection frame of each of the occupants in the vehicle image may include inputting the vehicle image into a pre-trained detection model to determine a detection frame corresponding to each of the occupants in the vehicle image.

[0091] In one possible embodiment, the detection model for detecting the face detection frame may be a first model, and the detection model for detecting the human body detection frame may be a second model, and the supervision data for the first model and the second model may be different, and the supervision data for the first model may be a pre-labeled face frame, and the supervision data for the second model may be a pre-labeled human body frame.

[0092] Alternatively, in another possible embodiment, the detection model for detecting face detection frames and the detection model for detecting human body detection frames may be the same detection model, and the supervision data for the detection model includes two types: pre-labeled face frames and pre-labeled human body frames.

[0093] In practical applications, steps S102 and S103 can be performed simultaneously based on the sequence shown in FIG. 1, or steps S102 and S103 can be performed one after the other. The present solution does not place excessive restrictions on the prior / post-post division of the above two steps.

[0094] Regarding step S104, The first matching degree between the detection frame and each of the first seat areas represents the correlation of spatial positions and may be obtained based on the overlap area, overlap degree, etc. between the detection frame and each of the first seat areas.

[0095] In one possible implementation, the detection frame of each occupant can be matched with each of the first seat areas, and for example, the overlap area between the detection frame and the first seat area can be calculated, and the seat area among multiple seat areas in the vehicle cabin that has the largest overlap area with the detection frame can be determined to be the seat corresponding to the detection frame, thereby determining the seat in which the occupant is sitting within the detection frame.

[0096] In one possible embodiment, the above step of determining the seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas can be realized by steps S301 to S303.

[0097] In step S301, each of the first seating areas is traversed according to a preset order.

[0098] In one possible embodiment, the process of traversing each of the first seating areas according to a preset order may start from the driver's seat and traverse first from left to right, then from top to bottom. For example, as shown in FIG. 2, four first seating areas are included in FIG. 2, and the process of traversing each of the first seating areas may traverse sequentially according to the order of the driver's seat 201, the passenger seat 202, the passenger A seat 203, and the passenger B seat 204.

[0099] In another possible embodiment, starting from the driver's seat, the traverse can be performed from top to bottom, and then from left to right. For example, as shown in FIG. 2, four first seating areas are included in FIG. 2, and in the process of traversing each of the first seating areas, the traverse can be performed in the order of the driver's seat 201, the passenger A seat 203, the passenger seat 202, and the passenger B seat 204.

[0100] For each first seating area that is traversed, steps S302 and S303 are performed.

[0101] In step S302, the degree of overlap between the detection frame and the first seat area is set as a first matching degree between the detection frame and the first seat area.

[0102] The overlap degree may be, for example, an intersection-over-union (IoU) ratio, i.e., a ratio of the intersection to the union between the detection frame and the first seat area. For example, as shown in Fig. 4, the first seat area in Fig. 4 is a driver's seat area 401, where the part highlighted in medium gray is an area 403 commonly included by the driver's seat area 401 and the detection frame 402, i.e., the union area. The sum of all areas of the driver's seat area and the detection frame is the intersection area. Calculating the ratio of the intersection area to the union area, i.e., obtaining the overlap degree between the driver's seat area and the detection frame, this overlap degree is regarded as a first matching degree between the detection frame and the driver's seat area.

[0103] In step S303, if the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to another detection frame, the seat corresponding to the first seat area is bound to the detection frame, and the seat corresponding to the detection frame is determined to be the seat corresponding to the first seat area.

[0104] In the process of traversing the first seat area according to a preset order, it can be detected whether the first seat area is bound to another detection frame. When a first seat area is bound to a detection frame, it indicates that the first seat area is a seating position for the occupant corresponding to the detection frame. Since multiple occupants cannot normally be accommodated in the first seat area, when the first matching degree reaches a first threshold, it can be detected whether the first seat area is bound to another detection frame, thereby avoiding an unreasonable detection result in which multiple detection frames are bound to one seat area and multiple occupants are seated in the same seat.

[0105] In one possible embodiment, the first threshold value of the face detection frame and the first threshold value corresponding to the human body detection frame in the first seating area may be different. Because there may be differences in distances and relative angles between different seating areas and the camera that collects vehicle images, the face detection frame in each seating area may correspond to a respective first threshold value, and the human body detection frame in each seating area may correspond to a respective first threshold value.

[0106] In one possible embodiment, for a first seating area currently being traversed, if the first matching degree between a detection frame and the first seating area reaches a corresponding first threshold and the seat corresponding to the first seating area is not bound to any other detection frame, the seat corresponding to the first seating area is determined to be the seat corresponding to the detection frame, and the first seating area is bound to the detection frame with which the first matching degree reaches the corresponding first threshold.

[0107] In one possible embodiment, if the first matching degree calculated between a face detection frame of a certain occupant and the first seat area reaches a corresponding first threshold, but the first matching degree between the occupant's human body detection frame and the first seat area does not reach the corresponding first threshold, the first seat area is bound to the occupant's face detection frame, the seat corresponding to the face detection frame is determined to be the seat corresponding to the first seat area, and the occupant's face is further bound to the seat corresponding to the first seat area.

[0108] Conversely, if the first matching degree calculated between a certain occupant's human body detection frame and the first seat area reaches the corresponding first threshold, but the first matching degree calculated between the occupant's face detection frame and the first seat area does not reach the corresponding first threshold, the first seat area is bound to the occupant's human body detection frame, the seat corresponding to the human body detection frame is determined to be the seat corresponding to the first seat area, and the occupant's human body is further bound to the seat corresponding to the first seat area.

[0109] In practical applications, a double matching method can be adopted, in which the seat area is matched sequentially with the human body detection frame and the face detection frame. When the first matching degree between one of the human body detection frame and face detection frame of an occupant and a first seat area reaches a first threshold, the face detection frame and human body detection frame of the occupant are all bound to the first seat area, and the occupant is further bound to the first seat area, thereby improving the binding efficiency.

[0110] In another possible embodiment, a method using only a human body detection frame or only a face detection frame can be adopted to match seat areas. Here, in a solution using only a face detection frame for detection, if the matching degree between the face detection frame and a first seat area reaches a corresponding first threshold and the seat corresponding to the first seat area is not bound to any other detection frame, the face detection frame is bound to the first seat area, the seat corresponding to the face detection frame is determined to be the seat corresponding to the first seat area, and the face of the occupant is further bound to the first seat area. If the first matching degree does not exceed the corresponding matching threshold, the face of the occupant is not bound to the first seat area. The method for detecting using a human body detection frame is similar to the method for detecting using a face detection frame.

[0111] Regarding step S105, The area coverage of the second seating area is larger than that of the first seating area, and in one possible embodiment, the second seating area is a seating area that includes a portion of the vehicle's air gap spacing, and each of the second seating areas corresponds to one row or one column of seats.

[0112] In one possible embodiment, a second seating area can be determined based on the determined first seating area. Here, based on the determined first seating area and a vehicle image, some gaps in the first seating area can be filled in, and each row or column can be determined as a second seating area. Next, multiple areas within the second seating area can be divided based on the vehicle's vehicle type information, where each second seating area corresponds to a row of seats. For example, each second seating area can be further divided according to the following scheme: If each row of a vehicle has two seats, it can be divided at 1 / 2 and then into a left-column area and a right-column area. If each row of a vehicle has three seats, it can be divided at 1 / 3, as in this analogy.

[0113] In another possible embodiment, the second seat area can be directly determined based on the vehicle image. Here, in the solution of dividing according to rows, the gap intervals of each row in the vehicle image can be directly removed, and each row in the vehicle can be determined as a second seat area, and then the left column area and the left column area can be divided within the second seat area. Similarly, in the solution of dividing according to columns, the gap intervals of each column in the vehicle image can be directly removed, and each column in the vehicle can be determined as a second seat area, and then the upper column area and the lower column area can be divided within the second seat area.

[0114] For example, as shown in FIG. 5(a), FIG. 5(a) shows a schematic diagram of a second seating area of ​​a four-seater vehicle divided by rows, where FIG. 5(a) includes two second seating areas, which are second seating area A501 and second seating area B502, respectively, and each row includes a left column area and a right column area, and each left column area and each right column area corresponds to one seat.

[0115] Or, for example, as shown in FIG. 5(b), FIG. 5(b) shows a schematic diagram of a second seating area of ​​a four-seater vehicle divided by rows, where FIG. 5(b) includes two second seating areas, which are second seating area C503 and second seating area D504, respectively, and each row includes a front row area and a rear row area, and each front row area and each rear row area corresponds to one seat.

[0116] Regarding step S106, In one possible embodiment, the detection frame of the target occupant of the corresponding target first seat area is not determined based on the first matching degree, whereas the above step of determining the target second seat area corresponding to the detection frame of the target occupant based on the second matching degree between the detection frame of the target occupant and each of the second seat areas can be realized by steps S601 to S603, as shown in Figure 6.

[0117] In step S601, the second seating areas are traversed in a predetermined order, and for each of the traversed second seating areas, In one possible embodiment, in the process of traversing the second seating area according to a preset order, if there are multiple rows or multiple columns of seats, it can start from the second seating area where the driver's seat is located and traverse row by row or column by column depending on the actual situation.

[0118] For each second seating area that has been traversed, the following steps S602 and S603 are performed.

[0119] In step S602, the degree of overlap between the detection frame for the target occupant and the second seat area is set as a second matching degree between the detection frame for the target occupant and the second seat area.

[0120] In step S603, if the second matching degree reaches a second threshold, it is determined that the second seat area is a target second seat area corresponding to the detection frame of the target occupant.

[0121] For example, as shown in Figure 7(a), the degree of overlap between the target occupant detection frame 701 and the second seat area E702 and the degree of overlap between the target occupant detection frame 701 and the second seat area F703 are calculated as corresponding second matching degrees, and if the second matching degree corresponding to the target occupant detection frame and the second seat area E reaches a second threshold, the second seat area E is determined to be the target second seat area corresponding to the target occupant detection frame.

[0122] In one possible embodiment, for each traversed second seat area, if the degree of overlap does not reach a second threshold, the second seat area with the largest overlap area with the detection frame of the target occupant can be determined to be the target second seat area.

[0123] For example, as shown in Figure 7(b), the overlap area between the target occupant detection frame 701 and the second seat area G704 is larger than the overlap area between the target occupant detection frame 701 and the second seat area H705, so the second seat area G704 is determined as the target second seat area corresponding to the target occupant detection frame.

[0124] Regarding step S107, In one possible embodiment, the above step of determining the seat corresponding to the detection frame of the target occupant based on the position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant may include steps S801 to S802, as shown in Figure 8.

[0125] In step S801, based on the detection frame of the target occupant and the position range of each seat within the target second seat area, the seat corresponding to the position range in which the center point of the detection frame of the target occupant is located is determined to be the candidate seat corresponding to the detection frame of the target occupant.

[0126] In one possible embodiment, the position range in which the center point of the target occupant detection frame is located can be determined by calculating the Euclidean distance between the target occupant detection frame and the center point of each seat position range. For example, as shown in Fig. 9, the Euclidean distance between the target occupant detection frame 901 and the center point 902 of the left column region and the Euclidean distance between the target occupant detection frame 901 and the center point 903 of the right column region are calculated, i.e., the straight-line distance between the target occupant detection frame and the two center points is calculated.

[0127] Next, the seat corresponding to the position range with the shortest straight-line distance is determined as the candidate seat corresponding to the detection frame of the target occupant. In Fig. 9, the distance from the left-row region to the center point of the detection frame 901 of the target occupant to be bound is shorter than the distance from the right-row region, so the seat corresponding to the left-row region is determined as the candidate seat corresponding to the detection frame of the target occupant.

[0128] In another possible embodiment, an intersection between the detection frame of the target occupant and each seat in the target second seat area can be calculated, and the larger intersection is determined as the candidate seat corresponding to the detection frame of the target occupant.

[0129] For example, as shown in Figure 9, since the intersection between the detection frame of the target occupant to be bound and the left column area is larger than the intersection with the right column area, the seat corresponding to the left column area is determined to be the candidate seat corresponding to the detection frame of the target occupant.

[0130] In step S802, if the candidate seat is not bound to another detection frame, it is determined that the candidate seat is the seat corresponding to the detection frame of the target occupant, and the seat corresponding to the candidate seat is bound to the detection frame of the target occupant.

[0131] In one possible embodiment, if a target second seat area corresponding to the detection frame of the target occupant is not determined, a seat whose corresponding first seat area is closest to the detection frame of the target occupant and is not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.

[0132] In one possible embodiment, the detection frame of the target occupant and the center point of each of the first seat areas are first determined, and then, based on the center points, the seat with the shortest straight-line distance between the two center points and that is not bound to other detection frames is determined as the seat corresponding to the detection frame of the target occupant.

[0133] 10 exemplarily, Fig. 10 includes a plurality of second seating areas, which are a first seating area A1001, a first seating area B1002, a first seating area C1003, and a first seating area D1004, and the target occupant detection frame 1005 does not intersect with any of the above-mentioned four first seating areas. Since the linear distance between the center point of the target occupant detection frame and the center point of the first seating area A is the shortest and the first seating area A is not bound to other detection frames, the first seating area A1001 is determined as the seat corresponding to the target occupant detection frame 1005.

[0134] Based on the method for determining the seat of a passenger in a vehicle proposed in the present solution, an embodiment of the present disclosure further provides a vehicle control method executed by an electronic device. As shown in Figure 11, a schematic diagram of the vehicle control method according to the embodiment of the present disclosure includes the following steps S1101 and S1102.

[0135] In step S1101, a detection frame corresponding to each seat area is determined based on the method for determining the seats of passengers in a vehicle described in the above embodiment.

[0136] In step S1102, a target motion performed by a target occupant within the target detection frame is detected, and vehicle control is performed based on the target motion and the seat area corresponding to the target detection frame.

[0137] In practical applications, based on the method for determining a seat of a passenger in a vehicle proposed in the above embodiments, a vehicle control can be realized, in which, in response to a target command corresponding to a target movement of a passenger, the vehicle performs a target operation corresponding to the target command.

[0138] For example, the vehicle control method can be applied to the field of intelligent driving, in which after detecting that a target detection frame corresponding to a target occupant is bound to a target seat area, and the occupant makes a gesture to instruct "open the window", the vehicle opens the window above the corresponding door according to the occupant's gesture command and the occupant's seat position. For example, if the occupant who issued the gesture command is a passenger seat occupant, the vehicle can open the window above the passenger seat door, thereby realizing precise control based on passenger seat detection.

[0139] Those skilled in the art can understand that in the above method of the specific embodiment, the order in which each step is written does not mean a strict execution order and does not constitute any restriction on the implementation process, and the execution order of each step should be determined by its function and possible internal logic.

[0140] Based on the same inventive concept, an embodiment of the present disclosure further provides an apparatus for determining a seat of a passenger in a vehicle, which corresponds to the method for determining a seat of a passenger in a vehicle. The principle for solving the problem of the apparatus in the embodiment of the present disclosure is similar to the above-mentioned method for determining a seat of a passenger in a vehicle in the embodiment of the present disclosure, so that the implementation of the apparatus can refer to the implementation of the method.

[0141] Referring to FIG. 12 , there is shown a schematic architecture diagram of an apparatus for determining a seat of an occupant in a vehicle according to an embodiment of the present disclosure, the apparatus comprising an acquisition unit 1201, a first determination unit 1202, a second determination unit 1203, a third determination unit 1204, a fourth determination unit 1205, a fifth determination unit 1206, and a sixth determination unit 1207.

[0142] The acquisition unit 1201 is configured to acquire vehicle images of the interior of the vehicle.

[0143] The first determining unit 1202 is configured to determine a plurality of first seating areas in the vehicle image, each first seating area corresponding to one seat.

[0144] The second determination unit 1203 is configured to detect occupants in the vehicle based on the vehicle image, and determine a detection frame for each of the occupants in the vehicle image.

[0145] The third determination unit 1204 is configured to determine, for each of the occupant detection frames, a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas.

[0146] The fourth determining unit 1205 is configured to determine a plurality of second seating areas in the vehicle image, each second seating area corresponding to a row or a column of seats.

[0147] The fifth determination unit 1206 is configured to determine a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the second seat areas, while the detection frame of the target occupant of the corresponding target first seat area is not determined based on the first matching degree.

[0148] The sixth determination unit 1207 is configured to determine the seat corresponding to the detection frame of the target occupant based on the position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant.

[0149] In one possible embodiment, the first determining unit 1202 is further configured to determine a first seat area corresponding to each seat in the vehicle image based on seat distribution setting information of the vehicle.

[0150] In one possible embodiment, the third determination unit 1204 is further configured to traverse each of the first seat areas according to a preset order, and for each traversed first seat area, calculate the overlap degree between the detection frame and the first seat area as a first matching degree between the detection frame and the first seat area, and if the first matching degree reaches a first threshold and the seat corresponding to the first seat area is not bound to another detection frame, bind the seat corresponding to the first seat area to the detection frame and determine that the seat corresponding to the detection frame is the seat corresponding to the first seat area.

[0151] In one possible embodiment, the detection frame of each of the occupants in the vehicle image is at least one of a face detection frame and a human body detection frame.

[0152] In one possible embodiment, the fifth determination unit 1206 is further configured to traverse the second seat area according to a predetermined order for each target occupant detection frame, calculate the overlap degree between the target occupant detection frame and the second seat area for each traversed second seat area as the second matching degree, and determine that the second seat area is the target second seat area corresponding to the target occupant detection frame if the second matching degree reaches a second threshold.

[0153] In one possible embodiment, the sixth determination unit 1207 is further configured to determine, based on the detection frame of the target occupant and the position range of each seat within the target second seat area, that the seat corresponding to the position range in which the center point of the detection frame of the target occupant is located is a candidate seat corresponding to the detection frame of the target occupant, and if the candidate seat is not bound to any other detection frame, determine that the candidate seat is the seat corresponding to the detection frame of the target occupant, and bind the seat corresponding to the candidate seat to the detection frame of the target occupant.

[0154] In one possible embodiment, the fifth determination unit 1206 is further configured to determine, for each traversed second seat area, if the degree of overlap does not reach a second threshold, that second seat area having the largest overlap area with the detection frame of the target occupant is the target second seat area.

[0155] In one possible embodiment, the seventh determination unit 1208 is further configured to determine, when a target second seat area corresponding to the detection frame of the target occupant is not determined, a seat whose corresponding first seat area is closest to the detection frame of the target occupant and is not bound to other detection frames, as the seat corresponding to the detection frame of the target occupant.

[0156] For the description of the processing process of each part in the device and the interaction process between each part, please refer to the relevant description in the above-mentioned method embodiment.

[0157] Based on the same inventive concept, an embodiment of the present disclosure further provides a vehicle control device corresponding to the vehicle control method. The principle of solving the problem of the device in the embodiment of the present disclosure is similar to the above-mentioned method for determining the seat of a passenger in a vehicle in the embodiment of the present disclosure, so the implementation of the device can refer to the implementation of the method.

[0158] Referring to FIG. 13, there is shown a schematic architecture diagram of a vehicle control device according to an embodiment of the present disclosure, the device comprising an eighth determining unit 1301 and a detecting unit 1302.

[0159] The eighth determination unit 1301 is configured to determine a detection frame corresponding to each of the seat areas based on the method for determining the seats of passengers in a vehicle described in the above embodiments.

[0160] The detection unit 1302 is configured to detect a target action performed by a target occupant within the target detection frame, and to perform vehicle control based on the target action and the seat area corresponding to the target detection frame.

[0161] For the description of the processing process of each part in the device and the interaction process between each part, reference can be made to the relevant descriptions in the above-mentioned method embodiments.

[0162] In this and other embodiments of the present disclosure, a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., and may of course be a unit, a module, or non-modular.

[0163] Based on the same technical concept, an embodiment of the present disclosure further provides a computer device. Referring to FIG. 14, there is shown a structural schematic diagram of a computer device 1400 according to an embodiment of the present disclosure, which includes a processor 1401, a memory 1402, and a bus 1403. Here, the memory 1402 is used to store executable instructions and includes an internal memory 14021 and an external memory 14022. Here, the internal memory 14021 is also called an internal storage device and is used to temporarily store calculation data within the processor 1401 and data to be exchanged with the external memory 14022, such as a hard disk. The processor 1401 exchanges data with the external memory 14022 via the internal memory 14021. When the computer device 1400 operates, the processor 1401 and the memory 1402 communicate with each other via the bus 1403, and the processor 1401 receives and stores data. capturing vehicle images within the vehicle; determining a plurality of first seating areas within the vehicle image, each first seating area corresponding to a seat; detecting occupants in the vehicle based on the vehicle image and determining a detection frame for each of the occupants in the vehicle image; determining, for each of the occupant detection frames, a seat corresponding to the detection frame based on a first matching degree between the detection frame and each of the first seat areas; determining a plurality of second seating areas within the vehicle image, each second seating area corresponding to a row or column of seats; determining a target second seat area corresponding to the target occupant detection frame based on a second matching degree between the target occupant detection frame and each of the second seat areas, when the target occupant detection frame of the corresponding target first seat area is not determined based on the first matching degree; and The system executes the process of determining the seat corresponding to the detection frame of the target occupant based on the position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant.

[0164] Alternatively, the processor 1401 may: determining a detection frame corresponding to each of the seat areas based on the method for determining passenger seats in a vehicle described in the above embodiment; and It is also possible to detect a target action performed by a target occupant within the target detection frame, and to perform vehicle control based on the target action and the seat area corresponding to the target detection frame.

[0165] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, the computer program being configured to perform the steps of the method for determining a seat of an occupant in a vehicle and the vehicle control method described in the above embodiment of the method when executed by a processor, wherein the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0166] It should be pointed out here that the above description of the storage medium and device embodiments is similar to the description of the above method embodiments, and has the same beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure.

[0167] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that the electronic device that realizes the processor function may be other, but is not specifically limited in the embodiments of the present disclosure.

[0168] The above-mentioned computer storage medium / memory may be a memory such as a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM), and may also be various types of terminals, such as mobile phones, computers, tablet devices, personal digital assistants, etc., that include one or any combination of the above-mentioned memories.

[0169] An embodiment of the present disclosure further provides a computer program product, wherein the computer program product carries program code, and instructions included in the program code may be used to perform steps of the method for determining a seat of an occupant in a vehicle, the vehicle control method described in the above method embodiment, and reference may be made to the above method embodiment.

[0170] Here, the computer program product may be realized in hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK).

[0171] Those skilled in the art will clearly understand that, for convenience and conciseness, the operation processes of the above-described systems and devices can refer to the corresponding processes in the method embodiments. It should be understood that in some embodiments provided in the present disclosure, the disclosed systems, devices, and methods may be realized in other ways. The above-described device embodiments are merely illustrative. For example, the division of the units is merely a logical and functional division. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections via some communication interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0172] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units, and some or all of them can be selected according to actual needs to achieve the objective of the solution of this embodiment.

[0173] Furthermore, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit.

[0174] When the functions are realized in the form of software functional units and sold or used as independent products, they may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure, or portions that contribute to the prior art or portions of the technical solutions, may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program code, such as a USB flash disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0175] Finally, it should be explained that the above examples are only specific embodiments of the present disclosure, and are intended to illustrate the technical solutions of the present disclosure without limiting them, and the scope of protection of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above examples, those skilled in the art should understand that, within the technical scope disclosed in the present disclosure, any person skilled in the art may still easily think of modifying or changing the technical solutions described in the above examples, or make equivalent substitutions for part of the technical features, and these modifications, changes, or substitutions will not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and all should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be subject to the scope of protection of the claims. [Industrial Applicability]

[0176] An embodiment of the present disclosure provides a method for determining occupant seats in a vehicle, a vehicle control method, and an apparatus for controlling the vehicle, the method for determining occupant seats in a vehicle including: acquiring a vehicle image of the interior of the vehicle; determining a plurality of first seat areas in the vehicle image; detecting occupants in the vehicle based on the vehicle image and determining a detection frame for each of the occupants in the vehicle image; determining a seat corresponding to each occupant detection frame based on a first matching degree between the detection frame and each of the first seat areas; determining a plurality of second seat areas in the vehicle image; and, if a target occupant detection frame for a corresponding target first seat area is not determined based on the first matching degree, determining a target second seat area corresponding to the target occupant detection frame based on a second matching degree between the target occupant detection frame and each of the second seat areas; and determining a seat corresponding to the target occupant detection frame based on a position range of each seat in a row or column of seats corresponding to the target second seat area and the target occupant detection frame. When the detection frame of the target occupant of the seat cannot be determined by the first seat area, the area range of the first seat area is expanded, and the seat corresponding to the detection frame of the target occupant is further determined based on the second seat area of ​​the expanded area range, thereby reducing the probability of failure in matching or binding between the occupant and the seat and improving the success rate of matching the occupant to the seat.

Claims

1. 1. A method for determining an occupant's seat in a vehicle performed by an electronic device, comprising: acquiring vehicle images within the vehicle; determining a plurality of first seating areas within the vehicle image, each of the plurality of first seating areas corresponding to a seat; detecting an occupant in the vehicle based on the vehicle image and determining a detection frame for the occupant in the vehicle image; determining a seat corresponding to the occupant detection frame based on a first matching degree between the occupant detection frame and each of the plurality of first seat areas; determining a plurality of second seating areas within the vehicle image, each of the plurality of second seating areas corresponding to a row or column of seats; determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the plurality of second seat areas, when a detection frame of the target occupant of the corresponding target first seat area is not determined based on the first matching degree; determining a seat corresponding to the detection frame of the target occupant based on a position range of each seat in one row or one column of seats corresponding to the target second seat area and the detection frame of the target occupant; 1. A method for determining an occupant's seat in a vehicle, comprising:

2. Determining a plurality of first seating areas within the vehicle image includes: The method for determining passenger seats in a vehicle according to claim 1 , further comprising: determining a first seating area corresponding to each seat in the vehicle image based on seat distribution setting information of the vehicle.

3. determining a seat corresponding to the occupant detection frame based on a first matching degree between the occupant detection frame and each of the plurality of first seat areas; traversing each of the plurality of first seating areas according to a predetermined sequence; For each of the traversed first seat areas, an overlap degree between the occupant detection frame and the first seat area is determined as a first matching degree between the occupant detection frame and the first seat area; When the first matching degree reaches a first threshold value and the seat corresponding to the first seat area is not bound to another detection frame, binding the seat corresponding to the first seat area to the occupant detection frame, and determining that the seat corresponding to the occupant detection frame is the seat corresponding to the first seat area; 2. The method of claim 1, further comprising:

4. The method of claim 1 , wherein the occupant detection frame in the vehicle image is at least one of a face detection frame and a human body detection frame.

5. determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the plurality of second seat areas; traversing each of the plurality of second seating areas according to a predetermined sequence; For each of the traversed second seat areas, an overlap degree between the detection frame of the target occupant and the second seat area is set as a second matching degree between the detection frame of the target occupant and the second seat area; If the second matching degree reaches a second threshold, determining that the second seat area is a target second seat area corresponding to the detection frame of the target occupant; 2. The method of claim 1, further comprising:

6. determining a seat corresponding to the detection frame of the target occupant based on a position range of each seat in one row or one column of seats corresponding to the target second seat area and the detection frame of the target occupant, determining, based on the detection frame of the target occupant and a position range of each seat within the target second seat area, that a seat corresponding to the position range in which a center point of the detection frame of the target occupant is located, as a candidate seat corresponding to the detection frame of the target occupant; If the candidate seat is not bound to another detection frame, determining that the candidate seat is a seat corresponding to the detection frame of the target occupant, and binding the seat corresponding to the candidate seat to the detection frame of the target occupant; 2. The method of claim 1, further comprising:

7. determining a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the plurality of second seat areas; 6. The method for determining a seat of an occupant in a vehicle as described in claim 5, further comprising: for each of the traversed second seating areas, if the degree of overlap does not reach the second threshold, determining that the second seating area having the largest overlap area with the detection frame of the target occupant is the target second seating area.

8. 2. The method for determining a seat of an occupant in a vehicle as described in claim 1, further comprising: if a target second seat area corresponding to the detection frame of the target occupant is not determined, determining a seat corresponding to a first seat area among the plurality of first seat areas that is closest to the detection frame of the target occupant and is not bound to other detection frames as the seat corresponding to the detection frame of the target occupant.

9. A vehicle control method executed by an electronic device, comprising: determining a detection frame corresponding to each of the plurality of first seat areas based on the method for determining seats of occupants in a vehicle according to any one of claims 1 to 8; detecting a target action performed by a target occupant within the target detection frame, and performing vehicle control based on the target action and a seat area corresponding to the target detection frame; A vehicle control method comprising:

10. 1. An apparatus for determining a seat of an occupant in a vehicle, comprising: an acquisition unit configured to acquire vehicle images of the interior of the vehicle; a first determination unit configured to determine a plurality of first seating areas in the vehicle image, each of the plurality of first seating areas corresponding to one seat; and a second determination unit configured to detect an occupant in the vehicle based on the vehicle image and determine a detection frame for the occupant in the vehicle image; a third determination unit configured to determine, with respect to the occupant detection frame, a seat corresponding to the occupant detection frame based on a first matching degree between the occupant detection frame and each of the plurality of first seat areas; a fourth determination unit configured to determine a plurality of second seating areas in the vehicle image, each of the plurality of second seating areas corresponding to a row or a column of seats; a fifth determination unit configured to determine a target second seat area corresponding to the detection frame of the target occupant based on a second matching degree between the detection frame of the target occupant and each of the plurality of second seat areas when a detection frame of the target occupant of the corresponding target first seat area is not determined based on the first matching degree; a sixth determination unit configured to determine a seat corresponding to the detection frame of the target occupant based on a position range of each seat in a row or column of seats corresponding to the target second seat area and the detection frame of the target occupant; 1. An apparatus for determining a seat of an occupant in a vehicle, comprising:

11. A vehicle control device, an eighth determination unit configured to determine detection frames corresponding to the plurality of first seat areas based on the method for determining seats of occupants in a vehicle according to any one of claims 1 to 8; a detection unit configured to detect a target action performed by a target occupant within the target detection frame and to perform vehicle control based on the target action and a seat area corresponding to the target detection frame; A vehicle control device comprising:

12. 9. A computing device comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and wherein when the computing device is in operation, the processor and the memory communicate via the bus, and wherein the processor performs the method for determining an occupant seat in a vehicle according to any one of claims 1 to 8 by executing the machine-readable instructions.

13. A computer-readable storage medium on which a computer program is stored, which, when executed by a processor, causes the processor to perform a method for determining the seats of occupants in a vehicle as described in any one of claims 1 to 8.

14. A computer program that causes a computer to execute a method for determining the seats of occupants in a vehicle as described in any one of claims 1 to 8.

Images