Opening method and device for non-driving-position vehicle door, computer equipment and storage medium

Abstract

The invention relates to the technical field of vehicle automatic control, and discloses a non-driving-position vehicle door opening method and device, computer equipment and a storage medium. According to the method, before a vehicle door is opened, whether the non-driving-position vehicle door is opened or not is determined according to double verification that door opening is determined according to the door opening intention of a target object and a driving object; therefore, whether the non-driving position vehicle door is opened or not is judged more accurately; meanwhile, by improving the accuracy of vehicle door opening judgment, the situation that driving potential safety hazards of a driving object are caused by wrong door opening can be avoided, and then the user experience degree is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle automatic control technology, and in particular to a method, device, computer equipment and storage medium for opening a non-driver's door. Background Art

[0002] With the development of technology, the intelligence and convenience of vehicles have become important experiences that users pursue. Traditional vehicle doors need to be opened and closed manually, which is cumbersome and not convenient enough in some scenarios. In recent years, some systems that use sensors to automatically open and close doors have appeared on the market, but these systems usually cannot accurately judge the door opening scenario, resulting in incorrect door opening, which in turn leads to poor user experience.

[0003] Therefore, it is necessary to provide a non-driver's door opening method to solve the above problems. Summary of the invention

[0004] In view of the above problems, the present application provides a method, device, computer equipment and storage medium for opening a non-driver's door, which are used to solve the technical problem in the prior art that the door opening scenario cannot be accurately judged, the door is opened incorrectly, and the user experience is poor. Before opening the door, double verification is performed based on the target object's door opening intention and the driver's determination to open the door to determine whether to open the non-driver's door; thereby achieving a more accurate judgment on whether to open the non-driver's door; at the same time, by improving the accuracy of the door opening judgment, it is possible to avoid driving safety hazards to the driver due to incorrect door opening, thereby improving the user experience.

[0005] According to one aspect of an embodiment of the present application, a method for opening a non-driver's door is provided, wherein an image acquisition device is arranged outside the non-driver's door, and the method comprises: when the vehicle is in a non-driving state and a load signal of the driver's seat is sensed, if a target object is detected within a preset distance from the non-driver's door, then controlling the image acquisition device to acquire a video stream; wherein the load signal is a signal representing that the driver's seat carries a driving object; extracting facial features and posture features of the target object from the video stream, and if the facial features are non-preset features and the posture features represent that the target object has an intention to open the door, outputting a first inquiry message to inquire the driving object to determine whether to open the non-driver's door; and opening the non-driver's door according to an instruction to open the non-driver's door transmitted back by the driving object in response to the first inquiry message.

[0006] In an optional manner, the method further includes: if the facial feature is a preset feature, and the posture feature indicates that the target object has an intention to open the door, then opening the non-driver's door.

[0007] In an optional manner, after the step of outputting the first inquiry information to inquire the driving object to determine whether to open the non-driving door, it also includes: if it is determined to open the non-driving door, obtaining the last frame image of the video stream; detecting whether there are other objects within a preset distance threshold range of the non-driving door in the last frame image; wherein the other objects are objects other than the target object; if so, issuing a safety warning prompt to prompt that the non-driving door currently does not meet the door opening conditions; if not, opening the non-driving door.

[0008] In an optional manner, after the step of outputting the first inquiry information to inquire the driving subject to determine whether to open the non-driver's door, the method further includes: if it is determined to open the non-driver's door, outputting the second inquiry information to inquire the driving subject to determine whether to store the facial features as preset features.

[0009] In an optional manner, the method also includes: filtering out target image frames that meet preset conditions from the video stream; extracting posture features of the target object from the target image frames as target posture features; determining various behavioral actions of the target object based on the target posture features; matching each of the behavioral actions with a preset action set; wherein the preset action set includes multiple actions with the intention of opening the door; if at least one of the behavioral actions is successfully matched, it is determined that the target object has the intention to open the door.

[0010] In an optional manner, the preset condition includes that the facial orientation of the target object is a preset orientation; the step of filtering out target image frames that meet the preset conditions from the video stream further includes: filtering out image frames in which the facial orientation of the target object is a preset orientation from the video stream as to-be-selected image frames; and filtering out image frames with continuous acquisition time from the to-be-selected image frames as the target image frames.

[0011] In an optional manner, the method further includes: determining each behavioral action of the target object based on the posture features; matching each behavioral action with a preset action set, and taking the successfully matched behavioral action as the target behavioral action; wherein the preset action set includes a plurality of actions with the intention of opening the door; detecting the facial orientation of the target object corresponding to each of the target behavioral actions; if at least one facial orientation is the target orientation, determining that the target object has the intention to open the door.

[0012] According to another aspect of an embodiment of the present application, a non-driver's door opening device is provided, comprising: a video acquisition module, for controlling the image acquisition device to capture a video stream when a target object is detected within a preset distance from the non-driver's door when the vehicle is in a non-driving state and a load signal of the driver's seat is sensed; wherein the load signal is a signal indicating that the driver's seat carries a driving object; a control module, for extracting facial features and posture features of the target object from the video stream, and outputting a first inquiry message to inquire the driving object whether to open the non-driver's door if the facial features are non-preset features and the posture features indicate that the target object has an intention to open the door; and a door opening module, for opening the non-driver's door according to an instruction to open the non-driver's door transmitted back by the driving object in response to the first inquiry message.

[0013] According to another aspect of an embodiment of the present application, a computer device is provided, comprising: a controller; and a memory for storing one or more programs, wherein when the one or more programs are executed by the controller, the controller implements the method for opening a non-driver's door in any one of the claims above.

[0014] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium. When the executable instruction is executed on a computer device / equipment, the computer device / equipment executes the operation of the method for opening a non-driver's door as described in any one of the claims above.

[0015] In the embodiment of the present application, when the vehicle is not in driving state and the load signal of the driver's seat is sensed, if the target object is detected within a preset distance from the non-driver's door, the image acquisition device is controlled to collect a video stream; at the same time, the facial features and posture features of the target object are extracted from the video stream, and when the facial features are non-preset features and the posture features indicate that the target object has the intention to open the door, the first inquiry information is output to inquire the driver to determine whether to open the non-driver's door. The double verification of the target object's intention to open the door and the driver's determination to open the door is achieved to determine whether to open the non-driver's door, so that it can more accurately judge whether to open the non-driver's door; at the same time, by improving the accuracy of the door opening judgment, it can avoid the situation where the driver's driving safety hazards caused by the wrong door opening are caused, thereby improving the user experience.

[0016] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0018] Figure 1 A schematic flow chart showing an embodiment of a method for opening a non-driver's door provided by the present application;

[0019] Figure 2 A flow chart showing another embodiment of a method for opening a non-driver's door provided by the present application;

[0020] Figure 3 A flow chart showing another embodiment of a method for opening a non-driver's door provided in the present application;

[0021] Figure 4 A schematic flow chart showing another embodiment of a method for opening a non-driver's door provided in the present application;

[0022] Figure 5 A flow chart showing another embodiment of a method for opening a non-driver's door provided by the present application;

[0023] Figure 6 A schematic structural diagram of an embodiment of a non-driver's door opening device provided by the present application is shown;

[0024] Figure 7 A schematic structural diagram of an embodiment of a computer device provided by the present application is shown.

[0025] Reference numerals:

[0026] 400-opening device of non-driver's door; 410-video acquisition module; 420-control module; 430-door opening module;

[0027] 500-computer system; 501-CPU; 502-ROM; 503-RAM; 504-bus; 505-I / O interface; 506-input part; 507-output part; 508-storage part; 509-communication part; 510-drive; 511-removable medium. DETAILED DESCRIPTION

[0028] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0029] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0031] The term "multiple" as used in this application refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0032] With the development of technology, the intelligence and convenience of vehicles have become important experiences pursued by users. Traditional vehicle doors need to be opened and closed manually, which is cumbersome and not convenient enough in some scenarios. In recent years, some systems that use sensors to automatically open and close doors have appeared on the market, but these systems usually lack interaction with the driving object and cannot accurately judge the intentions of passengers, resulting in misoperation or safety hazards.

[0033] Therefore, there is an urgent need for a control method that can intelligently identify the passenger's intention and safely and automatically open the car door to solve the above technical problems.

[0034] It should be noted that the vehicle of the present application is equipped with an image acquisition device outside the non-driver's door for video shooting and personnel identification.

[0035] Figure 1 A flowchart of an embodiment of a method for opening a non-driver's door of the present application is shown, and the method is executed by a computer device. Figure 1 As shown, the method comprises the following steps:

[0036] Step S110: When the vehicle is not in driving state and a load signal of the driver's seat is sensed, if a target object is detected within a preset distance from the non-driver's door, the image acquisition device is controlled to acquire a video stream.

[0037] The load signal is a signal indicating that the driver's seat carries a driving object, and the target object refers to a person within a preset distance of the non-driver's door.

[0038] Specifically, a sensor is set up within a certain range next to the door to detect whether a person is approaching. When the vehicle is stopped and unlocked, the sensor performs detection work. When the sensor detects that a person is approaching, the image acquisition device outside the vehicle is activated to start working and collect the behavior video of the person after approaching the preset distance.

[0039] Since the application scenario of this application is that there is already a driver in the car, and the driver has parked on the roadside without turning off the engine and locking the car, and needs to wait for passengers to get on the car, for the safety of the driver, the door that can be opened automatically can only be the door of the non-driver's seat. At the same time, the door can only be opened automatically when the vehicle is parked, so the application scenario is limited to "the vehicle is not in driving state" to avoid safety hazards caused by abnormal door opening in driving state.

[0040] Step S120: extracting facial features and posture features of the target object from the video stream. If the facial features are non-preset features and the posture features indicate that the target object has an intention to open the door, outputting a first inquiry message to inquire the driver whether to open the non-driver's door.

[0041] Among them, non-preset features are not preset features. Preset features are a set of facial features of people whose facial features have been stored. If the facial features are non-preset features, it means that the current target object is a person whose facial features have not been stored, such as a stranger. The target object is the person who is detected to have the intention to get on the bus.

[0042] Specifically, the person type of the target object is determined by identifying the human portrait in each image frame of the video stream. The types of people include but are not limited to "relatives", "friends", and "strangers". The relatives and friends here are people whose facial features have been stored, while strangers are people whose facial features have not been stored. Therefore, there are only two types of people, people whose facial features have been stored and people whose facial features have not been stored. At the same time, based on the video stream, combined with big data model analysis (such as facial orientation, walking path or specific actions, etc.), the intention of the person to get on the bus is detected. Only when the person has the intention to get on the bus, it is necessary to make a strategic judgment on whether to open the door to reduce the occurrence of misjudgment.

[0043] Meanwhile, if the target object's facial features are non-preset features, that is, the target object is a person whose facial features have not been stored, and also has the intention to get on the car, it is necessary to send a first inquiry message to ask the driver whether to open the door. After the driver agrees, the target door automatically opens to allow the person to sit down.

[0044] Step S130: Open the non-driver's door according to the instruction to open the non-driver's door transmitted back by the driver in response to the first inquiry information.

[0045] Among them, since the first inquiry information can be sent by voice, can also be displayed on the central control screen, or can be prompted by both voice and the central control screen. The corresponding instruction sent back by the driving subject in response to the first inquiry information can also be prompted by voice and / or clicking on the central control screen. For example, when it is sent by voice, when the driving subject receives the inquiry of the voice assistant, he confirms whether to open the door by answering "yes" or "no"; the voice assistant recognizes the answer of the driving subject, and if the driving subject answers yes, the door control system is started to execute the door opening action.

[0046] Beneficial effect: By double-verifying the target object's door-opening intention and the driver's determination to open the door before opening the door, whether to open the non-driver's door can be determined, thereby achieving a more accurate judgment on whether to open the non-driver's door; at the same time, by improving the accuracy of door opening judgment, it can avoid driving safety hazards caused by incorrect door opening during driving, thereby improving the user experience.

[0047] In some embodiments, the method further includes: if the facial feature is a preset feature, and the posture feature indicates that the target object has an intention to open the door, then opening the non-driver's door.

[0048] The preset features are a set of facial features of people whose facial features have been stored. The facial features being preset features indicate that the current target object is a person whose facial features have been stored, such as a relative, friend, etc.

[0049] Specifically, if the facial features are preset features, that is, the target object is a person whose facial features have been stored and also has the intention to get on the vehicle, the non-driver's door can be opened directly without the driver's consent.

[0050] Beneficial effect: If the target object's facial features are preset features and have the intention to open the door, the non-driver's door is opened directly without the consent of the driver, which refines the door opening scenario and enriches the application scheme. At the same time, by judging that the facial features are preset features, the operation of sending the first inquiry information is reduced, and the efficiency of determining the door opening is improved.

[0051] In some embodiments, after the step of outputting the first inquiry information in step S120 to inquire the driver to determine whether to open the non-driver's door, it also includes: if it is determined to open the non-driver's door, outputting the second inquiry information to inquire the driver to determine whether to store the facial features as preset features.

[0052] The preset features are a set of facial features of people whose facial features have been stored. The facial features are stored as preset features, that is, the current target object is regarded as an object with a storage record. Then, in combination with the previous embodiment, when the target object appears again later, once he intends to get on the car, the door can be opened directly without the consent of the driver.

[0053] Specifically, after the driver agrees, the target door automatically opens to allow the person to sit down. For the person who sits down for the first time, the driver confirms whether to store the portrait information in the corresponding person category, that is, whether to store the facial features of the target object as preset features.

[0054] The second inquiry information is the same as the first inquiry information. It can be issued by voice, or issued by the central control screen, or prompted by voice and the central control screen, prompting the driving subject whether to store his or her portrait information in the corresponding personnel category to simplify the next ride process for the person. If the driving subject answers / clicks "yes", it will be stored; if the driving subject answers "no" / clicks "no", or does not answer / click, it will not be stored. The second inquiry information is preferably issued through the central control screen to avoid attracting the attention of the target object due to voice prompts, which will affect the target object's riding experience. Especially in the scenario where the online car-hailing driver picks up and drops off passengers, the voice prompt will affect the passenger's riding experience, so it is preferably issued through the central control screen.

[0055] Beneficial effect: By issuing a second inquiry message to determine whether to store facial features as preset features, it is possible to further determine whether to store the facial features of the target object based on the interaction with the driving object, thereby enriching the preset features and whether to store the target object to simplify the subsequent judgment process of the target object getting on the car and opening the door, further improving the present application scheme.

[0056] In some embodiments, Figure 2 As shown, after the step of outputting the first inquiry information in step S120 to inquire the driver whether to open the non-driver's door, the method further includes:

[0057] Step S140: If it is determined that the non-driver's door is opened, the last frame image of the video stream is obtained.

[0058] Specifically, if the driver's answer to the first inquiry information is to confirm opening the non-driver's door, it is necessary to obtain the last frame image in the video stream to detect whether the door opening condition is met.

[0059] Step S150: Detect whether there are other objects within a preset distance threshold range of the non-driver's door in the last frame of the image.

[0060] Among them, other objects are objects other than the target object, and the objects here can be various people and objects such as cars, people, obstacles, etc.

[0061] Specifically, the preset distance threshold range is a distance range centered on the non-driver's door. This range is the range that the non-driver's door can radiate when it is subsequently opened. In addition to the collision of conventional static objects during the door opening, this range also takes into account the collision of other objects that change dynamically during the door opening. For example, the risk of collision with the rear vehicle moving forward when the door is opened, or the risk of collision when the door is opened by passers-by.

[0062] Step S160: If yes, a safety warning prompt is issued to indicate that the current non-driver's door does not meet the door opening conditions.

[0063] Specifically, if there are other objects within this range, it means that there may be a risk of collision when opening the door later. Therefore, if there are other objects, the door cannot be opened immediately for safety reasons, and a safety warning prompt needs to be issued to remind that the current non-driver's door does not meet the door opening conditions. The safety warning prompt will also perform differentiated identification and prompts based on the existence of different other objects, and perform different attention warnings. For example, if the other object is a vehicle coming from behind, while prompting that the door opening conditions are not met, the language device set on the current non-driver's door will be used to remind the current target object to pay attention to the vehicle coming from behind and pay attention to avoidance to avoid safety accidents. If the other object is a passerby, while prompting that the door opening conditions are not met, the current target object will be reminded to pay attention to the presence of passersby, and the door will be opened when it is safe later.

[0064] Step S170: If not present, open the non-driver's door.

[0065] Specifically, if in the last frame of the image, there is only the target object within the preset distance threshold range of the non-driver's door, and no other objects exist, it indicates that the non-driver's door meets the door opening condition at this time, so the corresponding non-driver's door is opened. It should be noted that step S160 and step S170 are performed simultaneously, and there is no order relationship.

[0066] Beneficial effect: By acquiring the last frame of the video stream, it is detected whether there are other objects within the preset distance threshold range of the non-driver's door to determine whether the door opening conditions are met. Thus, the door opening condition detection is realized before the door is opened, and safety accidents such as collision caused by opening the door when there are other objects are avoided, thereby improving the safety of door opening.

[0067] In some embodiments, Figure 3 As shown, the method also includes:

[0068] Step S210: Filtering out target image frames that meet preset conditions from the video stream.

[0069] Specifically, the preset condition includes that the facial orientation of the target object is a preset orientation. That is, all image frames in which the facial orientation of the target object in the video stream is a preset orientation are extracted, and the image frames that are continuous in time in the extracted images are used as target image frames. The target image frames need to be continuous, because the essence of analyzing the target object's intention to open the door is to analyze the target object's movements and facial orientation. It is generally believed that walking straight towards the car door is the most obvious intention to open the door. Therefore, to analyze the target object's intention to open the door, it is necessary to analyze its movements and facial orientation. A single image frame is a static image, and the target object cannot form a dynamic action, so the dynamic action displayed in the image frame is continuous and gradually changing. Therefore, the target image frame also needs to be continuous in time to analyze the corresponding dynamic action.

[0070] Step S220: extracting the posture features of the target object from the target image frame as target posture features.

[0071] Specifically, from the image frame in which the face of the target object is oriented in a preset orientation, the posture feature of the target object is extracted as the target posture feature. The target posture feature is used to analyze the behavior of the target object.

[0072] Step S230: Determine various behavior actions of the target object according to the target posture features.

[0073] Specifically, by combining and splicing according to the target posture features, various behavioral actions performed by the target object can be determined.

[0074] Step S240: Match each behavior action with a preset action set.

[0075] Among them, the preset action set includes multiple actions with the intention of opening the door. The preset action set includes two types of preset actions, one of which is the preset door opening action that most people use by default. Common ones are walking straight toward the car door, waving near the front of the car, or pulling the car door. The other type is the preset door opening trigger action. For example, waving up and down at the car door twice is preset as the preset door opening trigger action. Waving up and down twice here means waving up and down twice in a row. As long as it is detected that the target object is facing the car door and waving up and down at the car door twice, it is considered that the target object has the intention to open the door. The preset door opening trigger action can be customized by the user to improve the user experience.

[0076] Specifically, each determined behavior action is matched with a preset action set one by one to determine whether the target object has performed an action with the intention of opening the door. If the behavior action successfully matches any preset action in the preset action set, it means that the target object has performed an action with the intention of opening the door.

[0077] Step S250: If at least one behavior action is successfully matched, it is determined that the target object has the intention to open the door.

[0078] Specifically, as long as one behavioral action is matched successfully, it means that the target object has performed an action with the intention of opening the door, that is, the target object has the intention of opening the door.

[0079] Beneficial effect: The solution of how to determine whether the target object has the intention to open the door based on posture features is further refined, thereby improving and enriching the content of this application.

[0080] In some embodiments, Figure 4 As shown, the preset condition includes that the face orientation of the target object is the preset orientation. Step S210 further includes:

[0081] Step S211: Filter out image frames in which the face orientation of the target object is a preset orientation from the video stream as image frames to be selected.

[0082] Specifically, firstly, a plurality of continuously changing image frames are extracted from the video stream, and among the plurality of continuously changing image frames, all image frames in which the face of the target object is oriented in a preset orientation are extracted as the image frames to be selected.

[0083] Step S212: Filter out image frames with continuous acquisition time from the candidate image frames as target image frames.

[0084] Specifically, because the analysis of the target object's intention to open the door is essentially to analyze the target object's movements and facial orientation. However, a single image frame is a static image, and the target object therein cannot form a dynamic movement, so the movement displayed in the image frame is continuous and gradually changes. Therefore, the target image frame also needs to be continuous in time to analyze the movement. Therefore, the target image frames with continuous acquisition time are screened out from the candidate image frames. It should be noted that multiple target image frames can be collected at the same time, and each target image frame is composed of several candidate image frames that are continuous in time.

[0085] Beneficial effects: The scheme for screening out target image frames that meet preset conditions from the video stream is further refined. The target image frames can be quickly determined through two screenings, thereby improving the implementation efficiency of the method.

[0086] In some embodiments, Figure 5 As shown, the method also includes:

[0087] Step S310: Determine various behavior actions of the target object according to the posture features.

[0088] Specifically, by combining and splicing according to the target posture features, various behavioral actions performed by the target object can be determined.

[0089] Step S320: Match each behavior action with a preset action set, and use the successfully matched behavior action as the target behavior action.

[0090] Among them, the preset action set includes multiple actions with the intention of opening the door. The preset action set includes two types of preset actions, one of which is the preset door opening action that most people use by default. Common ones are walking straight toward the car door, waving near the front of the car, or pulling the car door. The other type is the preset door opening trigger action. For example, waving up and down at the car door twice is preset as the preset door opening trigger action. Waving up and down twice here means waving up and down twice in a row. As long as it is detected that the target object is facing the car door and waving up and down at the car door twice, it is considered that the target object has the intention to open the door. The preset door opening trigger action can be customized by the user to improve the user experience.

[0091] Specifically, each determined action is matched with the preset action set one by one to determine whether the target object has performed an action with the intention of opening the door. If the action successfully matches any preset action in the preset action set, the successfully matched action is used as the target action.

[0092] Step S330: Detect the facial orientation of the target object corresponding to each target behavior action.

[0093] Specifically, the facial orientation of the target object corresponding to each target behavior action is detected to determine whether the target object actually has the intention to open the door when performing the target behavior action, so as to avoid recognition errors.

[0094] Step S340: If there is at least one facial orientation that is the target orientation, it is determined that the target object has the intention to open the door.

[0095] Specifically, if there is at least one facial orientation that is the target orientation, it means that the target object really has the intention to open the door when performing the target behavior action, and it can be determined that the target object has the intention to open the door.

[0096] Beneficial effect: By first identifying the action, then matching it with the preset action set, we can get the target action that meets the intention of opening the door, and finally combine the facial orientation corresponding to the target action to verify whether the target object really has the intention to open the door when performing the target action. This can realize the verification of facial orientation while quickly determining the action, thus improving the accuracy of the recognition of the intention of opening the door.

[0097] It should be noted that the method provided in this embodiment is compared with the previous embodiment of first screening the orientation and then determining the action. Although both methods determine the facial orientation and action, the data processing amount is different. For different application scenarios, the data processing amount will be different. When the video stream includes relatively few image frames, it is preferred to first determine the action and then determine the orientation, which can be more accurate and improve the accuracy of data processing. When the video stream includes relatively few image frames, it is preferred to first screen and then determine the action, which can reduce the amount of action determination, thereby reducing the amount of data processing and improving data processing efficiency.

[0098] The present invention is applicable to various passenger cars, commercial vehicles and other vehicles that have door switch requirements. With the continuous development of intelligent technology and the continuous growth of market demand, the present invention is expected to be widely used in the automotive industry, bringing users a more convenient and safe travel experience.

[0099] Figure 6 The schematic diagram of the structure of the embodiment of the opening device of the non-driver's door of the present application is shown. Figure 6 As shown, the device 400 includes: a video acquisition module 410 , a control module 420 and a door opening module 430 .

[0100] The video acquisition module 410 is used to control the image acquisition device to acquire a video stream when the vehicle is in a non-driving state and a load signal of the driver's seat is sensed, if a target object is detected within a preset distance from the non-driver's door; wherein the load signal is a signal indicating that the driver's seat carries a driving object;

[0101] The control module 420 is used to extract the facial features and posture features of the target object from the video stream, and if the facial features are non-preset features and the posture features indicate that the target object has an intention to open the door, output a first inquiry message to inquire the driver to determine whether to open the non-driver's door;

[0102] The door opening module 430 is used to open the non-driver's door according to the instruction to open the non-driver's door transmitted back by the driver in response to the first inquiry information.

[0103] Beneficial effect: When the vehicle is not in a driving state and the load signal of the driver's seat is sensed, if the target object is detected within a preset distance from the non-driver's door, the image acquisition device is controlled to collect a video stream; at the same time, the facial features and posture features of the target object are extracted from the video stream. When the facial features are non-preset features and the posture features indicate that the target object has the intention to open the door, the first inquiry information is output to inquire the driver to determine whether to open the non-driver's door. The double verification of the target object's intention to open the door and the driver's determination to open the door is achieved to determine whether to open the non-driver's door, so that it can more accurately judge whether to open the non-driver's door; at the same time, by improving the accuracy of the door opening judgment, it can avoid the situation where the driver's driving safety hazards caused by the wrong door opening are caused, thereby improving the user experience.

[0104] It should be noted that the non-driver's door opening device provided in the above embodiment and the non-driver's door opening method provided in the above embodiment belong to the same concept, and the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0105] Figure 7 A structural schematic diagram of an embodiment of a computer device of the present application is shown, which shows a structural schematic diagram of a computer system of a computer device suitable for implementing an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the computer device.

[0106] See also Figure 7 As shown, the computer device includes: a controller; a memory for storing one or more programs, and when the one or more programs are executed by the controller, the above-mentioned method for opening the non-driver's door is executed.

[0107] Please continue reading Figure 7As shown, the computer system 500 of the computer device includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method in the above embodiment. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, ROM 502 and RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0108] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0109] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the system of the present application are executed.

[0110] Another aspect of the present application further provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on a computer device / equipment, the computer device / equipment executes the method for opening a non-driver's door in any of the above embodiments.

[0111] Beneficial effect: When the vehicle is not in a driving state and the load signal of the driver's seat is sensed, if the target object is detected within a preset distance from the non-driver's door, the image acquisition device is controlled to collect a video stream; at the same time, the facial features and posture features of the target object are extracted from the video stream. When the facial features are non-preset features and the posture features indicate that the target object has the intention to open the door, the first inquiry information is output to inquire the driver to determine whether to open the non-driver's door. The double verification of the target object's intention to open the door and the driver's determination to open the door is achieved to determine whether to open the non-driver's door, so that it can more accurately judge whether to open the non-driver's door; at the same time, by improving the accuracy of the door opening judgment, it can avoid the situation where the driver's driving safety hazards caused by the wrong door opening are caused, thereby improving the user experience.

[0112] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0113] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0114] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0115] According to one aspect of an embodiment of the present application, a computer system is also provided, including a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM), such as executing the method in the above embodiment. In RAM, various programs and data required for system operation are also stored. CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0116] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0117] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

[0118] The collection and processing of relevant data in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject (or with a legal basis) should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0119] In this application, the face (or other biometric) recognition technology involved, when the above embodiments of this application are applied to specific products or technologies, the relevant data collection, use and processing processes should comply with national laws and regulations. Before collecting facial information, the information processing rules should be informed and the target object’s separate consent (or legal basis) should be sought, and facial information should be processed in strict accordance with the requirements of laws and regulations and personal information processing rules, and technical measures should be taken to ensure the security of relevant data.

Claims

1. A method for opening a non-driver's door, wherein an image acquisition device is arranged outside the non-driver's door, characterized in that: The method comprises: When the vehicle is in a non-driving state and a load signal of the driver's seat is sensed, if a target object is detected within a preset distance from the non-driver's door, the image acquisition device is controlled to acquire a video stream; wherein the load signal is a signal indicating that the driver's seat carries a driving object; Extracting facial features and posture features of the target object from the video stream, and if the facial features are non-preset features and the posture features indicate that the target object has an intention to open the door, outputting first inquiry information to inquire the driver whether to open the non-driver door; The non-driver's door is opened according to an instruction to open the non-driver's door transmitted back by the driving object in response to the first inquiry information.

2. The method according to claim 1, characterized in that The method further comprises: If the facial feature is a preset feature, and the posture feature indicates that the target object has an intention to open the door, then the non-driver's door is opened.

3. The method according to claim 1, characterized in that After the step of outputting the first inquiry information to inquire the driver whether to open the non-driver door, the method further includes: If it is determined that the non-driver's door is opened, acquiring the last frame image of the video stream; Detecting whether there are other objects within a preset distance threshold range of the non-driver's door in the last frame of image; wherein the other objects are objects other than the target object; If yes, a safety warning prompt is issued to indicate that the non-driver's door does not meet the door opening condition; If not present, open the non-driver's door.

4. The method according to claim 3, characterized in that After the step of outputting the first inquiry information to inquire the driver whether to open the non-driver door, the method further includes: If it is determined to open the non-driver's door, a second inquiry message is output to inquire the driver whether to store the facial feature as a preset feature.

5. The method according to claim 1, characterized in that: The method further comprises: Filtering target image frames that meet preset conditions from the video stream; Extracting the posture feature of the target object from the target image frame as the target posture feature; Determining various behavior actions of the target object according to the target posture features; Matching each of the behavioral actions with a preset action set respectively; wherein the preset action set includes a plurality of actions with the intention of opening the door; If at least one of the behavioral actions matches successfully, it is determined that the target object has the intention to open the door.

6. The method according to claim 5, characterized in that The preset condition includes that the facial orientation of the target object is a preset orientation; The step of selecting target image frames that meet preset conditions from the video stream further comprises: Filtering out image frames in which the face orientation of the target object is a preset orientation from the video stream as to-be-selected image frames; Image frames with continuous acquisition time are selected from the candidate image frames as the target image frames.

7. The method according to claim 1, characterized in that The method further comprises: Determining various behavior actions of the target object according to the posture features; Matching each of the above mentioned actions with a preset action set respectively, and taking the successfully matched actions as target actions; wherein the preset action set includes a plurality of actions with the intention of opening the door; Detecting the facial orientation of the target object corresponding to each of the target behavior actions; If at least one of the facial orientations is the target orientation, it is determined that the target object has the intention to open the door.

8. A device for opening a non-driver's door, wherein an image acquisition device is arranged outside the non-driver's door, characterized in that: The device comprises: A video acquisition module, for controlling the image acquisition device to acquire a video stream when the vehicle is in a non-driving state and a load signal of the driver's seat is sensed, if a target object is detected within a preset distance from the non-driver's door; wherein the load signal is a signal indicating that the driver's seat carries a driving object; A control module, configured to extract facial features and posture features of the target object from the video stream, and if the facial features are non-preset features and the posture features indicate that the target object has an intention to open the door, output a first inquiry message to inquire the driver whether to open the non-driver's door; The door opening module is used to open the non-driver's door according to the instruction to open the non-driver's door transmitted back by the driving object in response to the first inquiry information.

9. A computer device, characterized in that: include: Controller; A memory is used to store one or more programs. When the one or more programs are executed by the controller, the controller implements the method for opening the non-driver's door as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction, and when the executable instruction is executed on a computer device / equipment, the computer device / equipment executes the operation of the method for opening a non-driver's door as described in any one of claims 1 to 7.

Images