Vehicle start-stop control method and device, storage medium, equipment and vehicle

Abstract

The invention provides a vehicle start-stop control method and device, a storage medium, equipment and a vehicle, and relates to the technical field of unmanned driving. The method comprises the steps that in response to a vehicle start-stop control instruction, passenger motion state data and sitting posture stable state data of each passenger in a to-be-controlled vehicle are acquired in real time; according to the passenger motion state data, posture recognition is conducted on each passenger in the to-be-controlled vehicle, and a passenger posture recognition result is obtained; based on the passenger posture recognition result, performing action prediction on each passenger in the to-be-controlled vehicle to obtain a passenger action prediction result; and controlling the speed of the to-be-controlled vehicle based on the passenger action prediction result and the sitting posture stable state data so as to perform start-stop control on the to-be-controlled vehicle. According to the system and the method, the posture of the passenger is analyzed in real time and the instant response is made according to the change, so that the delay possibly occurring during manual operation is reduced, the response speed and the start-stop efficiency of the vehicle are improved, and the automatic start-stop control of the vehicle is realized.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a vehicle start-stop control method, device, storage medium, equipment and vehicle. Background Art

[0002] In most traditional sightseeing car systems, the start-stop control is manually operated by the driver or the staff in the car, usually by buttons, remote controls or in-car operation panels to start or stop the vehicle. In the early applications of driverless sightseeing cars, manual start-stop systems are still used in some relatively simple environments, such as scenic parks, museums, zoos, etc. These environments usually require the vehicle to stop at different stations to facilitate passengers to get on and off.

[0003] For operators, the manual start-stop method is relatively simple. It does not require complex algorithms and sensors to monitor the status of passengers. It only needs to visually judge the passengers' boarding and disembarking conditions and control the vehicle's start and stop as needed. However, the manual start-stop system relies on the operator's reaction speed. Usually when passengers stand up to get off or get on the bus, the operator needs to control the vehicle's start and stop through manual observation and manual operation. This manual response may cause delays, especially when passengers change frequently or the vehicle speed is high, it is difficult to respond in time, which may lead to safety hazards. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a vehicle start-stop control method, device, storage medium, equipment and vehicle to solve the above technical problems.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: a vehicle start-stop control method, comprising: in response to a vehicle start-stop control instruction, obtaining in real time the passenger motion state data and sitting posture stability state data of each passenger in the vehicle to be controlled; based on the passenger motion state data, performing posture recognition on each passenger in the vehicle to be controlled to obtain a passenger posture recognition result; based on the passenger posture recognition result, performing action prediction on each passenger in the vehicle to be controlled to obtain a passenger action prediction result, and the passenger action prediction result is used to characterize the action intention of each passenger in the vehicle to be controlled at the next time step corresponding to the current moment; based on the passenger action prediction result and the sitting posture stability state data, controlling the speed of the vehicle to be controlled to start and stop the vehicle to be controlled.

[0006] The beneficial effects of the present invention are as follows: the present invention performs posture recognition on each passenger in the vehicle to be controlled by acquiring the passenger motion state data corresponding to each passenger, and predicts the action of each passenger based on the posture recognition result. Finally, based on the passenger action prediction result and the acquired sitting posture stability state data, the start and stop of the vehicle are controlled. Through the present invention, the passenger posture can be analyzed in real time and reacted to according to the changes, the automatic start and stop control of the vehicle is realized, and the vehicle response speed and start and stop efficiency are improved.

[0007] Based on the above technical solution, the present invention can also be improved as follows.

[0008] Furthermore, the posture recognition of each passenger in the vehicle to be controlled is performed based on the passenger motion state data to obtain a passenger posture recognition result, including: obtaining a three-dimensional posture graph corresponding to each passenger in the vehicle to be controlled based on the passenger motion state data; and inputting the three-dimensional posture graph corresponding to each passenger in the vehicle to be controlled into a trained action classification model to obtain the passenger posture recognition result, wherein the passenger posture recognition result includes the action category corresponding to each passenger in the vehicle to be controlled at the current moment.

[0009] Furthermore, based on the passenger posture recognition result, the action of each passenger in the vehicle to be controlled is predicted to obtain the passenger action prediction result, including: when the passenger posture recognition result includes a preset target action category, according to the passenger motion state data, a three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled is obtained, and the three-dimensional posture graph sequence includes multiple frames of three-dimensional posture graphs corresponding to the passenger; the three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled is input into a trained action sequence analysis model respectively to obtain the action intention category result output by the action sequence analysis model; according to the action intention category result, the passenger action prediction result is determined.

[0010] Furthermore, based on the passenger action prediction result and the sitting posture stability state data, the speed of the vehicle to be controlled is controlled to start and stop the vehicle to be controlled, including: acquiring the vehicle operating status of the vehicle to be controlled in real time, the vehicle operating status being a waiting-to-start state or a driving state; based on the vehicle operating status, determining a vehicle control strategy according to the passenger action prediction result and the sitting posture stability state data; and controlling the speed of the vehicle to be controlled according to the vehicle control strategy to start and stop the vehicle to be controlled.

[0011] Furthermore, based on the vehicle running state, the vehicle control strategy is determined according to the passenger action prediction result and the sitting posture stability state data, including: when the vehicle running state is a waiting-to-start state, determining whether the action intention corresponding to each passenger in the vehicle to be controlled is to ride the vehicle, and whether the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets the preset sitting posture stability condition; when the action intention corresponding to each passenger in the vehicle to be controlled is to ride the vehicle, and the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets the preset sitting posture stability condition, the vehicle control strategy includes controlling the acceleration of the vehicle to be controlled, so as to start the vehicle to be controlled according to the vehicle control strategy.

[0012] Furthermore, based on the vehicle running state, the vehicle control strategy is determined according to the passenger action prediction result and the sitting posture stability state data, including: when the vehicle running state is the driving state, determining whether there are one or more passengers in the vehicle to be controlled whose corresponding action intention is to get off the vehicle, and whether the sitting posture stability state data corresponding to the one or more passengers meets the preset leaving seat condition; when there are one or more passengers in the vehicle to be controlled whose corresponding action intention is to get off the vehicle, and the sitting posture stability state data corresponding to the one or more passengers meets the preset leaving seat condition, the vehicle control strategy includes controlling the vehicle to be controlled to decelerate, determining the parking position and parking time, so as to perform parking control on the vehicle to be controlled according to the vehicle control strategy.

[0013] In order to solve the above technical problems, the present invention further proposes a vehicle start-stop control device, comprising:

[0014] A data acquisition module, for acquiring in real time passenger motion state data and sitting posture stability state data of each passenger in the vehicle to be controlled in response to a vehicle start-stop control instruction;

[0015] A posture recognition module, used to perform posture recognition on each passenger in the vehicle to be controlled according to the passenger motion state data, and obtain a passenger posture recognition result;

[0016] an action prediction module, used to predict the action of each passenger in the vehicle to be controlled based on the passenger posture recognition result, and obtain a passenger action prediction result, wherein the passenger action prediction result is used to characterize the action intention of each passenger in the vehicle to be controlled at the next time step at the current moment;

[0017] The vehicle control module is used to control the speed of the vehicle to be controlled based on the passenger action prediction result and the sitting posture stability state data, so as to start and stop the vehicle to be controlled.

[0018] In order to solve the above technical problems, the present invention also proposes a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the vehicle start-stop control method as described above.

[0019] In order to solve the above technical problems, the present invention also proposes an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the vehicle start-stop control method as described above when executing the computer program.

[0020] In order to solve the above technical problems, the present invention also proposes a vehicle, comprising the vehicle start-stop control device as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flow chart of a vehicle start-stop control method according to the present invention;

[0022] Figure 2 Schematic diagram of a vehicle start-stop control device according to the present invention. DETAILED DESCRIPTION

[0023] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0024] With the development of driverless technology, manual start-stop systems are gradually becoming incompatible with large-scale and efficient operation needs. In related technologies, the application scenarios of driverless sightseeing vehicles have gradually expanded from a single park to a wider range of scenarios, such as large-scale tourist pick-ups and transportation between multiple parking sites, and manual start-stop systems are not efficient enough.

[0025] As mentioned above, the manual start-stop system commonly used in the prior art has the following disadvantages:

[0026] 1. In high-frequency start-stop operations, it is difficult for operators to maintain concentration for a long time, which easily leads to fatigue, thus affecting work efficiency and safety. Especially during peak hours or long-term operation, the safety and accuracy of manual operation will be reduced.

[0027] 2. Prolonged manual operation may lead to driver fatigue, especially in busy stations or scenic spots. Due to fatigue, the driver may not be able to monitor the actions of each passenger in real time, causing delays in vehicle starting or stopping, thus affecting the passenger experience.

[0028] 3. Each driverless sightseeing vehicle needs to be equipped with a driver or operator who is responsible for starting, accelerating, decelerating and parking the vehicle. For large-scale applications, the cost of hiring and training a large number of operators will increase significantly, especially in scenarios such as scenic spots, airports, and parks where multiple sightseeing vehicles are required.

[0029] 4. When the driver is tired, distracted or busy with other matters, he may misjudge the passenger's actions, resulting in incorrect operation, such as starting or stopping the vehicle too early or too late, affecting the safety and comfort of the passengers.

[0030] 5. The driver may not be able to fully observe all passengers due to the layout or angle limitations of the vehicle, especially when there are multiple passengers in the vehicle or the seat layout is complex. There are blind spots in the field of vision, resulting in missing the opportunity for passengers to stand up or sit down.

[0031] In view of this, the present embodiment provides a vehicle start-stop control method, device, storage medium, equipment and vehicle to solve the above technical problems.

[0032] Embodiment 1

[0033] like Figure 1 As shown, this embodiment provides a vehicle start-stop control method, including:

[0034] S101. In response to a vehicle start-stop control instruction, real-time acquisition of passenger motion state data and sitting posture stability state data of each passenger in the vehicle to be controlled.

[0035] The vehicle start-stop control instruction refers to the instruction issued by the vehicle to be controlled, which is used to control the vehicle to be controlled to enter the automatic start-stop control condition. It can be that the vehicle to be controlled enters the automatic start-stop control condition after entering a stable and safe driving state, and the vehicle start-stop control instruction is issued to the start-stop control of the vehicle to be controlled; it can also be that according to the scene information in the area where the vehicle to be controlled is located, the automatic start-stop control condition is entered in a preset or real-time determined, safe section where the automatic start-stop control can be entered, and the vehicle start-stop control instruction is issued to the start-stop control of the vehicle to be controlled.

[0036] The passenger's passenger motion state data and sitting posture stability state data are both dynamic data acquired in real time. In this embodiment, the passenger motion state data is a depth image captured by a depth camera. A depth camera is installed in the vehicle to be controlled, and the motion state of the passengers in the vehicle is captured using the depth camera, such as a kinect camera. The sitting posture stability state data is a pressure value. A pressure sensor is installed in each seat of the vehicle to be controlled, and the pressure value is used to reflect whether the passenger has sat firmly.

[0037] S102: performing posture recognition on each passenger in the vehicle to be controlled according to the passenger motion state data to obtain a passenger posture recognition result.

[0038] Optionally, in an embodiment, the posture recognition of each passenger in the vehicle to be controlled is performed based on the passenger motion state data to obtain a passenger posture recognition result, including: obtaining a three-dimensional posture map corresponding to each passenger in the vehicle to be controlled based on the passenger motion state data; inputting the three-dimensional posture map corresponding to each passenger in the vehicle to be controlled into a trained motion classification model respectively to obtain the passenger posture recognition result, and the passenger posture recognition result includes the action category corresponding to each passenger in the vehicle to be controlled at the current moment.

[0039] The depth camera can capture the depth information in the three-dimensional space, and a three-dimensional posture graph can be generated by combining this information with the posture estimation algorithm. Based on the existing image processing technology, a three-dimensional posture graph containing all people can be generated, or a separate three-dimensional posture graph can be generated for each person. In this embodiment, it is necessary to generate a three-dimensional posture graph corresponding to each passenger for a frame of depth image taken by the depth camera. The three-dimensional posture graph consists of a series of key points (such as shoulders, elbows, wrists, hips, knees and ankles, etc.) and lines connecting these points, which is an image that characterizes the corresponding postures of the key points of the human skeleton.

[0040] Generating a three-dimensional posture graph based on a depth image can also be achieved by constructing a posture detection model. Specifically, first collect training data, which is a multi-frame depth image. The training data can be obtained by simulating passenger behavior in an experimental environment and taking photos, or by using public posture detection data sets, such as COCOKeypoints, MPI IDataset, etc. Depth images under multiple angles and multiple lighting conditions are collected to form training data. For each image, human key points are annotated, including the head, shoulders, elbows, knees, etc. The annotated training data and the coordinates of the annotated key points are input into a convolutional neural network (CNN) for training to obtain a posture detection model. The trained posture detection model can extract image features (shape, position, motion state, and behavior pattern of the human body, etc.) based on the depth image, perform human key point detection on the feature layer, output the coordinates of each key point in the image, and form a three-dimensional posture graph.

[0041] For the training of the action classification model: First, collect data, and use a depth camera to collect images of passenger posture behaviors simulated by the human body in an experimental environment, including images under multiple angles and multiple lighting conditions. The training data for posture detection model training mentioned above can also be used. Output the coordinates of each key point in each image to form a three-dimensional posture graph for model training. And annotate each image to mark the action category corresponding to the human body in each image. Train the convolutional neural network with the annotated images to obtain the action classification model. Input each image into the CNN feature extractor, CNN extracts the spatial features of the posture skeleton, and the fully connected layer outputs the action category.

[0042] Among them, the action category representation is used to classify the categories of passengers' posture actions, and the action categories include sitting down action, maintaining a sitting posture, standing up action, leaning action, maintaining standing, etc., which can be set specifically according to actual conditions.

[0043] S103. Based on the passenger posture recognition result, predict the action of each passenger in the vehicle to be controlled to obtain a passenger action prediction result, wherein the passenger action prediction result is used to characterize the action intention of each passenger in the vehicle to be controlled at the next time step corresponding to the current moment.

[0044] Optionally, in an embodiment, based on the passenger posture recognition result, the action of each passenger in the vehicle to be controlled is predicted to obtain a passenger action prediction result, including: when the passenger posture recognition result includes a preset target action category, according to the passenger motion state data, obtaining a three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled, the three-dimensional posture graph sequence including multiple frames of three-dimensional posture graphs corresponding to the passenger; inputting the three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled into a trained action sequence analysis model respectively to obtain the action intention category result output by the action sequence analysis model; and determining the passenger action prediction result according to the action intention category result.

[0045] In this embodiment, the preset target action categories include sitting down action, sitting posture maintenance and standing up action. The sitting down action represents the action of the passenger changing from standing to sitting, the sitting posture maintenance represents the passenger keeping the sitting posture unchanged, and the standing up action represents the action of the passenger changing from sitting to standing.

[0046] According to the real-time acquired passenger motion state data, multiple frames of continuous depth images can be obtained. In the above manner, a three-dimensional posture map of the passenger is generated according to the depth image. Therefore, a three-dimensional posture map sequence can be obtained through multiple frames of continuous depth images.

[0047] The action intention category results include the action intention category corresponding to each passenger in the vehicle to be controlled. The action intention categories include staying seated, walking toward the door, walking toward other seats, and staying standing, etc., which can be set according to the actual situation. According to the action intention category of each passenger, the action intention corresponding to each passenger is determined. The action intentions include getting on the vehicle, getting off the vehicle, changing seats, etc. When the action intention category is staying seated, the corresponding action intention is getting on the vehicle. When the action intention category is walking toward the door, the corresponding action intention is getting off the vehicle, which can be adjusted according to the actual situation.

[0048] Specifically, the action sequence analysis model is composed of CNN and Long Short-Term Memory (LSTM). For the training of the action sequence analysis model: collect videos of simulated passenger posture behaviors, generate three-dimensional posture graphs corresponding to each frame image in each video, and obtain a three-dimensional posture graph sequence for model training. And annotate each three-dimensional posture graph sequence, annotate the action intention category corresponding to the human body in each three-dimensional posture graph sequence. Perform model training through the annotated three-dimensional posture graph sequence to obtain the action sequence analysis model.

[0049] After the 3D posture graph sequence is input into the action sequence analysis model, CNN extracts the features of each frame (using the convolution kernel to slide on the image to extract the features of the local area, such as edges, corners, contours, etc.). LSTM captures the timing information between frames (controls the flow of information through three gates, namely the input gate, the forget gate, and the output gate). LSTM accumulates the dynamic change information between frames and generates a global representation of the time series. For example, when a passenger stands up, the key points of the legs move up frame by frame. LSTM will capture this trend and encode it into a hidden state. Finally, the classification head is connected to output the predicted action intention category.

[0050] S104: Based on the passenger action prediction result and the sitting posture stability state data, the speed of the vehicle to be controlled is controlled to perform start-stop control on the vehicle to be controlled.

[0051] Optionally, in an embodiment, the speed of the vehicle to be controlled is controlled based on the passenger action prediction result and the sitting posture stability state data to start and stop the vehicle to be controlled, including: acquiring the vehicle operating status of the vehicle to be controlled in real time, the vehicle operating status being a waiting-to-start state or a driving state; determining a vehicle control strategy based on the vehicle operating status according to the passenger action prediction result and the sitting posture stability state data; controlling the speed of the vehicle to be controlled according to the vehicle control strategy to start and stop the vehicle to be controlled.

[0052] Specifically, the waiting-to-start state represents that the vehicle to be controlled is in a state of waiting to be started and departed, and the driving state represents that the vehicle to be controlled is in a state of driving.

[0053] Optionally, in an embodiment, the vehicle control strategy is determined based on the vehicle running state and according to the passenger action prediction results and the sitting posture stability state data, including: when the vehicle running state is a waiting-to-start state, determining whether the action intention corresponding to each passenger in the vehicle to be controlled is to ride the vehicle, and whether the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets the preset sitting posture stability conditions; when the action intention corresponding to each passenger in the vehicle to be controlled is to ride the vehicle, and the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets the preset sitting posture stability conditions, the vehicle control strategy includes controlling the acceleration of the vehicle to be controlled so as to start the vehicle to be controlled according to the vehicle control strategy.

[0054] When the vehicle to be controlled is in the waiting state, a start judgment needs to be made. The vehicle can be controlled to start only when all passengers in the vehicle are firmly seated. Whether the passengers are firmly seated needs to be judged in combination with the passenger's action intention and the sitting posture stability state data.

[0055] The preset sitting posture stability condition may be a pressure threshold. When the sitting posture stability state data corresponding to the passenger, that is, the pressure value corresponding to the passenger, is greater than or equal to the pressure threshold, it is determined that the sitting posture stability state data corresponding to the passenger meets the preset sitting posture stability condition; otherwise, it is determined that the sitting posture stability state data corresponding to the passenger does not meet the preset sitting posture stability condition. The pressure threshold is a preset value, which can be set based on experience, experimental data, etc.

[0056] Therefore, the combination of action intention and pressure value can determine whether the passenger is sitting down and maintaining a stable sitting posture. Only when all passengers in the car are firmly seated can the vehicle be controlled to start and smoothly accelerate according to the road conditions (such as whether there are obstacles in front of the vehicle, the distance to the vehicle in front, etc.); otherwise, the vehicle is controlled to remain stationary and continue to determine whether the passengers are firmly seated.

[0057] Optionally, in an embodiment, the vehicle control strategy is determined based on the vehicle operating state and according to the passenger action prediction result and the sitting posture stability state data, including: when the vehicle operating state is the driving state, determining whether there are one or more passengers in the vehicle to be controlled whose corresponding action intention is to get off the vehicle, and whether the sitting posture stability state data corresponding to the one or more passengers meet the preset leaving seat conditions; when there are one or more passengers in the vehicle to be controlled whose corresponding action intention is to get off the vehicle, and the sitting posture stability state data corresponding to the one or more passengers meet the preset leaving seat conditions, the vehicle control strategy includes controlling the vehicle to be controlled to decelerate, determining the parking position and parking time, so as to perform parking control on the vehicle to be controlled according to the vehicle control strategy.

[0058] When the vehicle to be controlled is in a driving state, a parking determination is required. When there are passengers who want to get off in the vehicle, the vehicle can be controlled to decelerate until it stops in the target parking area so that the passengers who need to get off can get off. When there are passengers in the vehicle whose corresponding action intention is to get off and whose corresponding sitting posture stability state data meets the preset conditions for leaving the seat, the vehicle to be controlled to stop is controlled to stop in the target parking area. In addition, the appropriate vehicle deceleration can be calculated based on the parking area sign and the current vehicle speed to control the degree of braking. During the parking process, the system reaction time and the additional time required for passenger behavior need to be considered to ensure that the vehicle can stop safely within the parking range.

[0059] The preset leaving seat condition may be a pressure threshold. When the stable sitting state data corresponding to the passenger, that is, the pressure value corresponding to the passenger, is less than the pressure threshold, it is determined that the stable sitting state data corresponding to the passenger meets the preset leaving seat condition; otherwise, it is determined that the stable sitting state data corresponding to the passenger does not meet the preset leaving seat condition. Generally, the pressure threshold of the leaving seat condition here is the same as the pressure threshold of the above-mentioned stable sitting state condition.

[0060] A depth camera and a pressure sensor are installed in the vehicle, and an embedded computing platform is installed. The image data collected by the depth camera and the pressure value collected by the pressure sensor are processed in real time by the embedded computing platform, and the starting, acceleration, deceleration or parking of the vehicle is controlled by the above method.

[0061] Through this method, the automatic start-stop control of the vehicle is realized, and the passenger posture can be analyzed in real time and responded to immediately according to the changes, which reduces the delay that may occur during manual operation and improves the vehicle response speed and start-stop efficiency. The vehicle start and stop can be accurately controlled according to the passenger's action intention and sitting posture stability state data to avoid discomfort to passengers caused by sudden start or stop. In addition, the camera and sensor can fully scan all passengers in the car without being restricted by the field of view, and there will be no blind spots that the driver cannot observe. This method is particularly important for the large-scale application of sightseeing vehicles in high-traffic areas such as popular attractions and airports. It can continuously provide consistent start-stop services and ensure operational efficiency. Even in large-scale applications where multiple vehicles are running at the same time, there is no need to increase staffing, saving a lot of human resources and related expenses. A one-time investment in automation equipment can meet the control needs of large quantities of sightseeing vehicles.

[0062] Embodiment 2

[0063] like Figure 2 As shown, this embodiment provides a vehicle start-stop control device 200, including:

[0064] The data acquisition module 201 is used to obtain the passenger motion state data and sitting posture stability state data of each passenger in the vehicle to be controlled in real time in response to the vehicle start-stop control command;

[0065] A posture recognition module 202, configured to perform posture recognition on each passenger in the vehicle to be controlled according to the passenger motion state data, and obtain a passenger posture recognition result;

[0066] The action prediction module 203 is used to predict the action of each passenger in the vehicle to be controlled based on the passenger posture recognition result to obtain a passenger action prediction result, wherein the passenger action prediction result is used to characterize the action intention of each passenger in the vehicle to be controlled at the next time step at the current moment;

[0067] The vehicle control module 204 is used to control the speed of the vehicle to be controlled based on the passenger action prediction result and the sitting posture stability state data, so as to start and stop the vehicle to be controlled.

[0068] Optionally, in an embodiment, the gesture recognition module 202 includes:

[0069] An image generating unit, configured to obtain a three-dimensional posture image corresponding to each passenger in the vehicle to be controlled according to the passenger motion state data;

[0070] The posture recognition unit is used to input the three-dimensional posture diagram corresponding to each passenger in the vehicle to be controlled into the trained action classification model to obtain the passenger posture recognition result, and the passenger posture recognition result includes the action category corresponding to each passenger in the vehicle to be controlled at the current moment.

[0071] Optionally, in an embodiment, the action prediction module 203 includes:

[0072] an image processing unit, configured to obtain, when the passenger posture recognition result includes a preset target action category, a three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled according to the passenger motion state data, wherein the three-dimensional posture graph sequence includes multiple frames of three-dimensional posture graphs corresponding to the passenger;

[0073] An action prediction unit, used for inputting the three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled into the trained action sequence analysis model, and obtaining the action intention category result output by the action sequence analysis model;

[0074] The result generating unit is used to determine the passenger action prediction result according to the action intention category result.

[0075] Optionally, in an embodiment, the vehicle control module 204 includes:

[0076] A state acquisition unit, used for acquiring the vehicle running state of the vehicle to be controlled in real time, wherein the vehicle running state is a waiting-to-start state or a driving state;

[0077] a strategy acquisition unit, configured to determine a vehicle control strategy based on the vehicle running state, the passenger action prediction result and the sitting posture stability state data;

[0078] The vehicle control unit is used to control the speed of the vehicle to be controlled according to the vehicle control strategy so as to start and stop the vehicle to be controlled.

[0079] Optionally, in an embodiment, the policy acquisition unit includes:

[0080] A first judgment subunit is used to determine whether the action intention corresponding to each passenger in the vehicle to be controlled is to ride the vehicle, and whether the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets a preset sitting posture stability condition when the vehicle running state is a waiting-to-start state;

[0081] The first strategy acquisition sub-unit is used for, when the action intention corresponding to each passenger in the vehicle to be controlled is to get on the vehicle, and the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets the preset sitting posture stability conditions, the vehicle control strategy includes controlling the acceleration of the vehicle to be controlled so as to start the vehicle to be controlled according to the vehicle control strategy.

[0082] Optionally, in an embodiment, the policy acquisition unit includes:

[0083] The second judgment subunit is used to determine whether there are one or more passengers in the vehicle to be controlled whose corresponding action intention is to get off the vehicle, and whether the sitting posture stability state data corresponding to the one or more passengers meet the preset leaving seat condition when the vehicle is in the driving state;

[0084] The second strategy acquisition sub-unit is used for controlling the vehicle to be controlled to decelerate, determine the parking position and parking time, and perform parking control on the vehicle to be controlled according to the vehicle control strategy when there are one or more passengers in the vehicle to be controlled whose corresponding action intention is to get off the vehicle and the sitting posture stability state data corresponding to the one or more passengers meet the preset leaving seat conditions.

[0085] Embodiment 3

[0086] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the vehicle start-stop control method as described in the first embodiment.

[0087] Embodiment 4

[0088] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the vehicle start-stop control method as described in the first embodiment when executing the computer program.

[0089] Embodiment 5

[0090] This embodiment provides a vehicle, including the vehicle start-stop control device as described in the second embodiment.

[0091] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0092] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A vehicle start-stop control method, characterized in that: include: In response to the vehicle start-stop control command, real-time acquisition of passenger motion state data and sitting posture stability state data of each passenger in the vehicle to be controlled; According to the passenger motion state data, posture recognition is performed on each passenger in the vehicle to be controlled to obtain a passenger posture recognition result; Based on the passenger posture recognition result, predict the action of each passenger in the vehicle to be controlled to obtain a passenger action prediction result, wherein the passenger action prediction result is used to characterize the action intention of each passenger in the vehicle to be controlled at the next time step corresponding to the current moment; Based on the passenger action prediction result and the sitting posture stability state data, the speed of the vehicle to be controlled is controlled to perform start-stop control on the vehicle to be controlled.

2. The vehicle start-stop control method according to claim 1, characterized in that: The step of performing posture recognition on each passenger in the vehicle to be controlled according to the passenger motion state data to obtain a passenger posture recognition result includes: According to the passenger motion state data, a three-dimensional posture diagram corresponding to each passenger in the vehicle to be controlled is obtained; The three-dimensional posture graph corresponding to each passenger in the vehicle to be controlled is respectively input into the trained action classification model to obtain the passenger posture recognition result, and the passenger posture recognition result includes the action category corresponding to each passenger in the vehicle to be controlled at the current moment.

3. The vehicle start-stop control method according to claim 1, characterized in that: The step of performing motion prediction on each passenger in the vehicle to be controlled based on the passenger posture recognition result to obtain the passenger motion prediction result includes: In a case where the passenger posture recognition result includes a preset target action category, obtaining a three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled according to the passenger motion state data, wherein the three-dimensional posture graph sequence includes multiple frames of three-dimensional posture graphs corresponding to the passenger; Inputting the three-dimensional posture graph sequence corresponding to each passenger in the vehicle to be controlled into the trained motion sequence analysis model respectively, and obtaining the motion intention category result output by the motion sequence analysis model; The passenger action prediction result is determined according to the action intention category result.

4. The vehicle start-stop control method according to claim 1, characterized in that: The controlling the speed of the vehicle to be controlled based on the passenger action prediction result and the sitting posture stability state data to start and stop the vehicle to be controlled includes: Acquiring the vehicle running state of the vehicle to be controlled in real time, wherein the vehicle running state is a waiting-to-start state or a driving state; Based on the vehicle running state, determining a vehicle control strategy according to the passenger action prediction result and the sitting posture stability state data; The speed of the vehicle to be controlled is controlled according to the vehicle control strategy to perform start-stop control on the vehicle to be controlled.

5. The vehicle start-stop control method according to claim 4, characterized in that: The determining of the vehicle control strategy based on the vehicle running state, according to the passenger action prediction result and the sitting posture stability state data, includes: When the vehicle running state is the waiting-to-start state, determining whether the action intention corresponding to each passenger in the vehicle to be controlled is to ride the vehicle, and whether the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets a preset sitting posture stability condition; When the action intention corresponding to each passenger in the vehicle to be controlled is to get on the vehicle, and the sitting posture stability state data corresponding to each passenger in the vehicle to be controlled meets the preset sitting posture stability conditions, the vehicle control strategy includes controlling the acceleration of the vehicle to be controlled so as to start the vehicle to be controlled according to the vehicle control strategy.

6. The vehicle start-stop control method according to claim 4, characterized in that: The determining of the vehicle control strategy based on the vehicle running state, according to the passenger action prediction result and the sitting posture stability state data, includes: When the vehicle is in the driving state, determining whether there are one or more passengers in the vehicle to be controlled whose corresponding action intention is to get off the vehicle, and whether the sitting posture stability state data corresponding to the one or more passengers meets the preset leaving seat condition; When there are one or more passengers in the vehicle to be controlled whose action intention is to get off the vehicle, and the sitting posture stability state data corresponding to the one or more passengers meets the preset seat leaving conditions, the vehicle control strategy includes controlling the vehicle to be controlled to decelerate, determine the parking position and parking time, so as to perform parking control on the vehicle to be controlled according to the vehicle control strategy.

7. A vehicle start-stop control device, characterized in that: include: A data acquisition module, for acquiring in real time passenger motion state data and sitting posture stability state data of each passenger in the vehicle to be controlled in response to a vehicle start-stop control instruction; A posture recognition module, used to perform posture recognition on each passenger in the vehicle to be controlled according to the passenger motion state data, and obtain a passenger posture recognition result; an action prediction module, for predicting the action of each passenger in the vehicle to be controlled based on the passenger posture recognition result, and obtaining a passenger action prediction result, wherein the passenger action prediction result is used to characterize the action intention of each passenger in the vehicle to be controlled at the next time step at the current moment; The vehicle control module is used to control the speed of the vehicle to be controlled based on the passenger action prediction result and the sitting posture stability state data, so as to start and stop the vehicle to be controlled.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the vehicle start-stop control method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the vehicle start-stop control method as claimed in any one of claims 1 to 6 when executing the computer program.

10. A vehicle, characterized in that: Including the vehicle start-stop control device as described in claim 7.

Images