Vehicle control method, device and equipment and vehicle

Abstract

The invention discloses a vehicle control method, device and equipment and a vehicle, and the method comprises the steps: determining the state information of the vehicle and the state information of passengers in the vehicle based on the collection data of the sensing equipment of the vehicle; and under the condition that the state information of the passenger represents that the passenger is in the photographing state, controlling the vehicle based on the state information of the vehicle.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of vehicle technology, and in particular to a vehicle control method, device, equipment and vehicle. Background Art

[0002] At present, passengers in vehicles often need to take photos. In related solutions, passengers control the vehicle through voice, but the vehicle needs to parse and process the voice command after receiving it, which may cause the vehicle to miss the best photo opportunity when executing the voice command, resulting in poor photo effects for passengers. It can be seen that there is currently no solution that allows users to obtain better photo effects by controlling the vehicle when passengers need to take photos. Summary of the invention

[0003] In view of this, the embodiments of the present application at least provide a vehicle control method, device, equipment and vehicle.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] On the one hand, an embodiment of the present application provides a vehicle control method, including:

[0006] Determine the status information of the vehicle and the status information of the passengers in the vehicle based on the collected data of the sensor equipment of the vehicle;

[0007] In a case where the state information of the passenger indicates that the passenger is in a photographing state, the vehicle is controlled based on the state information of the vehicle.

[0008] On the other hand, an embodiment of the present application provides a vehicle control device, the device comprising:

[0009] A determination module, for determining the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the sensor equipment of the vehicle;

[0010] The control module is used to control the vehicle based on the vehicle status information when the passenger status information indicates that the passenger is in a photo taking state.

[0011] In yet another aspect, an embodiment of the present application provides a vehicle, comprising: a sensor device and a controller;

[0012] A sensor device is used to obtain collected data and send the collected data to a controller;

[0013] The controller is used to determine the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the vehicle's sensor equipment; when the state information of the passengers indicates that the passengers are in a state of taking photos, the vehicle is controlled based on the state information of the vehicle.

[0014] On the other hand, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a data processing program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0015] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0017] Figure 1 A schematic diagram of an implementation flow of a vehicle control method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the implementation flow of a vehicle control method for an unmanned online car-hailing vehicle provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application;

[0020] Figure 4 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0022] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0023] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application are merely to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0024] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those skilled in the art in the field to which the embodiments of the present application belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0025] The present application embodiment provides a vehicle control method, such as Figure 1 As shown, it may include step S101 and step S102:

[0026] Step S101: determining the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the sensor equipment of the vehicle;

[0027] Here, the sensing device is a device that can convert non-electrical signals during the operation of the vehicle into electrical signals and transmit various operating condition information to the vehicle's electronic control unit (ECU). It is an important part of the vehicle's electronic control system. They can monitor various parameters of the vehicle in real time to ensure the normal operation and optimized performance of the vehicle.

[0028] In some embodiments, the vehicle's status information includes the vehicle's driving speed and the vehicle's window status. In the case where the vehicle's status information includes the vehicle's driving speed, the vehicle's sensing device may include a speed sensor and / or an on-board navigation device on the vehicle; in the case where the vehicle's status information includes the vehicle's window status, the vehicle's sensing device may include a Hall sensor acquisition and / or an image acquisition device.

[0029] In some embodiments, the status information of the passengers in the vehicle includes a photographing status, a disturbable status, and an undisturbing status, wherein the disturbable status includes: an entertainment status and a chatting status, and the undisturbing status includes a sleeping status and a talking status. Wherein, in the case where the status information of the passenger includes a photographing status or an undisturbing status, the sensing device of the vehicle may include an image acquisition device and / or a voice acquisition device. Where the status information of the passenger includes an disturbable status, the sensing device of the vehicle may include at least one of an image acquisition device, a voice acquisition device, and an application installed on the vehicle.

[0030] It can be understood that the image acquisition device can be a device with an image acquisition function such as a camera and a video camera, and the voice acquisition device can be a device with a voice acquisition function such as a microphone and a recorder.

[0031] Step S102: When the passenger's status information indicates that the passenger is in a state of taking a photo, the vehicle is controlled based on the vehicle's status information.

[0032] Here, the photo taking status refers to that the passenger is about to or is taking a photo.

[0033] In some embodiments, when a passenger is in a photo-taking state, it may refer to the passenger taking photos of scene information inside the vehicle. For example, the passenger may be taking a selfie, or the passenger may be taking photos of a target object inside the vehicle, or the passenger may be taking photos of scene information outside the vehicle through the vehicle's windows.

[0034] In some embodiments, it is possible to determine whether the passenger is in a state of taking a photo based on the image data collected by the image acquisition device. For example, it is possible to determine whether the passenger is in a state of taking a photo by using a feature extraction method; it is also possible to determine whether the passenger is in a state of taking a photo by using an image acquisition device; or it is also possible to determine whether the passenger is in a state of taking a photo by using a deep neural network model.

[0035] For example, the feature extraction method is used for explanation. First, the image data reported by the image acquisition device is preprocessed, and then the feature information in the preprocessed image data is extracted. Finally, whether the passenger is in a state of taking a photo is determined based on the feature information. For example, the feature information of the hand position of the passenger in the image data can be obtained. When the feature information of the hand position indicates that the passenger is raising or about to raise an image acquisition device (e.g., a mobile phone or a camera), it is determined that the passenger is in a state of taking a photo. Alternatively, the feature information of the image acquisition device in the image data can be obtained. When the feature information of the image acquisition device indicates that the image acquisition device is in an image acquisition mode, it is determined that the passenger is in a state of taking a photo. Alternatively, the feature information of the hand position of the passenger in the image data and the feature information of the image acquisition device can be used to determine that the passenger is in a state of taking a photo.

[0036] For example, an image acquisition device is used for illustration. The image acquisition device has a built-in behavior recognition algorithm. After the image acquisition device acquires image data in the vehicle, the image data is input into the behavior recognition algorithm for processing. After the behavior recognition algorithm completes the processing, the behavior information corresponding to the image data is output. In this way, it is possible to determine whether the passenger is in the state of taking photos through the output of the image acquisition device. The behavior recognition algorithm includes the recognition of the photo-taking behavior. The recognition algorithm for the photo-taking behavior will focus on collecting the characteristic information of the passenger's hand position, the passenger's line of sight information, etc. It should be noted that the behavior recognition algorithm needs to be optimized through a large amount of training data to improve its recognition accuracy and robustness. The training data should include various passenger behavior scenarios, especially the various manifestations of photo-taking behavior.

[0037] For example, taking a deep neural network model as an example, the image data reported by the image acquisition device is input into the deep neural network model, so that whether the passenger is in a state of taking pictures can be determined through the output of the deep neural network model, wherein the deep neural network model extracts feature information from the input image data through multiple convolutional layers, pooling layers and fully connected layers. The feature information may include the passenger's contour information, posture information, hand movement information and image acquisition device information, thereby determining whether the passenger is in a state of taking pictures through the feature information.

[0038] In some embodiments, controlling the vehicle based on the vehicle status information includes: controlling the vehicle's driving speed according to the vehicle's current speed.

[0039] In some implementations, whether the passenger is in a photo-taking state may be determined based on voice data collected by a voice collection device.

[0040] For example, the voice data collected by the voice collection device is input into the artificial intelligence model, so that the voice data is analyzed by the artificial intelligence model to determine the passenger's intention, and then based on the passenger's intention, it is determined whether the passenger is in a state of taking pictures. The voice data collected by the voice collection device can be voice commands issued by the passenger through the on-board smart assistant, or it can be voice data of passengers in the car collected in real time.

[0041] In some implementations, whether the passenger is in the photo-taking state may be determined based on voice data collected by the voice collection device and image data collected by the image collection device.

[0042] In some implementations, the passenger's behavior information may be determined based on the image data captured by the image capture device, and whether the passenger is in a photo-taking state may be determined based on the behavior information.

[0043] In some embodiments, the implementation of controlling a vehicle based on the vehicle's status information may be: determining the vehicle's current control strategy based on the vehicle's status information, and then generating control instructions based on the control strategy and sending them to the corresponding control system, so that the control system can operate based on the control instructions.

[0044] In some implementations, controlling the vehicle based on the vehicle status information may also be implemented by: determining a control system to be controlled based on the vehicle status information, thereby generating a control instruction for the control system to be controlled and sending it to the control system.

[0045] In some embodiments, controlling the vehicle may include controlling the vehicle's driving route, controlling the vehicle's driving speed, controlling the vehicle's parking position, controlling a suspension system in the vehicle, controlling the vehicle's applications, etc.

[0046] In an embodiment of the present application, first, based on the collected data of the vehicle's sensor equipment, the vehicle's status information and the passenger's status information are determined, and then, when the passenger's status information indicates that the passenger is in a photo-taking state, the vehicle is controlled based on the vehicle's status information. This allows the passenger to automatically control the vehicle through the vehicle's status information without having to issue control instructions to the vehicle, so that the passenger can obtain better image data, which not only improves the passenger's photo-taking experience in the vehicle, but also improves the passenger's driving experience.

[0047] In some embodiments, the above step S102, controlling the vehicle based on the vehicle status information, includes at least one of the following steps S1021 and S1022:

[0048] Step S1021: when the vehicle status information indicates that the vehicle is in a driving state, adjusting the driving route of the vehicle based on the passenger's photo-taking action;

[0049] In some embodiments, the vehicle status information includes the vehicle's driving speed. When the vehicle's driving speed is not 0, it is determined that the vehicle is in a driving state.

[0050] In some embodiments, when the subject photographed by the passenger is located outside the vehicle, based on the passenger's photographing action, image data in an image acquisition device used by the passenger to take the photo is determined; based on the image data, the subject photographed is determined; and based on the location information of the subject photographed, the vehicle's driving route is adjusted.

[0051] In this way, by adjusting the driving route of the vehicle, the distance between the driving route of the vehicle and the photographed object can be reduced, thereby improving the quality of image data collected by passengers when taking photos of the photographed object.

[0052] Step S1022: When the vehicle status information indicates that the vehicle is in a driving state, adjust the driving speed of the vehicle.

[0053] In some embodiments, after determining the photographed object, the photographing time for the passenger to take a photograph of the photographed object is determined; based on the photographing time, the driving speed of the vehicle is adjusted. In this way, by adjusting the driving speed of the vehicle, when the passenger takes a photograph of the photographed object, the occurrence of blurring problems when the passenger uses the image acquisition device to focus on the photographed object can be reduced, thereby improving the quality of image data collected by the passenger when taking a photograph of the photographed object.

[0054] It is understandable that when the vehicle is in motion, the vehicle's speed is adjusted, and the vehicle's route is adjusted based on the passenger's photo-taking action. In this way, the vehicle's speed and route can be flexibly adapted to the passenger's photo-taking needs when the vehicle is in motion, further improving the quality of the image data collected when the passenger takes photos of the photographed object.

[0055] In some embodiments, the above step S102, controlling the vehicle based on the vehicle status information, includes the following steps S1023:

[0056] Step S1023: Control the vehicle based on the vehicle's exterior scene information and the vehicle's status information; wherein the exterior scene information includes at least one of the following: exterior image information, weather information, and road condition information.

[0057] In some embodiments, the vehicle exterior scene information may be acquired through at least one type of sensing device selected from the group consisting of an image acquisition device, a positioning device, and a radar device.

[0058] In some embodiments, image data reported by an image acquisition device installed outside the vehicle is preprocessed, and then feature information in the preprocessed image data is extracted; based on the feature information, the outside vehicle image information, weather information, and road condition information are determined.

[0059] In some implementations, radar data reported by a radar device installed on a vehicle is preprocessed and feature information is extracted from the preprocessed radar data, and the feature information is input into a machine learning algorithm to classify and identify the extracted feature information to determine the road condition information of the vehicle's driving road. The radar device may be at least one of an ultrasonic radar, a millimeter wave radar, and a laser radar.

[0060] In some embodiments, the vehicle is integrated with a navigation system, and the navigation system determines the road condition information of the vehicle's driving road by matching the vehicle's current location information with the vehicle's location information.

[0061] In some implementations, the vehicle may be connected to the Internet via an in-vehicle Internet device to determine weather information for the vehicle's current location.

[0062] In some embodiments, an onboard weather station is integrated into the vehicle, and weather information at the current location of the vehicle can be determined through the onboard weather station.

[0063] It can be understood that this embodiment is applied to a scene where the photographed object is located outside the vehicle.

[0064] In the above embodiment, the vehicle is controlled by the vehicle's external scene information and the vehicle's status information, so that the passengers' photography needs are met through the control of the vehicle, and the quality of image data collected by the passengers when taking photos of the subjects is improved.

[0065] In some embodiments, the above step S1023, based on the vehicle's external scene information and the vehicle's state information, controls the vehicle, including one of the following steps S10231 to S10234:

[0066] Step S10231: When the vehicle status information indicates that the window corresponding to the passenger is in the first window state, and the weather information indicates the first weather, the window is controlled to be in the second window state; the opening degree of the window in the second window state is greater than the opening degree of the window in the first window state;

[0067] Here, the window corresponding to the passenger refers to the window used by the passenger to take pictures of the scene outside the vehicle. The first window state may include a fully closed state or a partially open state; the second window state may include a fully open state or a partially open state. The first weather refers to weather that can improve or not affect the comfort of the passenger when the window is in a partially open or fully open state, such as a sunny day or a cloudy day.

[0068] In some implementations, the state of the vehicle window may be acquired by a sensor installed on the vehicle window.

[0069] In some implementations, the state of the vehicle window may be determined through image data captured by an image capture device installed in the vehicle.

[0070] In some implementations, when the first window state is a fully closed state, the second window state may be a fully opened state or a partially opened state.

[0071] In some implementations, when the first window state is a partially open state, the second window state may be a partially open state with a greater degree of opening than the first window state, or may be a fully open state.

[0072] In this way, when the weather information represents the first weather, by adjusting the state of the car window from the first window state to the second window state, passengers do not need to take pictures of the subject through the glass, thereby improving the quality of image data collected by the passengers when taking pictures of the subject.

[0073] Step S10232: when the vehicle state information indicates that the vehicle is stationary and the image information outside the vehicle indicates that the photographed object is blocked, determine a first position of the vehicle based on the photographed object; control the vehicle to move to the first position; the first position is a position where the photographed object is not blocked;

[0074] In some embodiments, the vehicle status information includes the vehicle's driving speed. When the vehicle's driving speed is 0, it is determined that the vehicle is stationary.

[0075] In some embodiments, the vehicle exterior image information indicates that the photographed object is blocked, which may be that a window of the current vehicle is blocked, or that there is an obstruction between the current position of the vehicle and the photographed object.

[0076] In some embodiments, the photographed object can be determined first, and then the outside image information between the vehicle and the photographed object and the position information of the photographed object can be determined through image data collected by an image acquisition device installed outside the vehicle. Finally, the first position of the vehicle is determined based on the outside image information and the position information of the photographed object.

[0077] In this way, by moving the vehicle to the first position, when the passenger takes a photo of the subject, there is no obstruction between the passenger and the subject, so that the passenger can collect complete image data of the subject, thereby improving the quality of the image data collected by the passenger when taking a photo of the subject.

[0078] It can be understood that when the vehicle is stationary, by simultaneously adjusting the vehicle windows and the parking position of the vehicle, not only can the passengers collect complete image data of the subject being photographed, but also the interference of the window glass on the passengers' collection of the subject being photographed can be reduced, thereby further improving the quality of image data collected by the passengers when taking pictures of the subject being photographed.

[0079] In some embodiments, based on the photographed object and the image information outside the vehicle, the second position of the vehicle is determined, and the passenger is prompted whether to move to the second position, the second position being the best photographing position for the passenger to photograph the photographed object, wherein the best photographing position may be the position with the brightest light for photographing the photographed object, and / or the position closest to the photographed object, so that the image data collected at the best photographing position has better data quality.

[0080] Step S10233: when the vehicle status information indicates that the vehicle is in a driving state, and the image information outside the vehicle indicates that the photographed object is blocked, determining a target driving route of the vehicle based on the photographed object; adjusting the vehicle's driving route to the target driving route; the target driving route is a route where the photographed object is not blocked;

[0081] In some embodiments, the target driving route of the vehicle is determined by using image data collected by an image acquisition device installed outside the vehicle, location information of the photographed object, and target address information of the vehicle.

[0082] In this way, when the vehicle travels along the target driving route, there is no obstruction between the passenger and the subject when the passenger takes a photo of the subject, so that the passenger can collect complete image data of the subject, thereby improving the quality of the image data collected by the passenger when taking a photo of the subject.

[0083] It can be understood that when the vehicle is in motion, by simultaneously adjusting the vehicle windows and the vehicle's driving route, not only can the passengers collect complete image data of the subject being photographed, but also the interference of the window glass on the passengers' collection of the subject being photographed can be reduced, thereby further improving the quality of image data collected by the passengers when taking pictures of the subject being photographed.

[0084] In some embodiments, based on the photographed object and the image information outside the vehicle, the second position of the vehicle is determined, and the passenger is prompted whether to park at the second position, the second position being the best photographing position for the passenger to photograph the photographed object, wherein the best photographing position may be the position with the brightest light for photographing the photographed object, and / or the position closest to the photographed object, so that the image data collected at the best photographing position has better data quality.

[0085] Step S10234: When the vehicle status information indicates that the vehicle is in a driving state, and the road condition information indicates that the vehicle is entering a target road condition, the operating state of the vehicle's suspension system is adjusted.

[0086] Here, the target road condition refers to the uneven road on which the vehicle is traveling. The suspension system is a general term for all the force transmission connection devices between the vehicle frame and the axle or wheel. Its function is to transmit the force and torque acting between the wheel and the frame, and to buffer the impact force transmitted to the frame or body by the uneven road surface, and to attenuate the vibration caused thereby, so as to ensure the smooth driving of the vehicle.

[0087] In some implementations, when the vehicle status information indicates that the vehicle is in a driving state and the vehicle enters a speed bump, the suspension system of the vehicle is adjusted.

[0088] In some embodiments, when the vehicle status information indicates that the vehicle is in a driving state and the vehicle enters a flooded area, the suspension system of the vehicle is adjusted.

[0089] Here, by adjusting the operating state of the vehicle's suspension system, the impact force transmitted to the frame or body by the target road condition can be buffered, and the vibration caused thereby can be attenuated to ensure smooth driving of the car. This can reduce the bumps experienced by passengers when taking pictures of the subject, thereby allowing a clear capture of the subject. This not only improves the quality of image data captured by passengers when taking pictures of the subject, but also reduces the problem of image capture equipment falling due to bumps when passengers are taking pictures.

[0090] It can be understood that when the vehicle is in motion, by simultaneously adjusting the vehicle windows and the vehicle's driving route, not only can the passengers collect complete image data of the subject being photographed, but also the problem of the window glass interfering with the passengers' collection of the subject being photographed can be reduced. Therefore, when the vehicle enters the target road conditions, by adjusting the vehicle's suspension system, the bumps that the passengers experience when taking pictures of the subject, and even the problem of the image acquisition device falling due to the bumps, can be reduced, thereby further improving the quality of the image data collected by the passengers when taking pictures of the subject.

[0091] In some embodiments, the above step S102, based on the target information of the vehicle, controls the vehicle, including the following steps S1024:

[0092] Step S1024: Based on the state information of the target application and the state information of the vehicle, the vehicle is controlled so that the target application can execute the task associated with the passenger's photo-taking behavior.

[0093] In some implementations, the target application may be installed in a built-in system for an in-vehicle display screen.

[0094] In the above embodiment, the vehicle is controlled by the status information of the target application of the vehicle and the status information of the vehicle, so that the passengers' photography needs are met through the control of the vehicle, and the quality of the image data collected by the passengers when taking pictures of the subjects is improved.

[0095] In some embodiments, the above step S1024, based on the state information of the target application and the state information of the vehicle, controls the vehicle, including step S10241:

[0096] Step S10241: When the state information of the target application indicates that the target application is in a playback state of information explaining the photographed subject, and the state information of the vehicle indicates that the vehicle is in a driving state, determine the duration of the target application's explanation of the photographed subject; and adjust the driving speed of the vehicle based on the explanation duration.

[0097] In some implementations, the target application may explain the subject information by playing video or audio, or by statically displaying the subject information on the display screen.

[0098] In the above embodiment, when the vehicle is in motion, the driving speed is adjusted according to the duration of the explanation of the photographed object by the target application, so that the driving speed of the vehicle can be adapted to the duration of the explanation, so that the vehicle can synchronously complete the explanation of the photographed object when passing the photographed object, thereby improving the driving experience of the passengers.

[0099] In some embodiments, the method includes step S10242:

[0100] Step S10242: When the state information of the target application indicates that the target application has no target priority task to execute, and the image information outside the vehicle indicates that the position of the vehicle corresponds to the photographed object, control the target application to explain the photographed object; the target priority task is a task triggered by the passenger.

[0101] Here, the correspondence between the position of the vehicle and the photographed object can be understood as the vehicle passing by the photographed object.

[0102] In some implementations, after the photographed object is acquired, a search may be performed based on the photographed object to determine explanation information of the photographed object, thereby controlling the target application to explain the photographed object.

[0103] In some embodiments, when the state information of the target application indicates that the target application has no target priority task to execute, and the external image information indicates that the location of the vehicle corresponds to the photographed object, the passenger can be prompted whether to switch the state of the target application to explain the photographed object.

[0104] In the above embodiment, when the target application has no target priority task to execute, the target application is controlled to automatically explain the photographed object, so that the passengers can understand the information of the photographed object.

[0105] The following describes the application of the embodiments of the present application in actual scenarios.

[0106] During the driving process of the driverless online ride-hailing vehicle, the driverless online ride-hailing vehicle cannot automatically control the vehicle's driving status and in-vehicle intelligent applications based on the passenger's status, behavior and external scenes.

[0107] The existing solution is to use voice control to control the vehicle's driving status and in-vehicle smart applications. However, this solution has the following disadvantages:

[0108] 1. Voice commands cannot be used in some scenarios. For example, when there are many passengers in the car and the noise is loud, the voice commands cannot be accurately received, which may cause the vehicle to ask the passengers for voice commands again or execute the wrong voice commands; or, when the passengers gradually become sleepy and fall asleep while browsing their phones, the passengers cannot actively control the vehicle by voice, which would allow the vehicle to drive more smoothly.

[0109] 2. Voice commands require more command transmission time. For example, when a vehicle is driving past a scenic spot and a passenger wants to take a photo, they need to use voice commands to control the vehicle. However, by the time the vehicle receives the voice command and gives feedback, the opportunity to take the photo may have been missed.

[0110] 3. Failure to fine-tune vehicle control. For example, when a passenger is taking photos of a scenic spot, there may be obstacles blocking the passenger's view of the scenic spot. In this scenario, it may not be easy for the passenger to accurately describe the command wording for adjusting the vehicle's position.

[0111] In view of the above problems and the defects of existing solutions, the embodiment of the present application proposes a vehicle control method for an unmanned online car-hailing vehicle, and the vehicle control method has the following core points:

[0112] 1. The in-car camera detects the status and behavior of passengers in the car in real time, and the out-car camera detects the scene outside the car in real time. Combined with the road conditions, vehicle status and other information of intelligent driving, a linkage perception decision-making system is jointly constructed.

[0113] 2. The in-vehicle application will detect changes in the environment inside and outside the vehicle in real time and map them with the tasks of its own application. Based on the matching relationship between tasks and scenarios, it will intelligently adjust driving decisions and intelligent application status. For example, when a passenger raises his mobile phone to the window, the passenger's photo-taking behavior is detected, and the vehicle can be decelerated appropriately; or when the vehicle approaches a scenic spot, the system will automatically start the scenic spot explanation function.

[0114] 3. During the execution of the task, the status and satisfaction of the passengers are continuously and dynamically detected, and the decision is adjusted according to the feedback information to form new instructions. For example, when the vehicle is stationary, there are passengers in the vehicle taking pictures. After detecting that the window is blocked by other objects, the parking position of the vehicle is automatically moved. Or, when the vehicle passes through a speed bump, the vehicle is controlled to slow down and detect that the passenger is not satisfied with the bump amplitude. In this way, the speed can be reduced to a lower value the next time the speed bump is encountered.

[0115] The present application embodiment proposes a vehicle control method for an unmanned online car-hailing vehicle, such as Figure 2 As shown, it may include steps S201 to S205:

[0116] Step S201: The camera (ie, the above-mentioned sensing device) obtains detection data of passengers inside the vehicle and scenes outside the vehicle (ie, the above-mentioned collected data).

[0117] Here, the detection data of the passengers in the car can be obtained through a camera installed in the car, and the detection data of the passengers in the car can be obtained through a camera installed outside the car, and the detection data can be sent to the perception decision system.

[0118] Step S202: The perception decision system determines the interior scene, exterior scene, vehicle status, and application status based on the trained AI model and the detection data inside and outside the vehicle.

[0119] Here, the in-vehicle and out-vehicle detection data are used as the input information of the AI ​​model, and the detection results are determined based on the output information of the AI ​​model. The detection results include: in-vehicle scene, out-vehicle scene, vehicle status and application status, among which the in-vehicle scene is used to indicate the passenger status information.

[0120] Step S203: The perception decision system determines driving decision-making and intelligent application control requirements based on the detection results of the in-vehicle scene, the out-of-vehicle scene, the vehicle status and the application status, and generates corresponding tasks.

[0121] Here, the perception decision system is preset with a mapping table between detection results and tasks. After obtaining the current in-vehicle scene, out-of-vehicle scene, vehicle status, and application status, the corresponding task is generated based on the mapping table. The mapping table is shown in Table 1 below.

[0122] Table 1 Mapping relationship table

[0123]

[0124]

[0125] Step S204: The perception decision system generates control instructions based on the task, and sends the control instructions to each application for execution.

[0126] Step S205: The perception decision system continuously and dynamically detects the in-vehicle scene, the out-vehicle scene, the vehicle status and the application status, and forms new control instructions.

[0127] It should be noted that the outside-vehicle scene is used to indicate at least one of the outside-vehicle image information, weather information and road condition information in the above embodiments, the vehicle status is used to indicate the status information of the vehicle in the above embodiments, the application status is used to indicate the status information of the target application in the above embodiments, the high-priority task is used to indicate the target priority task in the above embodiments, and the inside-vehicle scene is used to indicate the status information of the passengers in the above embodiments.

[0128] The embodiments of the present application can bring the following beneficial effects: 1. The combination of in-vehicle applications and the linkage perception decision-making system of intelligent driving makes the recognition results of scenes inside and outside the car more accurate. 2. Intelligently adjust driving decisions to create an intelligent and humanized driving experience. 3. The feedback mechanism self-learning optimization can make decisions that can satisfy passengers through continuous learning optimization, thereby improving passengers' ride satisfaction. 4. The time from the recognition of scenes inside and outside the car to the execution of tasks is short.

[0129] Based on the foregoing embodiments, an embodiment of the present application provides a control device for a vehicle, which includes the modules included and the units included in the modules, and can be implemented by a processor in a computer device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0130] Figure 3 A schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the vehicle control device 300 includes:

[0131] A determination module 301 is used to determine the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the sensor equipment of the vehicle;

[0132] The control module 302 is used to control the vehicle based on the vehicle status information when the passenger status information indicates that the passenger is in a photo taking state.

[0133] In some embodiments, the control module includes at least one of the following: a first adjustment unit, used to adjust the vehicle's driving route based on the passenger's photo-taking action when the vehicle's status information indicates that the vehicle is in a driving state; a second adjustment unit, used to adjust the vehicle's driving speed when the vehicle's status information indicates that the vehicle is in a driving state.

[0134] In some embodiments, the control module includes: a first control unit for controlling the vehicle based on the vehicle's external scene information and the vehicle's status information; wherein the external scene information includes at least one of the following: external image information, weather information, and road condition information.

[0135] In some embodiments, the control unit includes at least one of the following: a first control subunit, used to control the window to be in a second window state when the vehicle status information indicates that the window corresponding to the passenger is in a first window state and the weather information indicates the first weather; the degree of opening of the window in the second window state is greater than the degree of opening of the window in the first window state; a second control subunit, used to determine the first position of the vehicle based on the photographed object when the vehicle status information indicates that the vehicle is stationary and the image information outside the vehicle indicates that the photographed object is blocked; control the vehicle to move to the first position; the first position is a position where the photographed object is not blocked; a first adjustment subunit, used to determine the target driving route of the vehicle based on the photographed object when the vehicle status information indicates that the vehicle is in a driving state and the image information outside the vehicle indicates that the photographed object is blocked; adjust the vehicle's driving route to a target driving route; the target driving route is a route where the photographed object is not blocked; a second adjustment subunit, used to adjust the operating state of the vehicle's suspension system when the vehicle status information indicates that the vehicle is in a driving state and the road condition information indicates that the vehicle enters a target road condition.

[0136] In some embodiments, the control module includes: a second control unit for controlling the vehicle based on the state information of the target application and the state information of the vehicle, and the target application can perform a task associated with the passenger's photo-taking behavior.

[0137] In some embodiments, the second control unit includes: a third adjustment subunit, which is used to determine the duration of the target application's explanation of the photographed subject when the status information of the target application indicates the playback status of the target application's explanation information for the photographed subject, and the status information of the vehicle indicates that the vehicle is in a driving state; and adjust the driving speed of the vehicle based on the explanation duration.

[0138] In some embodiments, the second control unit includes: a third control subunit, used to control the target application to explain the photographed object when the status information of the target application indicates that the target application has no target priority task to execute, and the image information outside the vehicle indicates that the position of the vehicle corresponds to the photographed object; the target priority task is a task triggered by a passenger.

[0139] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0140] It should be noted that in the embodiment of the present application, if the above-mentioned data processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0141] A vehicle, comprising: a sensing device and a controller;

[0142] A sensor device is used to obtain collected data and send the collected data to a controller;

[0143] The controller is used to determine the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the vehicle's sensor equipment; when the state information of the passengers indicates that the passengers are in a state of taking photos, the vehicle is controlled based on the state information of the vehicle.

[0144] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0145] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0146] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes some or all of the steps in the above method.

[0147] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0148] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0149] It should be noted that Figure 4 is a hardware entity schematic diagram of an electronic device in an embodiment of the present application, such as Figure 4 As shown, the hardware entity of the electronic device 400 includes: a processor 401, a communication interface 402 and a memory 403, wherein:

[0150] The processor 401 generally controls the overall operation of the electronic device 400 .

[0151] The communication interface 402 enables the electronic device to communicate with other terminals or servers through a network.

[0152] The memory 403 is configured to store instructions and applications executable by the processor 401, and can also cache data to be processed or processed by the processor 401 and each module in the electronic device 400 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data transmission can be performed between the processor 401, the communication interface 402, and the memory 403 through the bus 404.

[0153] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial numbers of the above-mentioned steps / processes does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0154] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0156] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0157] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0158] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0159] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0160] The above are only implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A vehicle control method, comprising: Determining the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the sensor equipment of the vehicle; In a case where the status information of the passenger indicates that the passenger is in a photographing state, the vehicle is controlled based on the status information of the vehicle.

2. The method according to claim 1, wherein: The controlling the vehicle based on the status information of the vehicle comprises at least one of the following: When the state information of the vehicle indicates that the vehicle is in a driving state, adjusting the driving route of the vehicle based on the photo-taking action of the passenger; When the state information of the vehicle indicates that the vehicle is in a driving state, the driving speed of the vehicle is adjusted.

3. The method according to claim 1, wherein: The controlling the vehicle based on the state information of the vehicle comprises: The vehicle is controlled based on the vehicle's exterior scene information and the vehicle's status information; wherein the vehicle's exterior scene information includes at least one of the following: exterior image information, weather information, and road condition information.

4. The method according to claim 3, wherein: The controlling the vehicle based on the vehicle exterior scene information and the vehicle status information comprises at least one of the following: When the vehicle status information indicates that the vehicle window corresponding to the passenger is in a first window state, and the weather information indicates a first weather, controlling the vehicle window to be in a second window state; The opening degree of the vehicle window in the second window state is greater than the opening degree of the vehicle window in the first window state; When the vehicle status information indicates that the vehicle is stationary and the vehicle exterior image information indicates that the photographed object is blocked, determining a first position of the vehicle based on the photographed object; controlling the vehicle to move to the first position; the first position is a position where the photographed object is not blocked; When the vehicle status information indicates that the vehicle is in a driving state, and the vehicle exterior image information indicates that the photographed object is blocked, determining a target driving route of the vehicle based on the photographed object; adjusting the driving route of the vehicle to the target driving route; The target driving route is a route where the photographed object is not blocked; When the state information of the vehicle indicates that the vehicle is in a driving state, and the road condition information indicates that the vehicle is entering a target road condition, the operating state of the suspension system of the vehicle is adjusted.

5. The method according to any one of claims 1 to 4, wherein: The controlling the vehicle based on the state information of the vehicle comprises: Based on the state information of the target application and the state information of the vehicle, the vehicle is controlled, and the target application is capable of executing a task associated with the photo-taking behavior of the passenger.

6. The method according to claim 5, wherein: The controlling the vehicle based on the state information of the target application and the state information of the vehicle comprises: When the state information of the target application represents the playback state of the target application's explanation information for the photographed object, and the state information of the vehicle represents the vehicle is in a driving state, determine the duration of the target application's explanation of the photographed object; and adjust the driving speed of the vehicle based on the explanation duration.

7. The method according to claim 6, wherein: The method further comprises: When the status information of the target application indicates that the target application has no target priority task to execute, and the external image information indicates that the position of the vehicle corresponds to the photographed object, the target application is controlled to explain the photographed object; the target priority task is a task triggered by the passenger.

8. A vehicle control device, comprising: A determination module, configured to determine the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the sensor equipment of the vehicle; The control module is used to control the vehicle based on the status information of the vehicle when the status information of the passenger indicates that the passenger is in a state of taking a photo.

9. An electronic device comprising a memory and a processor, wherein the memory stores a data processing program that can be run on the processor, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the program.

10. A vehicle comprising: Sensing devices and controllers; The sensor device is used to obtain collected data and send the collected data to the controller; The controller is used to determine the state information of the vehicle and the state information of the passengers in the vehicle based on the collected data of the sensor equipment of the vehicle; In a case where the status information of the passenger indicates that the passenger is in a photographing state, the vehicle is controlled based on the status information of the vehicle.

Images