Vehicle Control Method, Device, Electronic Device, Storage Medium, Product and Vehicle

Abstract

This application relates to a vehicle control method, apparatus, electronic device, storage medium, product, and vehicle. The method comprises determining, based on first position information of a terminal device relative to the vehicle, a target vehicle device to be operated by a target subject holding the terminal device, the terminal device being in communication with the vehicle, and controlling the target vehicle device. This application can optimize the user's vehicle control experience.

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and in particular to a vehicle control method, device, electronic device, storage medium, product and vehicle. Background Art

[0002] The control systems of existing vehicles such as doors, tailgates, and front hoods rely on traditional control methods such as physical buttons, remote controls, or car keys.

[0003] This traditional control method requires manual operation by the user. The control process is cumbersome and not intelligent enough, which makes the user operation very inconvenient. Summary of the Invention

[0004] The embodiments of the present application provide a vehicle control method, device, electronic device, computer-readable storage medium, computer program product and vehicle to improve the convenience of vehicle control and thereby enhance the user's control experience of the vehicle, so as to at least partially solve the above-mentioned technical problems.

[0005] In order to achieve the above-mentioned object, according to a first aspect of the present application, a vehicle control method is provided, comprising:

[0006] Determining, based on first position information of the terminal device relative to the vehicle, a target vehicle device to be operated by a target object holding the terminal device on the vehicle, wherein the terminal device is in communication with the vehicle;

[0007] The target vehicle equipment is controlled.

[0008] Optionally, the step of acquiring the first location information includes:

[0009] The first position information between the vehicle and the terminal device is obtained according to the positioning signal sent by the terminal device.

[0010] Optionally, determining, based on the first position information of the terminal device relative to the vehicle, a target vehicle device to be operated by a target object holding the terminal device on the vehicle includes:

[0011] determining, based on the first position information, at least one candidate position information of the target object relative to the vehicle;

[0012] Based on the candidate position information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device is determined.

[0013] Optionally, the candidate location information includes at least one of the first location information, the second location information, and the third location information, and the method further includes:

[0014] According to the first position information of the terminal device relative to the vehicle, the second position information and / or third position information of the target object holding the terminal device relative to the vehicle is determined.

[0015] Optionally, the step of determining the second location information includes:

[0016] Determining, based on first position information between the vehicle and the terminal device, a first camera on the vehicle that captures the target object;

[0017] Based on the target image captured by the first camera, second position information of the target object relative to the vehicle is determined.

[0018] Optionally, the target image is a first image captured by the first camera when the environment in which the vehicle is located changes.

[0019] Optionally, the change in environment includes changing from an unmanned environment to a manned environment.

[0020] Optionally, determining second position information of the target object relative to the vehicle based on the target image captured by the first camera includes:

[0021] performing object recognition according to the target image;

[0022] According to the object recognition result, second position information of the target object relative to the vehicle is determined.

[0023] Optionally, the step of acquiring the third location information includes:

[0024] determining, based on the first position information and / or the second position information, at least one second camera on the vehicle that is within a field of view of the target object;

[0025] performing object recognition on the second images respectively captured by the second cameras to obtain target recognition results;

[0026] Determine third position information of the target object relative to the vehicle according to the target recognition result.

[0027] Optionally, performing object recognition on the second images respectively captured by the second cameras to obtain a target recognition result includes:

[0028] Object recognition is performed on the second images captured by each of the second cameras through a target recognition network to obtain target recognition results for each of the second images.

[0029] Optionally, determining third position information of the target object relative to the vehicle according to the target recognition result includes:

[0030] The target recognition results are compared, and third position information of the target object relative to the vehicle is determined according to the comparison result.

[0031] Optionally, determining third position information of the target object relative to the vehicle according to the comparison result includes:

[0032] In a case where the comparison result indicates that the same object is identified in the at least two target recognition results, third position information of the target object relative to the vehicle is determined based on the at least two target recognition results.

[0033] Optionally, the same object is a preset object having control authority over the vehicle.

[0034] Optionally, determining, based on the candidate location information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device includes:

[0035] determining target position information of the target object relative to the vehicle based on the candidate position information;

[0036] Based on the target position information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device is determined.

[0037] Optionally, determining the target position information of the target object relative to the vehicle based on the candidate position information includes:

[0038] In a case where there are at least two candidate position information matches, target position information of the target object relative to the vehicle is determined based on the at least two candidate position information.

[0039] Optionally, the matching of the at least two candidate position information includes: the at least two candidate position information are identical, or the position difference between the at least two candidate position information is within a preset difference range.

[0040] Optionally, determining, based on the target location information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device includes:

[0041] According to a preset mapping relationship between location information and vehicle equipment, the vehicle equipment corresponding to the target location information on the vehicle is determined to be the target vehicle equipment to be operated on the vehicle by the target object holding the terminal device.

[0042] Optionally, the controlling the target vehicle equipment includes:

[0043] Identifying the target operation intention of the target object on the target vehicle equipment;

[0044] Perform a target operation corresponding to the target operation intention on the target vehicle equipment.

[0045] Optionally, identifying the target operation intention of the target object on the target vehicle equipment includes:

[0046] Recognizing the gesture of the target object based on the image collected for the target object;

[0047] Based on the gesture action, a target operation intention of the target object on the target vehicle equipment is determined.

[0048] Optionally, before controlling the target vehicle equipment, the method further includes:

[0049] Outputting gesture action prompt information of the target vehicle equipment, wherein the gesture action prompt information is used to indicate the gesture action required to operate the target vehicle equipment.

[0050] According to a second aspect of the present application, there is provided a vehicle control device, comprising:

[0051] a determination module, configured to determine, based on first position information of the terminal device relative to the vehicle, a first target vehicle device to be operated by a target object holding the terminal device on the vehicle, wherein the terminal device is in communication with the vehicle;

[0052] A control module is used to control the first target vehicle equipment.

[0053] In a third aspect, this embodiment further provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0054] In a fourth aspect, this embodiment further provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the above method.

[0055] In a fifth aspect, this embodiment also provides a computer program product, including a computer program, which is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of the above method.

[0056] In a sixth aspect, this embodiment further provides a vehicle, on which at least one of a vehicle control device, an electronic device, a computer-readable storage medium, and a computer program product is provided.

[0057] In summary, in the embodiments of the present application, through the above technical solutions, the vehicle can directly control the target vehicle device to be operated by the target object holding the terminal device on the vehicle based on the position information of the terminal device relative to the vehicle. It can be seen that the present application can determine the target vehicle device that the user intends to operate and control it without user operation, avoiding the user's manual operation of the vehicle device, simplifying the control operation of the vehicle device, realizing intelligent control of the vehicle device, and effectively improving the user's vehicle operation experience.

[0058] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0060] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0061] Figure 1 is a first schematic diagram of a vehicle control process provided in an exemplary embodiment provided by this application;

[0062] Figure 2 is a second schematic diagram of a vehicle control process provided in an exemplary embodiment provided by this application;

[0063] Figure 3 is a third schematic diagram of a vehicle control process provided in an exemplary embodiment provided in this application;

[0064] Figure 4 is a schematic diagram of a vehicle control device provided in an exemplary embodiment of the present application;

[0065] Figure 5 Schematic diagram of the electronic device provided in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0067] With the continuous advancement of science and technology, the field of smart cars is experiencing rapid development. Smart cars are not just a means of transportation, but also a mobile intelligent space that integrates advanced technology and intelligent functions. Human-computer interaction technology plays a vital role in the development of smart cars.

[0068] Traditional methods of opening and closing car doors, tailgates, and front hoods mainly rely on physical buttons, remote controls, or car keys. However, these methods have certain limitations: (1) Traditional buttons and remote controls require manual operation to open or close the car doors, making the overall process relatively cumbersome and not intelligent enough; (2) Traditional methods are inconvenient, as passengers or drivers need to reach out and touch the door handle or buttons when they need to control the car doors, tailgate, or front hood.

[0069] With the development of science and technology and the maturity of human-computer interaction technology, biometric gesture recognition has become a hot research field and has been widely used in many fields, such as smart phones, televisions, and games. Biometric gesture recognition technology uses cameras to capture and analyze human gestures to achieve natural interaction with computers. This technology not only provides a more intuitive and convenient operation method, but also greatly enhances the user experience. In order to solve the above problems, based on biometric gesture recognition, this application proposes a vehicle control method, device, electronic device, computer-readable storage medium, computer program product and vehicle, such as Figure 1 As shown, it is designed to determine the relative position of the user and the vehicle body based on the relative positioning between the vehicle and the user's key, and perform precise position positioning based on the panoramic camera. Then, based on the relative position of the user and the vehicle, it judges the user's intention to open the device (front hood, tailgate, main driver's door, front passenger door, left rear door, right rear door), providing a new control opening and closing method for the smart car field and enhancing the human-computer interaction capability.

[0070] Meanwhile, traditional opening and closing control systems for vehicle doors, tailgates, and front hoods typically rely on contact-based operations such as physical buttons, remote controls, or car keys. However, this application utilizes biometric gesture recognition technology, allowing users to control the opening and closing of vehicle equipment through specific gestures without any physical contact, achieving true contactless operation and improving user convenience.

[0071] Specifically, for example, the vehicle control method in the present application can be applied to a vehicle, which can be a fuel vehicle, a plug-in hybrid vehicle, or a new energy vehicle, etc., and the present application does not make any specific limitations on this.

[0072] The vehicle control method in the embodiment of the present application is as follows: Figure 2 As shown, it can at least include the following steps:

[0073] S10, determining, based on first position information of the terminal device relative to the vehicle, a first target vehicle device to be operated by a target object holding the terminal device on the vehicle, wherein the terminal device is in communication with the vehicle;

[0074] In this embodiment, the terminal device can be a mobile phone, car key, or other device connected to the vehicle for communication, and there is no specific limitation on this, so that the vehicle can obtain the first position information of the terminal device relative to the vehicle, and based on the first position information, determine the first target vehicle device to be operated on the vehicle by the target object holding the terminal device.

[0075] Among them, the target object in this embodiment can be a user who intends to operate the vehicle equipment, and the vehicle equipment can include the front hood, tailgate, fast charging port cover, slow charging port cover, fuel filler cap, main driver's door, co-driver's door, left rear door, right rear door, etc. on the vehicle, and no longer list them one by one.

[0076] S20: Control the first target vehicle equipment.

[0077] After determining the target vehicle equipment that the target object intends to control, the vehicle can control the target vehicle equipment.

[0078] Therefore, in the embodiment of the present application, the vehicle can directly control the target vehicle device to be operated by the target object holding the terminal device on the vehicle based on the position information of the terminal device relative to the vehicle. It can be seen that the present application can determine the target vehicle device that the user intends to operate and control it without user operation, avoiding the user's manual operation of the vehicle device, simplifying the control operation of the vehicle device, realizing intelligent control of the vehicle device, and effectively improving the user's vehicle operation experience.

[0079] In one embodiment, the step of acquiring the first location information includes:

[0080] The first position information between the vehicle and the terminal device is obtained according to the positioning signal sent by the terminal device.

[0081] It should be noted that in this embodiment, taking the mobile phone held by the target object as an example, an ultra-wideband (UWB) positioning module can be installed in the vehicle and powered by the vehicle's uninterruptible power supply. When the vehicle is turned off and locked, the ultra-wideband positioning module can scan and search in real time in the low-power standby mode whether there is a mobile phone pre-bound to the vehicle in the nearby area (for example, within a radius of 3 meters). If there is no bound mobile phone in the nearby area, the ultra-wideband positioning module can determine that there is no user near the vehicle, thereby maintaining the low-power standby and search state to prevent the vehicle battery from running out of power.

[0082] When a user with a linked phone approaches the vehicle, the UWB module scans the nearby area for the phone that has been pre-linked to the vehicle. The module then determines that the user holding the linked phone has arrived near the vehicle, activates the short-range positioning function, and sends an UWB positioning request signal to the detected user's phone, requesting short-range positioning of the user.

[0083] The mobile phone can respond to the ultra-wideband positioning request signal sent by the bound vehicle and automatically feedback the ultra-wideband positioning signal based on the authorization instructions provided by the user during pre-binding. The ultra-wideband positioning signal can be a wireless carrier narrow pulse transmission data with a frequency bandwidth of more than 1GHz.

[0084] In this way, the ultra-wideband positioning module can determine the position of the mobile phone relative to the vehicle based on the ultra-wideband positioning signal (ie, the positioning signal in this embodiment) fed back by the user's mobile phone.

[0085] The ultra-wideband positioning module then sends the position data of the mobile phone relative to the vehicle to the vehicle's processor, and the processor can determine the first position information of the mobile phone relative to the vehicle in the vehicle coordinate system based on the position of the mobile phone relative to the vehicle.

[0086] In one embodiment, in the above S10, “determining, based on the first position information of the terminal device relative to the vehicle, a target vehicle device to be operated on the vehicle by a target object holding the terminal device” may include:

[0087] S101, determining at least one candidate position information of the target object relative to the vehicle based on the first position information;

[0088] S102: Based on the candidate position information, determine a target vehicle device to be operated on the vehicle by a target object holding the terminal device.

[0089] In this embodiment, after determining the position of the user relative to the vehicle (ie, the first position information in this embodiment), the vehicle can determine at least one candidate position information of the user holding the terminal device relative to the vehicle based on the first position information.

[0090] Furthermore, the target vehicle device that the user holding the terminal device intends to operate on the vehicle can be determined based on the candidate location information.

[0091] In one embodiment, the candidate location information includes at least one of the first location information, the second location information, and the third location information;

[0092] On this basis, the vehicle control method in this application may include:

[0093] S30: Determine second position information and / or third position information of a target object holding the terminal device relative to the vehicle based on the first position information of the terminal device relative to the vehicle.

[0094] In this embodiment, the vehicle can determine the second position information and / or third position information of the user holding the terminal device relative to the vehicle based on the first position information of the terminal device relative to the vehicle.

[0095] It can be understood that in this embodiment, the second position information and the third position information can be position information obtained in different ways. For example, the second position information can be position information calculated based on the environmental image captured by a single camera, and the third position information can be position information calculated based on multiple environmental images captured by multiple cameras, so as to improve the user positioning accuracy. The subsequent embodiments will explain in detail and will not be repeated here.

[0096] In one embodiment, the step of determining the second location information includes:

[0097] Determining, based on first position information between the vehicle and the terminal device, a first camera on the vehicle that captures the target object;

[0098] Based on the target image captured by the first camera, second position information of the target object relative to the vehicle is determined.

[0099] In this embodiment, after the vehicle determines the above-mentioned first position information, it can send the first position information to a camera installed at a corresponding position of the vehicle. For example, the vehicle's processor can determine the camera on the vehicle that is closest to the terminal device based on the first position information, and use the camera as the first camera. The first camera in this embodiment can specifically be a panoramic camera or other camera, and there is no specific limitation on this.

[0100] Furthermore, the second position information of the user relative to the vehicle can be determined based on the target image captured by the first camera.

[0101] In one embodiment, the target image is a first image captured by the first camera when the environment in which the vehicle is located changes.

[0102] In this embodiment, after the camera at the corresponding position receives the position information sent by the processor, it can trigger image data collection through vehicle environment change information. Once an environmental change is detected, the image triggered by the environmental change can be transmitted to the vehicle's domain controller. The domain controller recognizes the image and obtains the user's second position information relative to the vehicle.

[0103] In one embodiment, the change in environment includes changing from an unmanned environment to a manned environment.

[0104] In this embodiment, the change in environment may include a change from an unmanned environment to a manned environment. In addition, it may also include a person moving or changing his posture in a manned environment, etc., which is not specifically limited.

[0105] In a specific embodiment, the above-mentioned “determining second position information of the target object relative to the vehicle based on the target image captured by the first camera” may include:

[0106] performing object recognition according to the target image;

[0107] According to the object recognition result, second position information of the target object relative to the vehicle is determined.

[0108] It should be noted that, in this embodiment, the specific operation of object recognition may be segmenting the human body contour in the image to identify the user's human body data in the target image.

[0109] Correspondingly, the object recognition result may include recognized user body data, such as user's facial feature information.

[0110] The vehicle can then calculate the second position information of the user relative to the vehicle based on the user's body data and combined with the depth of field information of the camera. Please refer to the existing intelligent driving system and will not go into details here.

[0111] In one embodiment, the step of acquiring the third location information includes:

[0112] determining, based on the first position information and / or the second position information, at least one second camera on the vehicle that is within a field of view of the target object;

[0113] performing object recognition on the second images respectively captured by the second cameras to obtain target recognition results;

[0114] Determine third location information between the target object and the terminal device according to the target recognition result.

[0115] In this embodiment, after determining the position information of the user relative to the vehicle, the vehicle can determine at least one second camera on the vehicle that is within the field of view of the target object based on the first position information and / or the second position information.

[0116] Furthermore, the vehicle can perform object recognition on the second images respectively captured by each second camera to obtain a target recognition result, and determine the third position information between the target object and the terminal device based on the target recognition result.

[0117] In a specific embodiment, the above-mentioned “performing object recognition on the second images respectively captured by each of the second cameras to obtain a target recognition result” may include:

[0118] Object recognition is performed on the second images captured by each of the second cameras through a target recognition network to obtain target recognition results for each of the second images.

[0119] It should be noted that the target recognition network in this embodiment can be an R-CNN series network, a YOLO series network, etc., and there is no specific limitation on the type and structure of the target recognition network.

[0120] Among them, the training process of the target recognition network in this embodiment may include: first, based on the position of the detection frame in the sub-sample (specifically, it can be the environmental image of the vehicle environment collected historically), extracting the target features from the image features to obtain a target feature group, and then using each detection frame as input, performing deep learning on the target feature group to obtain an object recognition network across cameras.

[0121] On this basis, the target recognition network can process multiple input second images and output target recognition results corresponding to each second image.

[0122] The target recognition result in this embodiment may include the user's body data recognized from the image, such as the user's facial feature information.

[0123] In one embodiment, the above-mentioned “determining third position information of the target object relative to the vehicle based on the target recognition result” may include:

[0124] The target recognition results are compared, and third position information of the target object relative to the vehicle is determined according to the comparison result.

[0125] In this embodiment, after obtaining the target recognition results corresponding to the second images, the vehicle may compare the target recognition results and determine the third position information of the target object relative to the vehicle based on the comparison results.

[0126] In a specific embodiment, the above-mentioned “determining the third position information of the target object relative to the vehicle based on the comparison result” may include:

[0127] In a case where the comparison result indicates that the same object is identified in the at least two target recognition results, third position information of the target object relative to the vehicle is determined based on the at least two target recognition results.

[0128] It is understandable that in this embodiment, if multiple cameras recognize the same object, it can be determined that the object is the user who intends to operate the vehicle equipment.

[0129] The vehicle can then determine third position information of the target object relative to the vehicle based on any one of the at least two target recognition results and in combination with the depth information of the corresponding camera.

[0130] In a specific embodiment, the same object is a preset object having control authority over the vehicle.

[0131] In this embodiment, the vehicle allows the user to operate the vehicle equipment only when the vehicle determines, based on the target recognition result, that the object in the image is a preset object with vehicle control authority.

[0132] It is understandable that if the user wants to operate the vehicle equipment, he can enter his information, such as facial information, into the vehicle in advance.

[0133] In one embodiment, in S102 above, “determining, based on the candidate location information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device” may include:

[0134] S1021, determining target position information of the target object relative to the vehicle based on the candidate position information;

[0135] S1022: Based on the target location information, determine a target vehicle device to be operated on the vehicle by the target object holding the terminal device.

[0136] It should be noted that, in this embodiment, in combination with the above description, the candidate location information includes at least one of the first location information, the second location information, and the third location information.

[0137] On this basis, the vehicle can determine the target position information of the target object relative to the vehicle based on the candidate position information, wherein the target position information can be the candidate position information, and the target position information can also be the position information calculated from the candidate position information.

[0138] Furthermore, based on the above target location information, the target vehicle device that the user holding the terminal device intends to operate on the vehicle can be determined.

[0139] In a specific embodiment, in the above S1021, “determining the target position information of the target object relative to the vehicle based on the candidate position information” may include:

[0140] In a case where there are at least two candidate position information matches, target position information of the target object relative to the vehicle is determined based on the at least two candidate position information.

[0141] In this embodiment, when the vehicle detects that there are at least two candidate location information matches, for example, the second location information and the third location information are the same, any one of the at least two candidate location information can be used as the target location information of the user holding the terminal device relative to the vehicle.

[0142] In a specific embodiment, the matching of the at least two candidate position information includes: the at least two candidate position information are identical, or the position difference between the at least two candidate position information is within a preset difference range.

[0143] In this embodiment, the matching of at least two candidate position information may be that at least two candidate position information are identical, or the position difference between at least two candidate position information is within a preset difference range. The preset difference range can be set in advance and is not specifically limited to this.

[0144] In one embodiment, in S1022 above, “determining, based on the target location information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device” may include:

[0145] According to a preset mapping relationship between location information and vehicle equipment, the vehicle equipment corresponding to the target location information on the vehicle is determined to be the target vehicle equipment to be operated on the vehicle by the target object holding the terminal device.

[0146] In this embodiment, after determining the precise location of the user (ie, the target location information in this embodiment), the target vehicle device that the user intends to turn on is identified based on a preset mapping relationship between the vehicle device and the location information.

[0147] For example, if the user is directly in front of the vehicle, the target vehicle device is the front hood, and if the user is directly behind the vehicle, the target vehicle device is the tailgate, etc. In this application, the vehicle devices that can be opened include the front hood, tailgate, fast charging port cover, slow charging port cover, fuel filler cap, driver's door, passenger door, left rear door, right rear door, etc.

[0148] In one embodiment, in the above S20, “controlling the target vehicle equipment”, for example Figure 3 As shown, this may include:

[0149] S201, identifying the target operation intention of the target object on the target vehicle equipment;

[0150] S202: Perform a target operation corresponding to the target operation intention on the target vehicle equipment.

[0151] In this embodiment, the user may make a predefined gesture, which may represent the user's target operation intention on the target vehicle equipment.

[0152] Furthermore, the vehicle may perform a target operation corresponding to the target operation intention on the target vehicle device.

[0153] In a specific embodiment, the above S201 may include:

[0154] S2011, recognizing the gesture of the target object based on the image collected for the target object;

[0155] S2012: Determine the target operation intention of the target object on the target vehicle equipment based on the gesture action.

[0156] In this embodiment, the collected image may specifically be a biological posture image of the target object.

[0157] After the user completes the bio-gesture action according to the prompt, the camera will capture and save the bio-gesture image. The bio-gesture recognition model can then analyze the captured image, extract key bio-gesture features and motion trajectories, and match them with a predefined bio-gesture set. The model will classify the user's bio-gesture and identify the user's target operation intention. The model can adopt a system deep neural network algorithm to improve the accuracy and robustness of bio-gesture recognition.

[0158] Furthermore, control signals can be sent to actuators of vehicle equipment (such as electric limiters, closing motors, or unlocking motors). Upon receiving the signals, the actuators can execute corresponding actions, unlocking and opening the corresponding target vehicle equipment (front hood, tailgate, fast charging port cover, slow charging port cover, fuel filler cap, front door, front passenger door, left rear door, and right rear door). Simultaneously, this embodiment can monitor the status of relevant vehicle equipment in real time to ensure the smooth completion of operations, providing feedback or re-executing the operation when necessary.

[0159] For example, when it is necessary to close the corresponding vehicle equipment (front hood, tailgate, fast charging port cover, slow charging port cover, fuel filler cap, driver's door, passenger door, left rear door, right rear door), the user performs specific biometric gesture operations facing the panoramic camera, such as shaking head, shaking hands, specific gestures, etc.

[0160] After the user completes the biometric gesture, the camera captures and saves the biometric gesture image. The system then performs biometric gesture recognition. Upon successful biometric gesture recognition, the processor uses the vehicle's existing audio prompts, such as the horn, exterior speakers, or existing vehicle displays, such as projectors and headlights, to alert the user that the device is about to shut down.

[0161] It then sends a control signal to the vehicle's actuators (such as electric limiters, closing motors, or unlocking motors). Upon receiving the signal, the actuators execute the corresponding action, closing the relevant vehicle equipment (front hood, tailgate, fast charging port cover, slow charging port cover, fuel filler cap, front and rear doors, left and right rear doors).

[0162] In one embodiment, before the above-mentioned “controlling the target vehicle equipment”, the following steps may also be included:

[0163] Outputting gesture action prompt information of the target vehicle equipment, wherein the gesture action prompt information is used to indicate the gesture action required to operate the target vehicle equipment.

[0164] In combination with the above description, in this embodiment, the vehicle's existing sound prompt devices, such as speakers, external audio or the vehicle's existing display devices, such as projection lights, headlights, etc., can be used to remind the user to perform specific bio-gesture operations, such as nodding, waving, specific gestures, etc., to determine that the user needs to open relevant vehicle equipment (front hatch, tailgate, fast charging port cover, slow charging port cover, fuel filler cap, main driver's door, front passenger door, left rear door, right rear door).

[0165] After the user completes the bio-gesture action according to the prompt, the camera will capture and save the bio-gesture image, analyze the image to identify the user's target operation intention, and perform the target operation corresponding to the target operation intention on the target vehicle equipment. Please refer to the above embodiment and will not repeat it here.

[0166] In general, the embodiments of the present application provide a new method for controlling opening and closing in the field of smart cars, which enhances the human-computer interaction capability. Users can control the opening and closing of vehicle equipment through specific gestures without any physical contact, realizing true contactless operation and improving the convenience of use.

[0167] Accordingly, the embodiment of the present application also provides a vehicle control device, such as Figure 4 As shown, the device may include:

[0168] A determination module 1001 is configured to determine, based on first position information of a terminal device relative to the vehicle, a first target vehicle device to be operated by a target object holding the terminal device on the vehicle, wherein the terminal device is in communication with the vehicle;

[0169] The control module 1002 is configured to control the first target vehicle equipment.

[0170] Optionally, the acquiring of the first location information includes:

[0171] The first position information between the vehicle and the terminal device is obtained according to the positioning signal sent by the terminal device.

[0172] Optionally, the determining module 1001 is further configured to:

[0173] determining, based on the first position information, at least one candidate position information of the target object relative to the vehicle;

[0174] Based on the candidate position information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device is determined.

[0175] Optionally, the candidate position information includes at least one of the first position information, the second position information, and the third position information. The vehicle control device in the present application further includes:

[0176] The position determination module is used to determine the second position information and / or third position information of the target object holding the terminal device relative to the vehicle based on the first position information of the terminal device relative to the vehicle.

[0177] Optionally, the determining of the second location information includes:

[0178] Determining, based on first position information between the vehicle and the terminal device, a first camera on the vehicle that captures the target object;

[0179] Based on the target image captured by the first camera, second position information of the target object relative to the vehicle is determined.

[0180] Optionally, the target image is a first image captured by the first camera when the environment in which the vehicle is located changes.

[0181] Optionally, the change in environment includes changing from an unmanned environment to a manned environment.

[0182] Optionally, determining second position information of the target object relative to the vehicle based on the target image captured by the first camera includes:

[0183] performing object recognition according to the target image;

[0184] According to the object recognition result, second position information of the target object relative to the vehicle is determined.

[0185] Optionally, the acquiring of the third location information includes:

[0186] determining, based on the first position information and / or the second position information, at least one second camera on the vehicle that is within a field of view of the target object;

[0187] performing object recognition on the second images respectively captured by the second cameras to obtain target recognition results;

[0188] Determine third position information of the target object relative to the vehicle according to the target recognition result.

[0189] Optionally, performing object recognition on the second images respectively captured by the second cameras to obtain a target recognition result includes:

[0190] Object recognition is performed on the second images captured by each of the second cameras through a target recognition network to obtain target recognition results for each of the second images.

[0191] Optionally, determining third position information of the target object relative to the vehicle according to the target recognition result includes:

[0192] The target recognition results are compared, and third position information of the target object relative to the vehicle is determined according to the comparison result.

[0193] Optionally, determining third position information of the target object relative to the vehicle according to the comparison result includes:

[0194] In a case where the comparison result indicates that the same object is identified in the at least two target recognition results, third position information of the target object relative to the vehicle is determined based on the at least two target recognition results.

[0195] Optionally, the same object is a preset object having control authority over the vehicle.

[0196] Optionally, the determining module 1001 is further configured to:

[0197] determining target position information of the target object relative to the vehicle based on the candidate position information;

[0198] Based on the target position information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device is determined.

[0199] Optionally, the determining module 1001 is further configured to:

[0200] In a case where there are at least two candidate position information matches, target position information of the target object relative to the vehicle is determined based on the at least two candidate position information.

[0201] Optionally, the matching of the at least two candidate position information includes: the at least two candidate position information are identical, or the position difference between the at least two candidate position information is within a preset difference range.

[0202] Optionally, the determining module 1001 is further configured to:

[0203] According to a preset mapping relationship between location information and vehicle equipment, the vehicle equipment corresponding to the target location information on the vehicle is determined to be the target vehicle equipment to be operated on the vehicle by the target object holding the terminal device.

[0204] Optionally, the control module 1002 is further configured to:

[0205] Identifying the target operation intention of the target object on the target vehicle equipment;

[0206] Perform a target operation corresponding to the target operation intention on the target vehicle equipment.

[0207] Optionally, the control module 1002 is further configured to:

[0208] Recognizing the gesture of the target object based on the image collected for the target object;

[0209] Based on the gesture action, a target operation intention of the target object on the target vehicle equipment is determined.

[0210] Optionally, the vehicle control device in the present application further includes:

[0211] The prompt module is used to output gesture action prompt information of the target vehicle equipment, where the gesture action prompt information is used to indicate the gesture action required to operate the target vehicle equipment.

[0212] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0213] Accordingly, the embodiment of the present application further provides an electronic device, such as Figure 5 As shown, Figure 5 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1100 includes a processor 1101 having one or more processing cores, a memory 1102 having one or more computer-readable storage media, and a computer program stored in the memory 1102 and executable on the processor. The processor 1101 is electrically connected to the memory 1102. Those skilled in the art will understand that the vehicle structure shown in the figure does not constitute a limitation of the vehicle, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0214] Processor 1101 is the control center of electronic device 1100. It connects the various components of electronic device 1100 using various interfaces and lines. It executes various functions of electronic device 1100 and processes data by running or loading software programs and / or units stored in memory 1102 and calling data stored in memory 1102, thereby monitoring electronic device 1100 as a whole. Processor 1101 can be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc., and can implement or execute the various methods, steps, and logic blocks disclosed in the embodiments of this application.

[0215] In the embodiment of the present application, the processor 1101 in the electronic device 1100 loads instructions corresponding to one or more application processes into the memory 1102 according to the following steps, and the processor 1101 runs the application stored in the memory 1102 to implement various functions, such as:

[0216] Determining, based on first position information of the terminal device relative to the vehicle, a target vehicle device to be operated by a target object holding the terminal device on the vehicle, wherein the terminal device is in communication with the vehicle;

[0217] The target vehicle equipment is controlled.

[0218] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0219] Optional, such as Figure 5 As shown, the electronic device 1100 further includes: a touch screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. Among them, the processor 1101 is electrically connected to the touch screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107 respectively. Those skilled in the art will understand that Figure 5 The vehicle structure shown in the figure does not constitute a limitation to the vehicle, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0220] The touchscreen display 1103 can be used to display a graphical user interface (GUI) and receive user-generated operational commands generated by the GUI. The touchscreen display 1103 may include a display panel and a touch panel. The display panel can be used to display information input by or provided to the user, as well as various graphical user interfaces (GUIs) of the vehicle. These GUIs can be composed of graphics, text, icons, videos, or any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other form. The touch panel can be used to collect user touch operations on or near it (e.g., operations performed on or near the touch panel using a finger, stylus, or any other suitable object or accessory), generate corresponding operational commands, and execute corresponding programs in response to the operational commands. Optionally, the touch panel can include a touch display system and a touch controller. Among them, the touch display system detects the user's touch direction, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch display system, converts it into touch point coordinates, and then sends it to the processor 1101, and can receive commands sent by the processor 1101 and execute them. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 1101 to determine the type of touch event. The processor 1101 then provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 1103 to realize input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize input and output functions. That is, the touch display screen 1103 can also be used as part of the input unit 1106 to realize the input function.

[0221] The RF circuit 1104 may be used to transmit and receive RF signals, thereby establishing wireless communication with network devices or other vehicles through wireless communication, and transmitting and receiving signals with network devices or other vehicles.

[0222] Audio circuit 1105 can be used to provide an audio interface between the user and the vehicle through a speaker and microphone. Audio circuit 1105 converts received audio data into electrical signals and transmits them to the speaker, which then converts them into sound signals for output. The microphone, on the other hand, converts collected sound signals into electrical signals, which are received by audio circuit 1105 and converted into audio data. This audio data is then output to processor 1101 for processing, then transmitted via RF circuit 1104 to, for example, another vehicle, or to memory 1102 for further processing. Audio circuit 1105 may also include an earphone jack to allow communication between an external headset and the vehicle.

[0223] The input unit 1106 may be configured to receive input numbers, character information, or user feature information (such as fingerprint, iris, or facial information), and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control.

[0224] Power supply 1107 is used to supply power to various components of electronic device 1100. Optionally, power supply 1107 can be logically connected to processor 1101 via a power management device, thereby enabling the power management device to manage charging, discharging, and power consumption. Power supply 1107 can also include one or more DC or AC power supplies, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0225] although Figure 5 Not shown, the electronic device 1100 may further include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0226] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0227] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0228] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of computer programs. The computer programs can be loaded by a processor to execute any one of the vehicle control methods provided in the embodiments of the present application. The computer programs can execute the following steps of the vehicle control method:

[0229] Determining, based on first position information of the terminal device relative to the vehicle, a target vehicle device to be operated by a target object holding the terminal device on the vehicle, wherein the terminal device is in communication with the vehicle;

[0230] The target vehicle equipment is controlled.

[0231] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0232] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0233] Since the computer program stored in the computer-readable storage medium can execute any vehicle control method provided in the embodiments of the present application, the beneficial effects that can be achieved by any vehicle control method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0234] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0235] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0236] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0237] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0238] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0239] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0240] Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated communication signals and carrier waves.

[0241] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0242] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0243] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0244] The above are merely preferred embodiments of the present application and do not constitute any form of limitation to the present application. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A vehicle control method, characterized in that: include: Determine, based on the first position information of the terminal device relative to the vehicle, at least one candidate position information of the target object relative to the vehicle, the candidate position information including at least one of the first position information, the second position information, and the third position information; Based on the candidate location information, determining a target vehicle device to be operated by a target object holding the terminal device on the vehicle, wherein the terminal device has a communication connection with the vehicle; controlling the target vehicle equipment; The step of determining the second location information includes: Determining, based on first position information between the vehicle and the terminal device, a first camera on the vehicle that captures the target object; determining second position information of the target object relative to the vehicle based on the target image captured by the first camera; The step of acquiring the third location information includes: determining, based on the first position information and / or the second position information, at least one second camera on the vehicle that is within a field of view of the target object; performing object recognition on the second images respectively captured by the second cameras to obtain target recognition results; Determine third position information of the target object relative to the vehicle according to the target recognition result.

2. The vehicle control method according to claim 1, characterized in that: The step of acquiring the first location information includes: The first position information between the vehicle and the terminal device is obtained according to the positioning signal sent by the terminal device.

3. The vehicle control method according to claim 1, characterized in that: The target image is a first image captured by the first camera when the environment in which the vehicle is located changes.

4. The vehicle control method according to claim 3, characterized in that: The environment change includes changing from an unmanned environment to a manned environment.

5. The vehicle control method according to claim 1, characterized in that: The determining, based on the target image captured by the first camera, second position information of the target object relative to the vehicle includes: performing object recognition according to the target image; According to the object recognition result, second position information of the target object relative to the vehicle is determined.

6. The vehicle control method according to claim 1, characterized in that: The performing object recognition on the second images respectively captured by each of the second cameras to obtain a target recognition result includes: Object recognition is performed on the second images captured by each of the second cameras through a target recognition network to obtain target recognition results for each of the second images.

7. The vehicle control method according to claim 6, characterized in that: The determining, based on the target recognition result, third position information of the target object relative to the vehicle, includes: The target recognition results are compared, and third position information of the target object relative to the vehicle is determined according to the comparison result.

8. The vehicle control method according to claim 7, characterized in that: Determining third position information of the target object relative to the vehicle based on the comparison result includes: In a case where the comparison result indicates that the same object is identified in the at least two target recognition results, third position information of the target object relative to the vehicle is determined based on the at least two target recognition results.

9. The vehicle control method according to claim 8, characterized in that: The same object is a preset object having control authority over the vehicle.

10. The vehicle control method according to claim 1, wherein: The determining, based on the candidate position information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device, includes: determining target position information of the target object relative to the vehicle based on the candidate position information; Based on the target position information, a target vehicle device to be operated on the vehicle by a target object holding the terminal device is determined.

11. The vehicle control method according to claim 10, characterized in that: The determining, based on the candidate position information, target position information of the target object relative to the vehicle includes: In a case where there are at least two candidate position information matches, target position information of the target object relative to the vehicle is determined based on the at least two candidate position information.

12. The vehicle control method according to claim 11, characterized in that: The matching of the at least two candidate position information includes: the at least two candidate position information are identical, or the position difference between the at least two candidate position information is within a preset difference range.

13. The vehicle control method according to claim 10, characterized in that: The determining, based on the target position information, a target vehicle device to be operated on the vehicle by the target object holding the terminal device, includes: According to a preset mapping relationship between location information and vehicle equipment, the vehicle equipment corresponding to the target location information on the vehicle is determined to be the target vehicle equipment to be operated on the vehicle by the target object holding the terminal device.

14. The vehicle control method according to any one of claims 1 to 13, characterized in that: The controlling of the target vehicle equipment includes: Identifying the target operation intention of the target object on the target vehicle equipment; Perform a target operation corresponding to the target operation intention on the target vehicle equipment.

15. The vehicle control method according to claim 14, characterized in that: Identifying the target operation intention of the target object on the target vehicle equipment includes: Recognizing the gesture of the target object based on the image collected for the target object; Based on the gesture action, a target operation intention of the target object on the target vehicle equipment is determined.

16. The vehicle control method according to claim 15, characterized in that: Before controlling the target vehicle equipment, the method further includes: Outputting gesture action prompt information of the target vehicle equipment, wherein the gesture action prompt information is used to indicate the gesture action required to operate the target vehicle equipment.

17. A vehicle control device, characterized in that: The vehicle control device comprises: a determination module, configured to determine, based on first position information of a terminal device relative to a vehicle, at least one candidate position information of a target object relative to the vehicle, the candidate position information including at least one of the first position information, the second position information, and the third position information; and based on the candidate position information, determine a target vehicle device to be operated on the vehicle by the target object holding the terminal device, wherein the terminal device is in communication with the vehicle; The step of determining the second position information includes: determining a first camera on the vehicle that captures the target object based on the first position information between the vehicle and the terminal device; determining second position information of the target object relative to the vehicle based on the target image captured by the first camera; the step of acquiring the third position information includes: determining at least one second camera on the vehicle that is within the field of view of the target object based on the first position information and / or the second position information; performing object recognition on the second images respectively captured by each of the second cameras to obtain a target recognition result; and determining the third position information of the target object relative to the vehicle based on the target recognition result. A control module is used to control the target vehicle equipment.

18. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the method according to any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that The method comprises a computer program, and when the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the method according to any one of claims 1 to 16.

20. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the method according to any one of claims 1 to 16.

21. A vehicle, characterized in that: The vehicle is provided with at least one of the vehicle control device according to claim 17, the electronic device according to claim 18, the computer-readable storage medium according to claim 19, and the computer program product according to claim 20.

Images