Vehicle control method and device, electronic equipment and vehicle

Abstract

The invention discloses a vehicle control method and device, electronic equipment and a vehicle, and relates to the technical field of vehicle management, and the method comprises the steps: obtaining a target scene video of the vehicle, the target scene video being a video shot when the vehicle runs in a long-tail scene; and inputting the target scene video into a preset language model for analysis processing to obtain target decision information for controlling the vehicle, and controlling the vehicle to run according to the target decision information. According to the method and the device, a driver does not need to keep paying attention to the road surface all the time, the driving intensity of the driver is reduced, and the accident rate caused by the fact that the driver does not respond to a sudden long-tail scene in time is reduced.

Description

Technical Field

[0001] The present application relates to the field of vehicle management technology, and in particular to a vehicle control method, device, electronic equipment and vehicle. Background Art

[0002] With the rapid development and progress of science and technology, artificial intelligence is also developing rapidly. At present, the traditional automobile industry is vigorously developing unmanned driving technology based on the development of artificial intelligence. Unmanned driving technology plays an outstanding role in reducing the driving intensity of drivers and improving driving safety.

[0003] However, in real driving scenarios, driverless vehicles may encounter sudden obstacles or other long-tail scenarios that interfere with autonomous driving. Currently, when driverless vehicles deal with these unavoidable long-tail scenarios, manual decisions are still needed to ensure that the vehicle passes through these long-tail scenarios safely and smoothly.

[0004] However, this manual decision-making method requires the driver to always pay attention to the road in order to make quick decisions on sudden long-tail scenarios, which increases the driver's driving intensity and increases the accident rate caused by the driver's untimely response to sudden long-tail scenarios. Summary of the invention

[0005] In view of this, the present application provides a vehicle control method, device, electronic device and vehicle, so that the vehicle can automatically make decisions when facing long-tail scenarios, reducing the driver's driving intensity and reducing the accident rate caused by the driver's untimely response to sudden long-tail scenarios.

[0006] According to a first aspect of the present application, a vehicle control method is provided, comprising:

[0007] Acquire a target scene video of the vehicle, where the target scene video is a video shot when the vehicle is traveling in a long-tail scene;

[0008] The target scene video is input into a preset language model for analysis and processing to obtain target decision information for controlling the vehicle, and the vehicle is controlled to travel according to the target decision information.

[0009] According to a second aspect of the present application, a vehicle control device is provided, comprising:

[0010] An acquisition module, used to acquire a target scene video of a vehicle, wherein the target scene video is a video captured when the vehicle is traveling in a long-tail scene;

[0011] The analysis module is used to input the target scene video into a preset language model for analysis and processing, obtain target decision information for controlling the vehicle, and control the vehicle to travel according to the target decision information.

[0012] According to a third aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the vehicle control method described in the first aspect is implemented.

[0013] According to a fourth aspect of the present application, an electronic device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the vehicle control method described in the first aspect when executing the computer program.

[0014] In a fifth aspect, the present disclosure provides a vehicle, comprising: the device as described in the second aspect, the computer-readable storage medium as described in the third aspect, or the electronic device as described in the fourth aspect.

[0015] By means of the above technical solution, the present application provides a vehicle control method, device, electronic device and vehicle, which can obtain a target scene video of the vehicle, where the target scene video is a video taken when the vehicle is driving in a long-tail scene; the target scene video is input into a preset language model for analysis and processing, target decision information for controlling the vehicle is obtained, and the vehicle is controlled according to the target decision information. For the disclosed embodiment, by obtaining a target scene video in a long-tail scene, and analyzing and processing the target scene video using a preset language model, target decision information for controlling the vehicle is obtained, and the vehicle is controlled according to the target decision information, the driver does not need to keep paying attention to the road all the time, which reduces the driver's driving intensity and reduces the accident rate caused by the driver's untimely response to the sudden long-tail scene.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A schematic flow chart of a vehicle control method provided by an embodiment of the present disclosure;

[0020] Figure 2 A schematic flow chart of a vehicle control method provided by an embodiment of the present disclosure;

[0021] Figure 3 A schematic diagram of the principle flow of a preset language model training provided in an embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram of another principle flow of preset language model training provided in an embodiment of the present disclosure;

[0023] Figure 5 A schematic structural diagram of a vehicle control device provided in an embodiment of the present disclosure.

[0024] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to aid understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments of the present disclosure and the features therein may be combined with each other without conflict.

[0026] The following describes a vehicle control method, an apparatus, an electronic device, and a vehicle according to embodiments of the present disclosure with reference to the accompanying drawings.

[0027] In order to reduce the driving intensity of the driver and reduce the technical problem of the accident rate caused by the driver's untimely response to the sudden long-tail scene. This embodiment provides a vehicle control method, such as Figure 1 As shown, the method includes:

[0028] Step 101: Acquire a target scene video of a vehicle.

[0029] For the disclosed embodiments, the execution subject may be a control device or device of the vehicle, which may be configured on the vehicle side, etc. In this embodiment, the camera on the vehicle and / or the camera of some external device may be used to shoot the target scene video around the vehicle. For example, the external device may be a user's mobile phone, a surveillance camera of a parking lot, etc. The vehicle side may obtain the target scene video captured by these external devices through the server and / or wireless communication established with the external device.

[0030] The target scene video may be video information captured in a long-tail scene within a preset range around the vehicle. The preset range may be set according to actual needs, and the preset range may include a range size and an angle position relative to the vehicle, etc. For example, the preset range may be a circular area with the vehicle as the center and a preset length as the radius, or a square area with the vehicle as the center and a preset length as the side length, etc.

[0031] The “long tail” refers to edge cases in autonomous vehicles (AVs), which are possible scenarios with a low probability of occurring. These rare events can be easily missed and are therefore often missing in datasets.

[0032] Long-tail scenarios mainly refer to a wide variety of scenarios with low probability of occurrence or sudden situations, such as vehicles running red lights, pedestrians crossing the road, intersections with broken traffic lights, vehicles parked illegally on the roadside, etc. These scenarios are unpredictable, diverse, and difficult to handle, but they are one of the keys to the implementation of autonomous driving in Chinese cities.

[0033] Step 102: Input the target scene video into a preset language model for analysis and processing to obtain target decision information for controlling the vehicle, and control the vehicle driving according to the target decision information.

[0034] Among them, the preset language model can be a multimodal large language model (Multimodal Large Language Model, MLLM). The multimodal large language model is usually based on a large language model (Large Language Model, LLM), integrating other non-text modal information to complete various multimodal tasks, such as writing stories based on images and mathematical reasoning without optical character recognition.

[0035] For the embodiments of the present disclosure, a preset language model can be used to perform text description processing on the target scene video captured by the camera in the unmanned driving and / or the camera of some external device to obtain target scene description information describing the target scene video, and a preset language model can be used to perform labeling processing on the scene objects in the target scene video captured by the camera in the unmanned driving and / or the camera of some external device to obtain at least one labeled scene object information and its corresponding first weight value, wherein the target scene video contains at least one scene object, and the scene object is an instance object existing in the target scene video, for example, a traffic light, a crosswalk, an old man, a dog, etc. existing in the target scene video; the scene object information is the labeling result after recognition and classification of at least one scene object in the target scene video, for example, it is recognized that there are an old man and a dog in the target scene video, and the dog is classified as an animal for labeling, and the old man is classified as a pedestrian for labeling; the target scene description information can be a long-tail scene encountered by the vehicle during driving in detail in text language; the first weight value is the weight value of the influence of the scene object information on the vehicle driving behavior, and is positively correlated with the influence index of the vehicle driving behavior.

[0036] The scene object information with the largest first weight value among multiple first weight values, the attribute description information of the scene object information with the largest first weight value in the target scene video, and the impact degree index of the scene object information with the largest weight value on the vehicle are determined as target obstacle information, wherein the attribute description information may be a description of the moving position and direction of the scene object information with the largest first weight value in the target scene video; the impact degree index may be used to evaluate the impact degree of the scene object information on the vehicle speed, the turning impact degree or other indicators that affect the normal driving of the vehicle.

[0037] For example, the target scene description information of the target scene video is "the vehicle stops in the current lane and waits for the red light, and a puppy appears to cross the road in the left front." At this time, "red light, crosswalk, puppy" are identified, and after evaluation by the preset language model, it is determined that "puppy" has the greatest impact on the vehicle's driving behavior. The first weight value of "puppy" can be set to 0.8, and the first weight values ​​of "crosswalk" and "red light" are both set to 0.1. The moving position and direction of the "puppy" in the target scene video are described, and the moving position and direction of the "puppy" are obtained as "crossing the road from the left front"; the impact degree index of the "puppy" on the vehicle in the target scene video is described, and "affecting the vehicle's driving speed" is obtained.

[0038] A preset language model can be used to perform decision processing based on target scene description information and target obstacle information to obtain target decision information for controlling the vehicle, wherein the target decision information is used to decide how the vehicle should drive smoothly and safely through the current scene.

[0039] Correspondingly, the specific decision-making process can be described as follows: the preset language model can obtain multiple historical target scene videos in the training samples, as well as the historical decision information corresponding to the multiple historical target scene videos; multiple historical target scene videos are matched with the target scene video for similarity, and the historical target scene video with the highest matching degree and its corresponding historical decision information are determined; based on the historical target scene video and its corresponding historical decision information, the preset language model is used to generalize the vehicle decision through the target scene description information and the target obstacle information, and the target decision information for controlling the vehicle is obtained. Among them, the historical target scene video is the video information of the long-tail scene shot in a certain period of time in the past during the training model process; the historical decision information is the driving behavior decision information under the smooth and safe passage of the historical target scene.

[0040] For the embodiments of the present disclosure, the generalization ability of the model is utilized to select historical target scene videos that have a high degree of match with the current target scene video. By learning the historical decision information under the historical target scene videos, the target decision information under the target scene video can be accurately predicted, thereby passing the target scene smoothly and safely.

[0041] Among them, generalization refers to the ability of the model to make correct predictions using new, unseen data instead of the data used for training. In fact, this unseen data must be in the same distribution as the training data. Data that is not in the same distribution does not conform to the independent and identically distributed assumption. For example, to predict the probability of a data point in a Gaussian distribution, this data point may not be known during training, but this data point belongs to this Gaussian distribution, so its probability can still be accurately predicted. Therefore, the generalization of the model can learn the rules hidden behind the data. For data outside the learning set with the same rules, the trained network can also give appropriate outputs. This ability is called generalization ability. The model trained with training samples needs to make appropriate predictions for new samples, which is a manifestation of generalization ability.

[0042] For the disclosed embodiment, in order to determine whether the vehicle is traveling according to the target decision information, as a possible implementation method, the acquisition module in the vehicle can be used to obtain the initial parameter information of the vehicle. When the initial parameter information is adjusted to the target parameter information under the target decision information, it is determined that the vehicle is traveling according to the target decision information. Among them, the initial parameter information may include basic information of the vehicle such as vehicle speed, vehicle wheelbase, and vehicle track, which is used to reflect the vehicle structure and vehicle performance. The target parameter information may be basic information of the vehicle such as vehicle speed, vehicle wheelbase, and vehicle track corresponding to the driving behavior in the current scene in order to determine how the vehicle can smoothly and safely pass through the target scene.

[0043] In summary, according to a vehicle control method disclosed in the present invention, a target scene video of the vehicle can be obtained, and the target scene video is a video taken when the vehicle is driving in a long-tail scene; the target scene video is input into a preset language model for analysis and processing to obtain target decision information for controlling the vehicle, and the vehicle is controlled to travel according to the target decision information. For the disclosed embodiment, by obtaining a target scene video in a long-tail scene, and using a preset language model to analyze and process the target scene video, the target decision information for controlling the vehicle is obtained, and the vehicle is controlled to travel according to the target decision information, the driver does not need to keep paying attention to the road all the time, the driver's driving intensity is reduced, and the accident rate caused by the driver's untimely response to the sudden long-tail scene is reduced.

[0044] Further, as a refinement and extension of the above embodiment, in order to fully illustrate the specific implementation process of the method of this embodiment, this embodiment provides the following Figure 2 The specific method shown comprises the following steps:

[0045] Step 201: Acquire a target scene video of a vehicle.

[0046] Step 202: Input the target scene video into a preset language model for analysis and processing to obtain target decision information for controlling the vehicle, and control the vehicle to travel according to the target decision information.

[0047] For the embodiment of the present disclosure, the specific process of the training method of the preset language model can be described as follows:

[0048] Determine the sample scene video of the vehicle, as well as the sample scene description information, sample decision information and sample target obstacle information corresponding to the sample scene video; use the sample scene video, sample scene description information and sample target obstacle information as the input features of the preset language model, use the sample decision information as the training label of the preset language model, iteratively train the preset language model until the loss function of the preset language model is less than the preset loss function, and judge that the training of the preset language model is completed. Among them, the loss function is used to calculate the difference between the predicted value and the true value. The smaller the loss function value, the smaller the gap between the predicted value and the true value, and the better the robustness. When the loss function is less than the preset loss function threshold, the model training is completed; use the sample decision information as the true value of the preset language model; the preset loss function threshold is a value greater than 0 and less than 1. The closer the setting value of the preset loss function threshold is to 0, the higher the prediction accuracy of the preset language model. The specific value can be set according to the actual application scenario, and is not specifically limited here.

[0049] The sample scene description information may be a detailed description in text of the long-tail scenes encountered by the vehicle during driving in a certain period of time in the past; the sample decision information may be the driver's handling strategies when encountering such long-tail scenes in a certain period of time in the past, and serve as the true value for model training; the sample target obstacle information may include the sample scene object information that has the greatest impact on the vehicle's driving behavior, as well as the sample attribute description information of the sample scene object information in the sample scene video and the sample impact degree indicator on the vehicle.

[0050] For the embodiments of the present disclosure, there may be multiple methods for determining the sample scene video of a vehicle. As one possible implementation method, the historical scene video of the vehicle may be first obtained, and a video interval under a specific historical scene may be captured on the timeline corresponding to the historical scene video as the sample scene video.

[0051] Among them, the historical scene video can be the video information shot within a preset range around the vehicle within a certain period of time in the past; the timeline is to connect one or more aspects of events in series through Internet technology and according to the time sequence to form a relatively complete record system, and then present it to the user in the form of pictures and texts. The timeline can be used in different fields, and its biggest function is to systematize, complete, and accurately record past events. According to different fields and different things, the timeline can be divided into many types, such as for personal records, family records, historical records, etc.; there can be many specific historical scenes. In the embodiments of the present disclosure, the specific historical scenes are taken as long-tail scenes as an example to illustrate the technical solutions in the present disclosure, but it does not constitute a specific limitation on the technical solutions in the present disclosure.

[0052] For the embodiments of the present disclosure, there may be multiple methods for determining the sample target obstacle information corresponding to the sample scene video. As a possible implementation method, the sample scene objects in the sample scene video may be labeled based on the sample influence index of the sample decision information to obtain at least one labeled sample scene object information and its corresponding second weight value, wherein the labeling method may be multiple, including but not limited to the labeling method of the Transformer model, which is a model that uses the attention mechanism to increase the model training speed. For example, when a text is input, the text data will first pass through the Encoders module to encode the text, and then the encoded data will be passed to the Decoders module for decoding, and the translated text will be obtained after decoding.

[0053] The core logic of the attention mechanism is to shift from paying attention to everything to paying attention to key points, that is, to focus limited attention on key information, thereby saving resources and quickly obtaining the most effective information.

[0054] The sample scene video contains at least one sample scene object, and the second weight value is positively correlated with the sample influence index. The sample scene video contains at least one sample scene object, and the sample scene object is a historical instance object existing in the sample scene video; the sample scene description information can be a detailed description of the long-tail scene encountered by the vehicle during historical driving in text language; the second weight value is the historical weight value of the influence of the sample scene object information on the historical vehicle driving behavior, which is positively correlated with the sample influence index of the vehicle driving behavior.

[0055] The sample scene object information with the largest second weight value among multiple second weight values, the sample attribute description information of the sample scene object information with the largest second weight value in the sample scene video, and the sample influence degree index of the sample scene object information with the largest weight value on the vehicle are determined as sample target obstacle information, wherein the sample attribute description information may be a description of the moving position and direction of the sample scene object information with the largest second weight value in the sample scene video; the sample influence degree index may be used to evaluate the influence degree of the sample scene object information on the speed of historical vehicles, the turning influence degree or other indicators that affect the normal driving of vehicles.

[0056] The specific training process is as follows Figure 3 As shown, historical scene videos captured by cameras and / or cameras of some external devices within a certain period of time in the past can be obtained, as shown in the six video screenshots in the upper left corner, and video intervals under long-tail scenes can be intercepted under the time axis corresponding to the historical scene videos as sample scene videos, where the sample scene videos are as shown in Figure 4 Shown in the video screenshot in the upper left corner.

[0057] Get the sample scene description information corresponding to the sample scene video (i.e. Figure 3 or Figure 4 descriptive information under the key events in the Figure 3 or Figure 4 Descriptive information under the decision in the sample target obstacle information (i.e. Figure 3 or Figure 4 The object classification, state or attribute description, and description information of the impact on the vehicle under the obstacle id in the example are used, the sample scene video, the sample scene description information, and the sample target obstacle information are used as input features of the preset language model, and the sample decision information is used as the training label of the preset language model. The preset language model is iteratively trained until the loss function of the preset language model is less than the preset loss function, and the training of the preset language model is judged to be completed.

[0058] In summary, according to a vehicle control method disclosed in the present invention, a target scene video of the vehicle can be obtained, and the target scene video is a video taken when the vehicle is driving in a long-tail scene; the target scene video is input into a preset language model for analysis and processing to obtain target decision information for controlling the vehicle, and the vehicle is controlled to travel according to the target decision information. For the disclosed embodiment, by obtaining a target scene video in a long-tail scene, and using a preset language model to analyze and process the target scene video, the target decision information for controlling the vehicle is obtained, and the vehicle is controlled to travel according to the target decision information, the driver does not need to keep paying attention to the road all the time, the driver's driving intensity is reduced, and the accident rate caused by the driver's untimely response to the sudden long-tail scene is reduced.

[0059] Based on the above Figure 1 , Figure 2 The specific implementation of the method shown in the embodiment provides a vehicle control device, such as Figure 5 As shown, the device includes: an acquisition module 31, an analysis module 32;

[0060] An acquisition module 31 is used to acquire a target scene video of a vehicle, where the target scene video is a video captured when the vehicle is traveling in a long-tail scene;

[0061] The analysis module 32 is used to input the target scene video into a preset language model for analysis and processing, obtain target decision information for controlling the vehicle, and control the vehicle to travel according to the target decision information.

[0062] In a specific application scenario, the analysis module 32 can be used to use the preset language model to perform text description processing on the input target scene video to obtain target scene description information corresponding to the target scene video; use the preset language model to annotate the scene objects in the target scene video to obtain at least one annotated scene object information and its corresponding first weight value, wherein the target scene video contains at least one scene object, and the first weight value is the weight value of the degree of influence of the scene object information on the vehicle driving behavior, which is positively correlated with the degree of influence index of the vehicle driving behavior; the scene object information with the largest first weight value among multiple first weight values ​​is determined as the target obstacle information; and the preset language model is used to perform decision processing based on the target scene description information and the target obstacle information to obtain the target decision information for controlling the vehicle.

[0063] In a specific application scenario, the analysis module 32 can be used to determine the attribute description information of the scene object information corresponding to the maximum first weight value in the target scene video, and determine the influence degree index of the scene object information corresponding to the maximum first weight value on the vehicle; and determine the scene object information, the attribute description information and the influence degree index as the target obstacle information.

[0064] In a specific application scenario, the analysis module 32 can be used to obtain multiple historical target scene videos and historical decision information corresponding to the multiple historical target scene videos; perform similarity matching on the multiple historical target scene videos and the target scene video to determine the historical target scene video with the highest matching degree and its corresponding historical decision information; based on the historical target scene videos and their corresponding historical decision information, use the preset language model to generalize the decision of the vehicle through the target scene description information and the target obstacle information to obtain the target decision information for controlling the vehicle.

[0065] In a specific application scenario, the analysis module 32 can be used for training a preset language model, including: determining a sample scene video of the vehicle, and historical target scene description information, sample decision information, and sample target obstacle information corresponding to the historical target scene video; using the sample scene video, the sample scene description information, and the sample target obstacle information as input features of the preset language model, using the sample decision information as a training label of the preset language model, iteratively training the preset language model until the loss function of the preset language model is less than the preset loss function, and judging that the training of the preset language model is completed.

[0066] In a specific application scenario, the analysis module 32 can be used to obtain a historical scene video of the vehicle; and capture a video interval of a specific historical scene in the historical scene video as the sample scene video.

[0067] In a specific application scenario, the analysis module 32 can be used to label the sample scene objects in the sample scene video based on the sample influence index of the sample decision information, and obtain at least one labeled sample scene object information and its corresponding second weight value, wherein the sample scene video contains at least one sample scene object, and the second weight value is positively correlated with the sample influence index; the sample scene object information with the largest second weight value among multiple second weight values ​​is determined as the sample target obstacle information.

[0068] In a specific application scenario, the analysis module 32 can be used to determine the sample attribute description information of the sample scene object information corresponding to the maximum second weight value in the sample scene video, and determine the sample influence degree index of the sample scene object information corresponding to the maximum second weight value on the vehicle; and determine the sample scene object information, the sample attribute description information and the sample influence degree index as the sample target obstacle information.

[0069] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0070] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0071] Figure 6 is a block diagram of an electronic device 500 for implementing the above-mentioned vehicle control method according to an exemplary embodiment. For example, the electronic device 500 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0072] Reference Figure 6 , the electronic device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output (I / O) interface 512 , a sensor component 514 , and a communication component 516 .

[0073] The processing component 502 generally controls the overall operation of the electronic device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0074] The memory 504 is configured to store various types of data to support operations on the electronic device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0075] The power supply component 506 provides power to the various components of the electronic device 500. The power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 500.

[0076] The multimedia component 508 includes a screen that provides an output interface between the electronic device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the electronic device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0077] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the electronic device 500 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0078] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0079] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 can detect the open / closed state of the electronic device 500, the relative positioning of components, such as the display and keypad of the electronic device 500, and the sensor assembly 514 can also detect the position change of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, the orientation or acceleration / deceleration of the electronic device 500, and the temperature change of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0080] The communication component 516 is configured to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0081] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0082] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the instructions can be executed by a processor 520 of an electronic device 500 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0083] An embodiment of the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the vehicle control method described in the above embodiment of the present disclosure.

[0084] Based on the above electronic device, the embodiment of the present disclosure further provides a vehicle, which may specifically include: Figure 5 The image processing device shown, or the electronic device described above. The vehicle may be a new energy vehicle, or a traditional vehicle, etc.

[0085] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0086] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0087] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0088] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0089] It should be understood that the various parts of the embodiments of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0090] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0091] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0092] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present application.

Claims

1. A vehicle control method, characterized in that: include: Acquire a target scene video of the vehicle, where the target scene video is a video shot when the vehicle is traveling in a long-tail scene; The target scene video is input into a preset language model for analysis and processing to obtain target decision information for controlling the vehicle, and the vehicle is controlled to travel according to the target decision information.

2. The method according to claim 1, characterized in that The inputting the target scene video into a preset language model for analysis and processing to obtain target decision information for controlling the vehicle includes: Using the preset language model to perform text description processing on the input target scene video to obtain target scene description information corresponding to the target scene video; Using the preset language model to label the scene objects in the target scene video, obtaining at least one labeled scene object information and its corresponding first weight value, wherein the target scene video contains at least one scene object, and the first weight value is a weight value of the scene object information on the vehicle driving behavior influence index, which is positively correlated with the vehicle driving behavior influence index; Determine the scene object information having the largest first weight value among the plurality of first weight values ​​as target obstacle information; The preset language model is used to perform decision processing based on the target scene description information and the target obstacle information to obtain target decision information for controlling the vehicle.

3. The method according to claim 2, characterized in that The step of determining the scene object information having the largest first weight value among the plurality of first weight values ​​as the target obstacle information comprises: Determine the attribute description information of the scene object information corresponding to the maximum first weight value in the target scene video, and determine the influence degree index of the scene object information corresponding to the maximum first weight value on the vehicle; The scene object information, the attribute description information and the impact degree index are determined as the target obstacle information.

4. The method according to claim 2, characterized in that: The using the preset language model to perform decision processing based on the target scene description information and the target obstacle information to obtain target decision information for controlling the vehicle includes: Acquire multiple historical target scene videos and historical decision information corresponding to the multiple historical target scene videos; Performing similarity matching between the plurality of historical target scene videos and the target scene video, and determining the historical target scene video with the highest matching degree and its corresponding historical decision information; Based on the historical target scene video and its corresponding historical decision information, the preset language model is used to generalize the decision of the vehicle through the target scene description information and the target obstacle information to obtain target decision information for controlling the vehicle.

5. The method according to claim 1, characterized in that The training method of the preset language model comprises: Determining a sample scene video of the vehicle, and sample scene description information, sample decision information, and sample target obstacle information corresponding to the sample scene video; The sample scene video, the sample scene description information and the sample target obstacle information are used as input features of the preset language model, the sample decision information is used as the training label of the preset language model, and the preset language model is iteratively trained until the loss function of the preset language model is less than the preset loss function, and it is determined that the training of the preset language model is completed.

6. The method according to claim 5, characterized in that The determining of the sample scene video of the vehicle includes: Get the historical scene video of the vehicle; A video interval of a specific historical scene in the historical scene video is captured as the sample scene video.

7. The method according to claim 5, characterized in that Determining sample target obstacle information corresponding to the sample scene video includes: Based on the sample influence index of the sample decision information, a sample scene object in the sample scene video is labeled to obtain at least one labeled sample scene object information and its corresponding second weight value, wherein the sample scene video contains at least one sample scene object, and the second weight value is positively correlated with the sample influence index; The sample scene object information having the largest second weight value among the plurality of second weight values ​​is determined as the sample target obstacle information.

8. The method according to claim 7, characterized in that The step of determining the sample scene object information having the largest second weight value among the plurality of second weight values ​​as the sample target obstacle information comprises: Determine sample attribute description information of the sample scene object information corresponding to the maximum second weight value in the sample scene video, and determine a sample influence degree index of the sample scene object information corresponding to the maximum second weight value on the vehicle; The sample scene object information, the sample attribute description information and the sample influence degree index are determined as the sample target obstacle information.

9. A vehicle control device, characterized in that: include: An acquisition module is used to acquire a target scene video of a vehicle, where the target scene video is a video captured when the vehicle is traveling in a long-tail scene; The analysis module is used to input the target scene video into a preset language model for analysis and processing, obtain target decision information for controlling the vehicle, and control the vehicle to travel according to the target decision information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

12. A vehicle, characterized in that: include: The apparatus of claim 9, or the computer-readable storage medium of claim 10, or the electronic device of claim 11.

Images