Scene recognition method and apparatus

Abstract

The application discloses a scene recognition method and device, which can improve the calculation rate. The method comprises the following steps: acquiring vehicle driving information; processing the vehicle driving information by using a scene recognition model to determine the probability of each scene in multiple scenes, wherein the scene recognition model is a trained Gaussian mixture model; and determining the scene with the highest probability in the multiple scenes as a target scene.

Description

Technical Field

[0001] This application relates to the field of vehicles, and more specifically, to a scene recognition method and apparatus. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0003] Since autonomous driving algorithms cannot yet achieve end-to-end full-scene coverage, different algorithms can be used to handle different scenarios to improve adaptability to complex situations. Therefore, accurate identification of the driving environment is necessary.

[0004] Scene recognition can be achieved by manually setting rules, but this is costly. Neural network models can be used to process vehicle driving information to achieve scene recognition, but these models are time-consuming and inefficient. Summary of the Invention

[0005] This application provides a scene recognition method and apparatus that can improve the processing efficiency of scene recognition.

[0006] In a first aspect, a scene recognition method is provided, the method comprising: acquiring vehicle driving information; processing the vehicle driving information using a scene recognition model to determine the probability of each scene among multiple scenes, wherein the scene recognition model is a trained Gaussian mixture model; and determining the scene with the highest probability among the multiple scenes as the target scene.

[0007] By using the trained Gaussian mixture model to process vehicle driving information and determine the target scene, manual labor costs can be reduced, the time required for scene recognition can be decreased, and processing efficiency can be improved.

[0008] In conjunction with the first aspect, in some possible implementations, the scene recognition model includes multiple sub-models, each sub-model corresponding to one of the multiple scenes, and the sub-model corresponding to each scene is used to determine the probability of the scene based on the vehicle driving information.

[0009] By using Gaussian mixture models to process vehicle driving information, the probability of each scene can be calculated through the sub-models corresponding to each scene in the scene recognition model, thereby determining the scene with the highest probability as the target scene.

[0010] In conjunction with the first aspect, in some possible implementations, where at least one of the multiple scenarios is divided into multiple sub-scenarios, the method further includes: acquiring a second training data set, the second training data set including training vehicle driving information and training sub-scenarios corresponding to the training vehicle driving information, the multiple sub-scenarios including the training sub-scenarios; using the second training data set to train sub-models corresponding to the multiple sub-scenarios; and updating the scene recognition model according to the sub-models corresponding to the multiple sub-scenarios.

[0011] When at least one of the described scenarios is divided into multiple sub-scenarios, the scene recognition model can be updated by training only the sub-models corresponding to the multiple sub-scenarios. In other words, when at least one of the described scenarios is divided into multiple sub-scenarios, it is not necessary to retrain the sub-models corresponding to the unchanged scenarios, thus reducing the time and cost required for training.

[0012] In conjunction with the first aspect, in some possible implementations, the number of Gaussian kernels differs among the multiple sub-models.

[0013] Different sub-models can have different numbers of Gaussian kernels. The number of Gaussian kernels in the sub-models corresponding to each scene can be set according to the complexity of scene recognition. For scenes with lower complexity, the number of Gaussian kernels in the sub-model can be reduced, thereby reducing the time required to train the sub-model, reducing its complexity, shortening the time required for probability calculation using the sub-model, and improving the efficiency of scene recognition.

[0014] In conjunction with the first aspect, in some possible implementations, the vehicle driving information includes multiple physical information, and each sub-model is used to process at least one physical information corresponding to the sub-model in the vehicle driving information. Among the multiple sub-models, the first sub-model and the corresponding at least one physical information are different from the at least one physical information corresponding to the second sub-model.

[0015] At least one piece of physical information can differ between different sub-models. The physical information input to each sub-model corresponding to a scene can be set according to the complexity of scene recognition. For scenes with lower complexity, the amount of physical information input to the sub-model can be reduced, thereby reducing the time required to train the sub-model, reducing its complexity, shortening the time required for probability calculations using the sub-model, and improving the efficiency of scene recognition. Reducing the amount of physical information input to the sub-model can also be understood as reducing the dimensionality of the sub-model's input information. For example, based on whether the physical parameter affects the judgment of each scene, it can be determined whether the input information of the sub-model corresponding to each scene includes that physical parameter.

[0016] In conjunction with the first aspect, in some possible implementations, when multiple first scenarios in the plurality of scenarios are merged into a second scenario, the probability of the second scenario is the sum of the probabilities of the plurality of first scenarios.

[0017] When multiple first scenes are merged into a second scene, there is no need to modify the scene recognition model. The sum of the probabilities of the multiple first scenes determined by the scene recognition model can be used as the probability of the second scene.

[0018] In conjunction with the first aspect, in some possible implementations, the target scenario belongs to different target types in multiple application modules, the multiple application modules correspond to multiple scenario structure relationships, and the scenario structure relationship corresponding to each application module is used to indicate the type to which each scenario belongs in the application module.

[0019] Application modules can be used to process based on the type of the target scene. The multiple application modules correspond to multiple scene structure relationships, and each application module's scene structure relationship indicates the type of each scene among the multiple scenes. Therefore, different application modules can utilize the scene structure relationship corresponding to that application module to determine the target type of the target scene within that application module. Consequently, the scene recognition results can be applied to various application modules without requiring multiple scene recognitions to suit different modules, thus improving the applicability of the scene recognition results.

[0020] In a second aspect, a scene recognition device is provided, comprising modules for performing the method described in the first aspect or any implementation thereof.

[0021] Thirdly, an electronic device is provided, including a memory and a processor, the memory being used to store program instructions; when the program instructions are executed in the processor, the processor is used to perform the method described in the first aspect or any implementation thereof.

[0022] The processor mentioned in the third aspect above can be a central processing unit (CPU) or a combination of a CPU and a neural network processing processor. The neural network processing processor can include a graphics processing unit (GPU), a neural network processing unit (NPU), and a tensor processing unit (TPU), etc. The TPU is a dedicated integrated circuit developed by Google for a fully customized artificial intelligence accelerator for machine learning.

[0023] Fourthly, a computer-readable medium is provided that stores program code for execution by a device, the program code including methods for performing the first aspect or any implementation thereof.

[0024] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the method described in the first aspect or any implementation thereof.

[0025] In a sixth aspect, a chip is provided, the chip including a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface and executes the method described in the first aspect or any implementation thereof.

[0026] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in any of the implementations of the first aspect.

[0027] The aforementioned chip can be a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). Attached Figure Description

[0028] Figure 1 This is a functional block diagram of a vehicle to which this application embodiment applies.

[0029] Figure 2 This is a schematic diagram of an autonomous driving system applicable to the embodiments of this application.

[0030] Figure 3 This is a schematic diagram of a tree-structured scene model.

[0031] Figure 4 This is a schematic flowchart of a scene recognition method.

[0032] Figure 5 This is a schematic flowchart of a scene recognition method provided in an embodiment of this application.

[0033] Figure 6 This is a schematic diagram showing the vehicle's location indicated by the vehicle's positioning data.

[0034] Figure 7 This is a diagram illustrating the addition of a scene.

[0035] Figure 8 This is a schematic diagram of scene relationship information in a tree structure.

[0036] Figure 9 This is a schematic diagram of a scene structure relationship provided in an embodiment of this application.

[0037] Figure 10 This is a schematic diagram of a scene recognition device provided in an embodiment of this application.

[0038] Figure 11 This is a schematic structural diagram of another scene recognition device provided in the embodiments of this application. Detailed Implementation

[0039] The electromechanical braking methods and / or devices provided in this application can be applied to various types of vehicles. These methods and / or devices can be applied to manual driving, driver assistance, and autonomous driving. The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings.

[0040] Figure 1 This is a functional block diagram of a vehicle applicable to an embodiment of this application. The vehicle 100 can be a manually driven vehicle, or the vehicle 100 can be configured for fully or partially automated driving modes.

[0041] In one example, vehicle 100 can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human intervention, determine the possible behaviors of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of the other vehicle performing the possible behavior, and control vehicle 100 based on the determined information. When vehicle 100 is in autonomous driving mode, vehicle 100 can be set to operate without human interaction.

[0042] The vehicle 100 may include various subsystems, such as a sensing system 120, a computer system 150, and a user interface 170.

[0043] Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0044] The sensing system 120 may include several sensors that sense information about the environment surrounding the vehicle 100.

[0045] For example, the sensing system 120 may include a positioning system 121 (e.g., a global positioning system, BeiDou system, or other positioning system), an inertial measurement unit (IMU) 122, a radar 123, a laser rangefinder 124, a camera 125, and a vehicle speed sensor 126. The sensing system 120 may also include sensors from the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the autonomous vehicle 100.

[0046] The positioning system 121 can be used to estimate the geographical location of the vehicle 100.

[0047] For example, IMU 122 can be used to sense changes in the position and orientation of vehicle 100 based on inertial acceleration. In one embodiment, IMU 122 can be a combination of an accelerometer and a gyroscope.

[0048] For example, radar 123 can use radio information to sense objects in the surrounding environment of vehicle 100. In some embodiments, in addition to sensing objects, radar 123 can also be used to sense the speed and / or direction of travel of objects.

[0049] For example, the laser rangefinder 124 can use a laser to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 124 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0050] For example, camera 125 can be used to capture multiple images of the surrounding environment of vehicle 100. For example, camera 125 can be a still camera or a video camera.

[0051] For example, the vehicle speed sensor 126 can be used to measure the speed of the vehicle 100. For instance, the vehicle speed can be measured in real time. The measured vehicle speed can be transmitted to the computer system 150.

[0052] For example, some or all of the functions of vehicle 100 may be controlled by computer system 150, wherein computer system 150 may include at least one processor 151 that executes instructions 153 stored in a non-transitory computer-readable medium, such as memory 152. Computer system 150 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed manner.

[0053] For example, processor 151 can be any conventional processor, such as a commercially available central processing unit (CPU).

[0054] Alternatively, the processor can be a dedicated device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1 The illustrations functionally depict a processor, memory, and other components of a computer within the same block; however, those skilled in the art will understand that the processor, computer, or memory may actually include multiple processors, computers, or memories that may or may not be stored in the same physical enclosure. For example, memory may be a hard disk drive or other storage media located in an enclosure different from that of the computer. Therefore, references to a processor or computer will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration assemblies, may each have their own processor that performs calculations only relevant to the component's specific function.

[0055] In the various aspects described herein, the processor may be located remotely from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor located within the vehicle, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0056] In some embodiments, memory 152 may contain instructions 153 (e.g., program logic) that can be executed by processor 151 to perform various functions of vehicle 100.

[0057] For example, in addition to instruction 153, memory 152 may also store data, such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information can be used by vehicle 100 and computer system 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0058] like Figure 1 As shown, the user interface 170 can be used to provide information to or receive information from a user of the vehicle 100. Optionally, the user interface 170 may include one or more input / output devices, such as a wireless communication system, an onboard computer, a microphone, and a speaker.

[0059] In embodiments of this application, computer system 150 may control the functions of vehicle 100 based on input received from sensing system 120 and user interface 170. In some embodiments, computer system 150 is operable to provide control over a number of aspects of vehicle 100 and its subsystems.

[0060] Alternatively, one or more of these components may be installed separately from or associated with vehicle 100. For example, memory 152 may exist partially or completely separately from vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0061] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application.

[0062] Optionally, vehicle 100 may be an autonomous vehicle traveling on a road, capable of identifying objects in its surrounding environment to determine adjustments to its current speed. These objects may be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object may be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the autonomous vehicle can be determined.

[0063] Optionally, the vehicle 100 or a computing device associated with the vehicle 100 (such as...) Figure 1 The computer system 150 and processor 151 can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.).

[0064] Optionally, since each identified object depends on the behavior of others, the behavior of a single identified object can also be predicted by considering all identified objects together. Vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine, based on the predicted behavior of the objects, that the vehicle will need to adjust to a steady state (e.g., accelerate, decelerate, or stop). In this process, other factors can also be considered in determining the speed of vehicle 100, such as the lateral position of vehicle 100 on the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0065] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0066] The aforementioned vehicle 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0067] In one possible implementation, the above Figure 1 The vehicle 100 shown may be an autonomous vehicle. The autonomous driving system is described in detail below.

[0068] Figure 2 This is a schematic diagram of an autonomous driving system applicable to an embodiment of this application. For example... Figure 2 The illustrated autonomous driving system includes a computer system 201, which includes a processor 203 coupled to a system bus 205. The processor 203 can be one or more processors, each of which may include one or more processor cores. A display adapter 207 drives a display 209, which is coupled to the system bus 205. The system bus 205 is coupled to an input / output (I / O) bus 213 via a bus bridge 211, and an I / O interface 215 is coupled to the I / O bus. The I / O interface 215 communicates with various I / O devices, such as input devices 217 (e.g., keyboard, mouse, touchscreen), and media trays 221 (e.g., CD-ROM, multimedia interfaces). A transceiver 223 can send and / or receive radio communication information, and a camera 255 can capture static and dynamic digital video images. The interface connected to the I / O interface 215 may be a USB port 225.

[0069] The processor 203 can be any conventional processor, such as a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination thereof.

[0070] Alternatively, processor 203 may be a dedicated device such as an application-specific integrated circuit (ASIC); processor 203 may be a neural network processor or a combination of a neural network processor and the aforementioned conventional processor.

[0071] Optionally, in some embodiments, the computer system 201 may be located remotely from the autonomous vehicle and may communicate wirelessly with the autonomous vehicle. In other aspects, some processes described in this application are executed on a processor located within the autonomous vehicle, while others are executed by a remote processor, including taking actions necessary to perform a single manipulation.

[0072] Computer system 201 can communicate with software deployment server 249 via network interface 229. Network interface 229 can be a hardware network interface, such as a network interface card (NIC). Network 227 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Optionally, network 227 can also be a wireless network, such as a WiFi network or a cellular network.

[0073] like Figure 2 As shown, the hard disk drive interface is coupled to the system bus 205, the hardware driver interface 231 can be connected to the hard disk drive 233, and the system memory 235 is coupled to the system bus 205. Data running in the system memory 235 may include the operating system 237 and applications 243. The operating system 237 may include an interpreter (shell) 239 and a kernel 241. The shell 239 is an interface between the user and the operating system kernel. The shell can be the outermost layer of the operating system; the shell can manage the interaction between the user and the operating system, such as waiting for user input, interpreting user input for the operating system, and processing various operating system outputs. The kernel 241 may consist of the parts of the operating system used to manage memory, files, peripherals, and system resources. Interacting directly with the hardware, the operating system kernel typically runs processes and provides inter-process communication, CPU time slice management, interrupts, memory management, I / O management, etc. Application 243 includes programs related to controlling autonomous driving, such as programs managing the interaction between the autonomous vehicle and obstacles on the road, programs controlling the autonomous vehicle's route or speed, and programs controlling the interaction between the autonomous vehicle and other autonomous vehicles on the road. Application 243 also exists on the system of software deployment server 249. In one embodiment, when autonomous driving-related programs 247 need to be executed, computer system 201 can download the application from software deployment server 249.

[0074] For example, application 243 could also be a program that allows autonomous vehicles to interact with lane lines on the road, meaning a program that can track lane lines in real time.

[0075] For example, application 243 could also be a program that controls autonomous vehicles to perform automatic parking.

[0076] For example, operating system 237 could be an automotive operating system (AOS). The automotive operating system could be powered by a mobile datacenter (MDC) intelligent driving computing platform located in system memory 235. System memory 235 could also store an autonomous driving system framework initiative (ADSFI), which includes programs that provide a runtime environment for applications.

[0077] For example, sensor 253 may be associated with computer system 201 and may be used to detect the environment around computer 201.

[0078] For example, sensor 253 can detect lanes on the road, such as lane lines, and can track changes in lane lines within a certain range in front of the vehicle in real time as the vehicle moves (e.g., while driving). As another example, sensor 253 can detect animals, cars, obstacles, and pedestrian crossings. Furthermore, the sensor can also detect the environment surrounding these objects, such as the environment around the animal (e.g., other animals nearby), weather conditions, and ambient light levels.

[0079] Alternatively, if the computer 201 is located on an autonomous vehicle, the sensors may be cameras, infrared sensors, chemical detectors, microphones, etc.

[0080] For example, in a lane tracking scenario, sensor 253 can be used to detect lane lines in front of the vehicle, enabling the vehicle to perceive lane changes during travel and plan and adjust the vehicle's driving in real time accordingly.

[0081] For example, in an automatic parking scenario, sensor 253 can be used to detect the size or position of parking spaces and surrounding obstacles around the vehicle, thereby enabling the vehicle to perceive the distance between the parking spaces and surrounding obstacles, perform collision detection during parking, and prevent the vehicle from colliding with obstacles.

[0082] In one example Figure 1The computer system 150 shown can also receive information from other computer systems or transfer information to other computer systems. Alternatively, sensor data collected from the sensing system 120 of the vehicle 100 can be transferred to another computer for processing.

[0083] Since autonomous driving algorithms cannot yet achieve end-to-end full-scene coverage, different algorithms can be used to handle different scenarios to improve adaptability to complex situations. Therefore, accurate identification of the driving environment is necessary.

[0084] Neural network models can be used to recognize scenes.

[0085] Vehicle driving information can be input into a neural network model, which will then process the information and output the target scene.

[0086] When additional scenarios are required, the neural network model needs to be retrained, which is costly.

[0087] Scene recognition can also be achieved using a tree-structured scene model.

[0088] Figure 3 This is a schematic diagram of a tree-structured scene model.

[0089] The tree-structured scene model comprises multiple layers, with the first layer serving as the root node, which can be understood as the default initial scene. Each subsequent layer of the tree-structured scene model progressively subdivides the complex scene. For example, the second layer divides the scene into highways and urban roads, while the third layer further subdivides highways into highway exits, highway entrances, and merges highway lanes, and further subdivides urban roads into sidewalks, intersections, etc. This scene division method can clearly express the hierarchical structure of the scene and can adapt to the needs of autonomous driving algorithms for classifying complex scenes.

[0090] A high-speed node can be understood as the parent node of three nodes: high-speed exit, high-speed entrance, and high-speed lane merging. All three nodes are child nodes of the high-speed node.

[0091] Based on vehicle driving information, the activated nodes in each layer can be determined sequentially. Within the child nodes of an activated node, the activated node itself is determined based on the vehicle driving information.

[0092] On the one hand, it is necessary to determine the activated nodes in each layer sequentially, which requires a long computation time. On the other hand, when the scene classification needs to be adjusted, the activation conditions of nodes in each layer need to be modified, which is costly.

[0093] Figure 4 This is a schematic flowchart of a scene recognition method.

[0094] Based on vehicle driving information, it can be sequentially determined whether the vehicle is in an emergency takeover scenario, a reversing into a parking space scenario, a parallel parking scenario, an intersection scenario, or a stop-and-go scenario. If the vehicle driving information does not meet the criteria for any of the above scenarios, it can be considered a default scenario.

[0095] To reduce computational load, a check to determine whether the current scenario has ended can be added after the emergency takeover check.

[0096] Emergency takeover scenarios have the highest priority and can interrupt the execution of the current scenario at any time, thereby improving the safety of the autonomous driving system.

[0097] The judgment conditions for each scenario need to be pre-set manually based on the vehicle driving rules. When there are a large number of scenarios, it is difficult to manually set the vehicle driving rules for each scenario as the judgment conditions. Therefore, this method is not applicable when there are a large number of scenarios.

[0098] To address the aforementioned issues, this application provides a scene recognition method.

[0099] Figure 5 This is a schematic flowchart of a scene recognition method provided in an embodiment of this application. Method 500 includes steps S510 to S530.

[0100] In S510, vehicle driving information is obtained.

[0101] In S520, the vehicle driving information is processed using a scene recognition model to determine the probability of each scene among multiple scenes. The scene recognition model is a trained Gaussian mixture model.

[0102] The sum of the probabilities of the multiple scenes determined by the scene recognition model is 100%.

[0103] In S530, among the multiple scenarios, the scenario with the highest probability is determined as the target scenario.

[0104] Using S510 to S530, the trained Gaussian mixture model is used as the scene recognition model. Based on vehicle driving information, the target scene with the highest probability is determined from multiple scenarios. The scene recognition model is trained and can adapt to situations with a large number of scenes.

[0105] Vehicle driving information can include perception data and / or positioning data. Perception data can include data output by sensors, or data obtained by processing the sensor output. For example, images captured by cameras in the vehicle can be processed using a neural network model to determine whether the vehicle is on a highway or an urban road. Perception data can include road type information, which can be used to indicate whether the vehicle is on a highway or an urban road.

[0106] When vehicle driving information includes at least one of perception data and positioning data, the current scene of the vehicle can be determined.

[0107] Vehicle driving information can also include planning and control data. Based on the planning and control data, the scenarios the vehicle is about to enter can be predicted.

[0108] like Figure 6 As shown, the vehicle's positioning data indicates its current position as P0. Based on position P0, the vehicle speed measured by sensors in the perception data, and the path planning indicated by the planning and control data, the vehicle's position after a preset time can be determined. The path planning indicated by the planning and control data may, for example, include turning left at the next intersection.

[0109] If the vehicle speed is low, its position after a preset time might be P1. If the vehicle speed is high, its position after a preset time might be P2. In other words, the vehicle's scenario after a preset time can be predicted based on current vehicle driving data. It can be determined, based on perception data, positioning data, and planning and control data, that the vehicle's current scenario is a non-intersection scenario, and that the vehicle will enter an intersection scenario after a preset time.

[0110] Different autonomous driving algorithms can be used to handle different scenarios. Predicting the scenario the vehicle is about to enter provides preparation time for switching autonomous driving algorithms, allowing for a pre-emptive switch to the algorithm appropriate for the upcoming scenario. For example, if the current driving scenario is determined to be intersection driving, the autonomous driving algorithm corresponding to the intersection entrance can be switched in advance. Alternatively, if it is determined that the vehicle will enter the intersection after a preset time, the algorithm corresponding to the intersection entrance can also be switched in advance. The autonomous driving algorithm corresponding to the intersection entrance allows the vehicle to travel at a lower speed. In other words, the vehicle can slow down before entering the intersection.

[0111] The scene recognition model is a trained Gaussian mixture model (GMM). Unlike GMMs, which rely on manual pre-setting based on vehicle driving rules, scene recognition models reduce labor costs and increase the applicability of scene recognition. They can be applied to complex vehicle driving environments and remain usable even with a large number of scenes.

[0112] Each sub-model of the Gaussian mixture model can be represented as:

[0113]

[0114] Where K is the number of Gaussian kernels in this sub-model, a k Let θ be the weight corresponding to the k-th Gaussian kernel. k This represents the parameters of the k-th Gaussian kernel. Each Gaussian kernel's parameters include the mean and variance. φ represents a normal distribution, and x represents the input data for that sub-model. Vehicle driving information includes information from the input data of each sub-model.

[0115] The trained Gaussian mixture model is used as the scene recognition model for scene recognition. Based on the characteristics of the Gaussian mixture model, when the number of scenes increases, there is no need to retrain the entire scene recognition model.

[0116] The scene recognition model includes multiple sub-models, each corresponding one-to-one with a specific scene. Each sub-model for a scene determines the probability of that scene based on vehicle driving information. Adding a scene immediately divides the existing scene into multiple sub-scenes.

[0117] When at least one scene is divided into multiple sub-scenes, only the sub-models corresponding to those multiple sub-scenes can be trained without modifying the sub-models corresponding to other unchanged scenes.

[0118] Specifically, when at least one scene is divided into multiple sub-scenes, a first training data set can be obtained. This first training data set includes training vehicle driving information and the corresponding training sub-scenes. The multiple sub-scenes include the training sub-scenes. Using the first training data set, sub-models for these multiple sub-scenes are trained. Subsequently, the scene recognition model can be updated based on the sub-models corresponding to these multiple sub-scenes.

[0119] Before obtaining scene segmentation information, the scene recognition model includes N sub-models, each of which corresponds one-to-one with one of the N scenes.

[0120] like Figure 7As shown, when it is necessary to add a yield sign at an intersection, the scene division information can indicate that scene i+1 (i.e., scene without a sign at the intersection) in the N scenes is divided into scene i+1′ (yield sign at the intersection) and scene i+1″ (no sign at the intersection).

[0121] Only the sub-models corresponding to scene i+1′ (yield sign at intersection) and scene i+1″ (no sign at intersection) need to be trained. The sub-models for each of scenes 1 to i and scenes i+2 to N do not need adjustment; that is, there is no need to retrain these sub-models. Adding the scene "yield sign at intersection" has no impact on the sub-models for scenes other than "no sign at intersection i+1". The sum of the probabilities of scene i+1′ and scene i+1″ equals the probability of the original scene i+1. In other words, when re-dividing some existing scenes, only the sub-models corresponding to the sub-scenes after this re-division need to be trained; the sub-models for other scenes outside this re-division do not need to be retrained.

[0122] like Figure 7 The change shown is the opposite. The case where the yield sign at intersection i+1′ needs to be removed can be understood as merging scene i+1′ and scene i+1″ into scene i+1. The probability of the new scene intersection having no sign is the sum of the probability of the scene intersection having a yield sign and the probability of the original scene having no sign. Therefore, when deleting the original scene (such as the yield sign at intersection i+1′), that is, merging multiple original scenes, there is no need to train the scene recognition model. When multiple first scenes in a scene merge are combined into a second scene, the probability of the second scene is the sum of the probabilities of the multiple first scenes combined.

[0123] In some cases, some original scenes can be re-divided, while others can be merged. Then, the sub-models corresponding to the re-divided scenes can be retrained, and the sum of the probabilities of the other original scenes before merging can be used as the probability of the merged scene.

[0124] Method 500 is based on Gaussian mixture model, which makes it easier to add, delete and modify existing scene models, better supports the expansion of scenes based on existing scene definitions, and enables secondary development, thereby reducing development costs.

[0125] The scene recognition model can be trained using the expectation-maximum (EM) algorithm.

[0126] The EM algorithm employs an iterative optimization strategy. Each iteration of the EM algorithm consists of two steps: an expectation step (E-step) followed by a maximization step (M-step). The samples and the distribution model they follow are known. The EM algorithm is used to solve for the model parameters of the distribution model based on the samples and the distribution model they follow. Its basic idea is: first, the E-step is performed to calculate the expected conditional probability of the joint distribution based on the given samples; then, the M-step is performed to obtain the values ​​of the model parameters through maximum likelihood estimation, completing one iteration; then, the values ​​of the model parameters obtained in the M-step are used for the next iteration until the values ​​of the model parameters converge, at which point the iteration ends. Maximum likelihood estimation is a statistical method used to estimate model parameters.

[0127] The initialization of the model can be performed randomly, that is, by randomly setting the parameters in the initial scene recognition model. Alternatively, the initialization process can be performed manually. For example, the initial scene recognition model can be obtained manually by using rules or other prior knowledge.

[0128] A first training dataset can be obtained, which may include driving information from multiple training vehicles and the corresponding labeled scene for each driving information. Each driving information from a training vehicle is considered a sample. For each driving information from a training vehicle, the probability of the labeled scene corresponding to that driving information can be set to 1, and the probability of other scenes can be set to 0, thus determining the distribution model followed by each driving information from a training vehicle. Therefore, the EM algorithm can be used to adjust the parameters of the initial scene recognition model based on the first training dataset to obtain the scene recognition model.

[0129] The scene recognition model can be trained in a semi-supervised manner by adjusting the parameters of the initial scene recognition model.

[0130] In semi-supervised learning, the data packet contains both labeled and unlabeled data. First, the labeled data is used for training, and the parameters of the initial scene recognition model are adjusted. Then, the adjusted initial scene recognition model is used to process the unsupervised data (E-step), and the parameters corresponding to maximizing the expectation are calculated (M-step), solving for the local optima of the parameters. This process of using unsupervised data is repeated until the parameters converge. Thus, the scene recognition model is obtained.

[0131] Of course, a supervised training method can also be used to train the scene recognition model. This involves obtaining the driving information of each training vehicle and its corresponding training scene type, thereby training the scene recognition model.

[0132] Training scene recognition models using a semi-supervised approach can reduce the amount of labeled training data and lower manual labor costs.

[0133] Vehicle driving information can include multiple pieces of physical information. This physical information can be used to indicate, for example, the vehicle's position, speed, acceleration, the presence of traffic lights along the vehicle's direction of travel, following distance, current speed limit, and whether the vehicle is passing through a school zone. The input data for sub-models corresponding to different scenarios can include at least one piece of physical information that is the same or different from the vehicle driving information. The number of pieces of physical information in the input data of sub-models corresponding to different scenarios can also be the same or different. The number of pieces of physical information in the input data of a sub-model can be understood as the dimension of the sub-model's input data.

[0134] When fewer conditions are required for a scenario determination, the input data of the sub-model corresponding to that scenario can be manually configured to include less physical information. The number of conditions required for a scenario determination can also be understood as the amount of physical information involved in the rules governing whether a vehicle is driving in that scenario. In other words, by manually configuring the input, physical information that does not affect the scenario determination can be excluded from the sub-model's input, thus reducing the computational load on that sub-model.

[0135] When training this sub-model, physical information that does not affect the scene judgment can be removed from the input of the sub-model corresponding to that scene. This reduces the computational cost and training time required to train the sub-model. For example, in the initial scene recognition model, the input data of this sub-model may not include this physical information from the training vehicle driving information.

[0136] By manually setting the parameters, sub-models for scenarios with more judgment conditions and more complex judgments can have higher dimensionality.

[0137] Sub-models for different scenarios can have the same or different Gaussian kernels. Generally, the number of Gaussian kernels is equal to the number of peaks in the probability distribution. A smaller number of Gaussian kernels can be set manually for scenarios requiring fewer conditions for judgment.

[0138] The fewer Gaussian kernels there are in a sub-model corresponding to a scene, the shorter the computation time required to use that sub-model.

[0139] Generally, increasing the dimension of the input data for the sub-model corresponding to the scene or the number of Gaussian kernels can improve the fitting accuracy and make the calculation more accurate.

[0140] However, by removing physical information from the sub-model input data that has no impact on the scene judgment corresponding to the sub-model, the dimensionality of the sub-model input data is reduced, which has little impact on the fitting accuracy and calculation accuracy of the sub-model output results.

[0141] The number of Gaussian kernels can be set reasonably based on the amount of physical information that affects the judgment of a certain scene, which has little impact on the fitting accuracy and calculation accuracy of the sub-model output.

[0142] In some embodiments, based on the different parent nodes to which the child nodes of each sub-scene belong in the tree-structured scene relationship information, different numbers of Gaussian kernels and input data dimensions can be set for the sub-models corresponding to different scenes. The tree-structured scene relationship information is as follows: Figure 8 As shown. The node narrow U-bend only includes one scene narrow U-bend, so the number of Gaussian kernels and the dimension of input data can be set for the sub-model corresponding to the scene narrow U-bend. The node intersection driving corresponds to a large number of scenes, such as intersection with four-way traffic lights, intersection with only no left-turn protection traffic lights, intersection with only no right-turn protection traffic lights, intersection with no turn protection traffic lights, intersection with stop signs, intersection with yield signs, and intersection with no signs. So the number of Gaussian kernels and the dimension of the sub-models corresponding to the scenes of each sub-node of node intersection driving can be set for more.

[0143] Based on the target scenario defined in S530 and the specific application module requirements, the type of the target scenario can be further determined, thus adapting it to the needs of different application modules. Application modules can then handle different types differently. For example... Figure 9 As shown, corresponding to the first application module, scenarios 1 and 2 belong to type A, scenarios 3 and 4 belong to type B, and scenarios 5 and 6 belong to type C; corresponding to the second application module, scenarios 1 to 3 belong to type I, scenario 4 belongs to type II, and scenarios 5 and 6 belong to type III.

[0144] For example, for the two different application modules, the prediction module and the planning module, the planning module needs to obtain information indicating whether the traffic lights at intersections with traffic lights protect left turns and right turns respectively, and to plan vehicle behavior differently depending on the different protection conditions for turning by the traffic lights. That is, as... Figure 8 The scenario division shown indicates that intersections with four-way traffic lights, intersections without left-turn protection lights, intersections without right-turn protection lights, and intersections without turn protection lights can each correspond to a type. The prediction module does not need such detailed scenario division; it only needs to obtain information indicating whether there are traffic lights at the intersection. Therefore, for the prediction module, intersections with four-way traffic lights, intersections without left-turn protection lights, intersections without right-turn protection lights, and intersections without turn protection lights can each correspond to a type.

[0145] After S530, the target type to which the target scene belongs can be determined based on the scene structure relationship, which is used to indicate the type to which each scene belongs.

[0146] In other words, the target type of a target scenario can be different in multiple application modules. Multiple application modules correspond to multiple scenario structure relationships, and the scenario structure relationship corresponding to each application module is used to indicate the type to which each scenario belongs among the multiple scenarios.

[0147] Different application modules can correspond to different scene structure relationships. When adding or adjusting application modules, only the scene structure relationship needs to be set or adjusted. Even when different application modules have different methods for classifying scene types, there is no need to repeatedly develop scene recognition functions for different application modules. Therefore, Method 500 can adapt to the scene requirements of different application modules and has broad applicability.

[0148] Different application modules can be used to implement different functions. Different application modules can be different programs, or they can be processors used to run different programs, etc.

[0149] The above text combined Figures 1 to 9 The present application describes the method embodiments provided in the embodiments below. Figures 10 to 11 This section describes the apparatus embodiments of the present application. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be referred to in the above description.

[0150] Figure 10 This is a schematic diagram of a scene recognition device provided in an embodiment of this application.

[0151] The scene recognition device 3000 includes an acquisition module 3010 and a processing module 3020.

[0152] The acquisition module 3010 is used to acquire vehicle driving information.

[0153] The processing module 3020 is used to process the vehicle driving information using a scene recognition model to determine the probability of each scene among multiple scenes, wherein the scene recognition model is a trained Gaussian mixture model.

[0154] The processing module 3020 is also used to determine the scenario with the highest probability among the multiple scenarios as the target scenario.

[0155] Optionally, the scene recognition model includes multiple sub-models, each sub-model corresponding to one of the multiple scenes, and the sub-model corresponding to each scene is used to determine the probability of the scene based on the vehicle driving information.

[0156] Optionally, if at least one of the multiple scenarios is divided into multiple sub-scenarios, the acquisition module 3010 is further configured to acquire a training data set, the training data set including training vehicle driving information and training sub-scenarios corresponding to the training vehicle driving information, and the multiple sub-scenarios including the training sub-scenarios.

[0157] The processing module 3020 is also used to train the sub-models corresponding to the multiple sub-scenes using the training data set.

[0158] The processing module 3020 is further configured to update the scene recognition model based on the sub-models corresponding to the plurality of sub-scenes.

[0159] Optionally, the number of Gaussian kernels differs in the multiple sub-models.

[0160] Optionally, the vehicle driving information includes multiple physical information, and each sub-model is used to process at least one physical information corresponding to the sub-model in the vehicle driving information. The first sub-model and the corresponding at least one physical information are different from the at least one physical information corresponding to the second sub-model.

[0161] Optionally, when multiple first scenarios in the plurality of scenarios are merged into a second scenario, the probability of the second scenario is the sum of the probabilities of the plurality of first scenarios.

[0162] Optionally, the target scene belongs to different target types in multiple application modules, and the multiple application modules correspond to multiple scene structure relationships. The scene structure relationship corresponding to each application module is used to indicate the type to which each scene belongs in the multiple scenes.

[0163] Figure 11 This is a schematic structural diagram of a scene recognition device provided in an embodiment of this application.

[0164] The scene recognition device 4000 includes a memory 4001, a processor 4002, a communication interface 4003, and a bus 4004. The memory 4001, processor 4002, and communication interface 4003 are interconnected via the bus 4004.

[0165] The memory 4001 can be a ROM, a static storage device, or RAM. The memory 4001 can store a program, and when the program stored in the memory 4001 is executed by the processor 4002, the processor 4002 and the communication interface 4003 are used to execute the various steps of the scene recognition method of the embodiments of this application.

[0166] The processor 4002 may be a general-purpose CPU, microprocessor, ASIC, GPU, or one or more integrated circuits, used to execute relevant programs to achieve the functions required by the modules in the scene recognition device 3000 of this application embodiment, or to execute the scene recognition method of this application method embodiment.

[0167] The processor 4002 can also be an integrated circuit chip with signal processing capabilities; for example, it could be... Figure 4 The chip shown. In the implementation process, each step of the scene recognition method in this application embodiment can be completed by the integrated logic circuit of the hardware in the processor 4002 or by instructions in the form of software.

[0168] The processor 4002 described above can also be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 4001. The processor 4002 reads the information in memory 4001 and, in conjunction with its hardware, completes the functions required by the modules included in the scene recognition device 3000 of this application embodiment, or executes the scene recognition method of the method embodiment of this application.

[0169] The communication interface 4003 uses a transceiver device, such as, but not limited to, a transceiver, to enable communication between the device 4000 and other devices or communication networks. For example, vehicle driving information can be obtained through the communication interface 4003.

[0170] Bus 4004 may include a pathway for transmitting information between various components of device 4000 (e.g., memory 4001, processor 4002, communication interface 4003).

[0171] It should be noted that although the above-described device 4000 only shows a memory, processor, and communication interface, those skilled in the art should understand that in specific implementations, device 4000 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that device 4000 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that device 4000 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 11 All the devices shown.

[0172] This application provides a computer-readable medium storing program code for execution by a device, the program code including code for performing the scene recognition method described above.

[0173] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the scene recognition method described above.

[0174] This application provides a chip that includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface and executes the scene recognition method described above.

[0175] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0176] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0177] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0178] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0179] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0180] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0181] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0182] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0184] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0186] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0187] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A scene recognition method, characterized in that, include: Obtain vehicle driving information; The vehicle driving information includes at least one of perception data and positioning data. The perception data and the positioning data are used to determine the current scene, which is the scene where the vehicle is currently in. The vehicle driving information also includes planning and control data, which is used to determine the predicted scene, which is the scene that the vehicle will enter after a preset time. The vehicle driving information is processed using a scene recognition model to determine the probability of each scene among multiple scenes. The scene recognition model is a trained Gaussian mixture model. Among the multiple scenarios, the scenario with the highest probability is determined as the target scenario; The scene recognition model includes multiple sub-models, each corresponding to one of the multiple scenes. When at least one of the multiple scenes is divided into multiple sub-scenes, only the sub-models corresponding to the multiple sub-scenes are trained.

2. The method according to claim 1, characterized in that, The scene recognition model includes multiple sub-models, each corresponding to one of the multiple scenes. The sub-model corresponding to each scene is used to determine the probability of the scene based on the vehicle driving information.

3. The method according to claim 2, characterized in that, When at least one of the multiple scenarios is divided into multiple sub-scenarios, the method further includes: Obtain a training data set, the training data set including training vehicle driving information and training sub-scenes corresponding to the training vehicle driving information, the multiple sub-scenes including the training sub-scenes; Using the training dataset, train sub-models corresponding to the multiple sub-scenes; The scene recognition model is updated based on the sub-models corresponding to the multiple sub-scenes.

4. The method according to claim 2 or 3, characterized in that, The number of Gaussian kernels varies in the multiple sub-models.

5. The method according to any one of claims 2-4, characterized in that, The vehicle driving information includes multiple physical information. Each sub-model is used to process at least one physical information corresponding to the sub-model in the vehicle driving information. The first sub-model and its corresponding at least one physical information are different from the at least one physical information corresponding to the second sub-model.

6. The method according to any one of claims 2-5, characterized in that, When multiple first scenarios are merged into a second scenario, the probability of the second scenario is the sum of the probabilities of the multiple first scenarios.

7. The method according to any one of claims 1-6, characterized in that, The target scenario belongs to different target types in multiple application modules. The multiple application modules correspond to multiple scenario structure relationships. The scenario structure relationship corresponding to each application module is used to indicate the type to which each scenario belongs in the multiple scenarios.

8. A scene recognition device, characterized in that, include: Acquisition module and processing module; The acquisition module is used to acquire vehicle driving information; The vehicle driving information includes at least one of perception data and positioning data. The perception data and the positioning data are used to determine the current scene, which is the scene where the vehicle is currently in. The vehicle driving information also includes planning and control data, which is used to determine the predicted scene, which is the scene that the vehicle will enter after a preset time. The processing module is used to process the vehicle driving information using a scene recognition model to determine the probability of each scene among multiple scenes. The scene recognition model is a trained Gaussian mixture model. The processing module is further configured to determine the scenario with the highest probability among the plurality of scenarios as the target scenario; The scene recognition model includes multiple sub-models, each corresponding to one of the multiple scenes. When at least one of the multiple scenes is divided into multiple sub-scenes, only the sub-models corresponding to the multiple sub-scenes are trained.

9. The apparatus according to claim 8, characterized in that, The scene recognition model includes multiple sub-models, each corresponding to one of the multiple scenes. The sub-model corresponding to each scene is used to determine the probability of the scene based on the vehicle driving information.

10. The apparatus according to claim 9, characterized in that, In the case where at least one of the multiple scenarios is divided into multiple sub-scenarios. The acquisition module is further configured to acquire a training data set, the training data set including training vehicle driving information and training sub-scenes corresponding to the training vehicle driving information, and the plurality of sub-scenes including the training sub-scenes. The processing module is further configured to train sub-models corresponding to the multiple sub-scenes using the training data set; The processing module is further configured to update the scene recognition model based on the sub-models corresponding to the plurality of sub-scenes.

11. The apparatus according to claim 9 or 10, characterized in that, The number of Gaussian kernels varies in the multiple sub-models.

12. The apparatus according to any one of claims 9-11, characterized in that, The vehicle driving information includes multiple physical information. Each sub-model is used to process at least one physical information corresponding to the sub-model in the vehicle driving information. The first sub-model and its corresponding at least one physical information are different from the at least one physical information corresponding to the second sub-model.

13. The apparatus according to any one of claims 9-12, characterized in that, When multiple first scenarios are merged into a second scenario, the probability of the second scenario is the sum of the probabilities of the multiple first scenarios.

14. The apparatus according to any one of claims 9-13, characterized in that, The target scenario belongs to different target types in multiple application modules. The multiple application modules correspond to multiple scenario structure relationships. The scenario structure relationship corresponding to each application module is used to indicate the type to which each scenario belongs in the multiple scenarios.

15. A scene recognition device, characterized in that, It includes a memory and a processor, the memory being used to store program instructions; when the program instructions are executed in the processor, the processor is used to perform the method of any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, The computer-readable medium stores program code for execution by the device, the program code including methods for performing any one of claims 1 to 7.

17. A chip, characterized in that, The chip includes a processor and a data interface, wherein the processor reads instructions stored in the memory through the data interface to execute the method as described in any one of claims 1 to 7.

Images