A danger prompting method and device, vehicle and storage medium

Abstract

The application discloses a danger prompting method and device, a vehicle and a storage medium, and belongs to the technical field of vehicles. Through the technical scheme provided by the embodiment of the application, the vehicle terminal can directly identify the passenger in the intelligent cabin and obtain passenger information of the passenger. Based on the driving information of the vehicle, the passenger information and the vehicle information of the vehicle, the vehicle terminal can obtain the risk state of the vehicle. The vehicle terminal can promptly perform danger prompting based on the risk state, so that the passenger or the driver can promptly eliminate the risk and improve the safety of the passenger riding the vehicle.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a hazard warning method, device, vehicle, and storage medium. Background Technology

[0002] With the development of vehicle technology, the design and development of smart cockpits have attracted increasing attention from automakers. A smart cockpit transforms the vehicle into a digital platform, and is an intelligent product equipped with a variety of sensors.

[0003] In related technologies, the focus of smart cockpits is often on improving the driver's experience, passenger comfort, and entertainment features. However, compared to these improvements, enhancing passenger safety within the smart cockpit is far more important. Summary of the Invention

[0004] This application provides a hazard warning method, device, vehicle, and storage medium, which can improve passenger safety when riding in a vehicle. The technical solution is as follows:

[0005] On the one hand, a hazard warning method is provided, the method comprising:

[0006] The passengers in the smart cockpit are identified to obtain their passenger information;

[0007] Based on the vehicle's driving information, and at least one of the passenger's passenger information and the vehicle's vehicle information, the risk status of the passenger riding in the vehicle is determined, wherein the driving information is used to indicate the driving status of the vehicle, and the vehicle information is used to indicate the status of multiple functional components in the vehicle.

[0008] A hazard warning is issued based on the passenger's risk status, indicating that the passenger may be in danger.

[0009] In one possible implementation, the step of identifying the passenger's actions in the smart cockpit based on the image features of the passenger image using the action recognition model includes:

[0010] The action recognition model performs fully connected and normalized processing on the image features of the passenger image, and outputs the probability that the passenger image corresponds to multiple candidate actions.

[0011] The passenger's action is determined as the candidate action with the highest probability among the multiple candidate actions.

[0012] In one possible implementation, the step of identifying the passenger's location in the smart cockpit using sensors on the seats includes any of the following:

[0013] If any sensor on the seat of the smart cockpit detects a passenger, the position of the passenger is determined as the position of the sensor on the seat of the smart cockpit;

[0014] If the sensors on the seats of the smart cockpit do not detect a passenger, it is determined that the passenger is not sitting in the seat.

[0015] On the one hand, a hazard warning device is provided, the device comprising:

[0016] The identification module is used to identify passengers in the smart cockpit and obtain passenger information of the passengers;

[0017] A risk status determination module is used to determine the risk status of a passenger riding in the vehicle based on at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information. The driving information is used to indicate the driving status of the vehicle, and the vehicle information is used to indicate the status of multiple functional components in the vehicle.

[0018] The danger warning module is used to provide danger warnings based on the passenger's risk status, and the danger warnings are used to indicate that there may be danger to the passenger.

[0019] In one possible implementation, the identification module is used to perform motion recognition on passengers in the smart cockpit to obtain the actions of the passengers in the smart cockpit; and to perform location recognition on passengers in the smart cockpit to obtain the location of the passengers in the smart cockpit; the passenger information includes the actions and location of the passengers.

[0020] In one possible implementation, the recognition module is used to input passenger images collected in the smart cockpit into a motion recognition model, extract features from the passenger images using the motion recognition model to obtain image features of the passenger images, and then use the motion recognition model to identify the actions of the passengers in the smart cockpit based on the image features of the passenger images.

[0021] In one possible implementation, the recognition module is configured to perform fully connected and normalized image features of the passenger image through the action recognition model, output the probability of the passenger image corresponding to multiple candidate actions, and determine the candidate action with the highest probability among the multiple candidate actions as the passenger's action.

[0022] In one possible implementation, the identification module is configured to perform any of the following:

[0023] The location of the passenger in the smart cockpit is obtained by using sensors on the seats of the smart cockpit to identify the passenger's position.

[0024] The location of the passenger in the smart cockpit is obtained by identifying the passenger based on the passenger image using a location recognition model.

[0025] In one possible implementation, the identification module is configured to perform any of the following:

[0026] If any sensor on the seat of the smart cockpit detects a passenger, the position of the passenger is determined as the position of the sensor on the seat of the smart cockpit;

[0027] If the sensors on the seats of the smart cockpit do not detect a passenger, it is determined that the passenger is not sitting in the seat.

[0028] In one possible implementation, the risk status determination module is configured to perform any of the following:

[0029] If the driving information indicates that the vehicle is going straight or turning, and the passenger information indicates that the passenger's action is any one of sticking their hand out of the window, leaning their body out of the window, or throwing an object out of the window, then the risk status of the passenger riding in the vehicle is determined as a first candidate risk status. The first candidate risk status is used to indicate that the passenger has performed a dangerous action.

[0030] If the vehicle's driving information indicates that it is going straight or turning, and the passenger information indicates that the passenger is inside the smart cockpit but not sitting in a seat in the smart cockpit, then the risk status of the passenger riding in the vehicle is determined as a second candidate risk status, which is used to indicate that the passenger has left the seat.

[0031] If the vehicle's driving information indicates a turn, the vehicle's information indicates that the opening degree of the vehicle's windows is greater than a preset opening degree threshold, and the passenger's information indicates that the passenger is inside the smart cockpit, then the risk status of the passenger riding in the vehicle is determined to be a third candidate risk status, which indicates that the passenger may detach from the vehicle.

[0032] If the vehicle's driving information indicates that it is going straight or turning, the vehicle's information indicates that the vehicle's doors are not open during the current driving cycle, and the passenger's information indicates that the passenger is not inside the smart cockpit, then the risk status corresponding to riding in the vehicle is determined to be the fourth candidate risk status. The fourth candidate risk status is used to indicate that the passenger has left the vehicle.

[0033] In one possible implementation, the risk status determination module is used to input at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information into a risk status determination model, and the risk status determination model makes a prediction based on at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information, and outputs the risk status of the passenger riding in the vehicle.

[0034] In one possible implementation, the risk state determination module is used to predict at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information using the risk state determination model, to obtain the probability of multiple candidate risk states corresponding to the passenger riding in the vehicle; and to determine the candidate risk state with the highest probability among the multiple candidate risk states as the risk state of the passenger riding in the vehicle.

[0035] In one possible implementation, the danger warning module is configured to perform at least one of the following:

[0036] Play a danger warning voice message based on the type of risk status of the passenger;

[0037] Display a danger warning text based on the passenger's risk status;

[0038] A hazard warning action is performed based on the type of risk status of the passenger, which is any one of seat vibration, steering wheel vibration, and dashboard flashing.

[0039] On one hand, a vehicle is provided, the vehicle including an on-board terminal, the on-board terminal including one or more processors and one or more memories, the one or more memories storing at least one computer program, the computer program being loaded and executed by the one or more processors to implement the danger warning method.

[0040] On one hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer program, which is loaded and executed by a processor to implement the danger warning method.

[0041] On one hand, a computer program product or computer program is provided, which includes program code stored in a computer-readable storage medium. A processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the aforementioned danger warning method.

[0042] The technical solution provided in this application allows the in-vehicle terminal to directly identify passengers in the smart cockpit and obtain their information. Based on the vehicle's driving information, passenger information, and vehicle information, the in-vehicle terminal can determine the risk status of the vehicle. Based on this risk status, the in-vehicle terminal can promptly issue hazard warnings so that passengers or drivers can eliminate risks in a timely manner, thereby improving passenger safety. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the implementation environment of a hazard warning method provided in an embodiment of this application;

[0045] Figure 2 This is a flowchart of a danger warning method provided in an embodiment of this application;

[0046] Figure 3 This is a flowchart of another danger warning method provided in the embodiments of this application;

[0047] Figure 4 This is a schematic diagram of the structure of a hazard warning device provided in an embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the structure of a vehicle-mounted terminal provided in an embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0050] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "n," nor is there any limitation on the quantity or execution order.

[0051] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0052] Cloud computing refers to a delivery and usage model for IT (Internet Technology) infrastructure, meaning obtaining necessary resources through a network in an on-demand and easily scalable manner. In a broader sense, cloud computing also refers to a service delivery and usage model, meaning obtaining necessary services through a network in an on-demand and easily scalable manner. These services can be IT and software-related, internet-related, or other services. Cloud computing is a product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0053] With the development of the internet, real-time data streams, and the diversification of connected devices, as well as the demands for search services, social networks, mobile commerce, and open collaboration, cloud computing has rapidly developed. Unlike previous parallel distributed computing, cloud computing will fundamentally revolutionize the entire internet model and enterprise management model.

[0054] Smart Cockpit: A smart cockpit transforms the car into a digital platform. Traditional car cockpits only display various driving conditions, while the key feature of a smart cockpit lies in its "intelligent" aspect. These cockpits feature multiple displays, and operation changes from traditional buttons to touch or voice control. They also incorporate various sensors and AI intelligent products, considering the driver's habits and comfort to provide a more comfortable driving experience. Another characteristic of smart cockpits is their close integration with the driver's daily life and entertainment. The cockpit offers numerous entertainment options, allowing drivers to watch TV series, enjoy karaoke, play games, and chat in the car. Furthermore, it can display various external information, such as temperature, arrival status, charging station location, driving time, and traffic conditions. It can also monitor the driver's health and adjust the car's extracorporeal fluid accordingly.

[0055] Normalization: Mapping sequences of values ​​with different ranges to the interval (0, 1) to facilitate data processing. In some cases, normalized values ​​can be directly expressed as probabilities.

[0056] Embedded coding, mathematically speaking, represents a correspondence, that is, mapping data in space X to space Y using a function F. This function F is injective, and the mapping result preserves the structure. An injective function means that the mapped data uniquely corresponds to the original data, and preserving the structure means that the order of the original data remains the same. For example, if there are data X1 and X2 before mapping, after mapping we get Y1 corresponding to X1 and Y2 corresponding to X2. If the original data X1 > X2, then correspondingly, the mapped data Y1 > Y2. For words, this means mapping words to another space to facilitate subsequent machine learning and processing.

[0057] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0058] Figure 1 This is a schematic diagram illustrating the implementation environment of a hazard warning method provided in this application embodiment. See also... Figure 1 The implementation environment may include an in-vehicle terminal 110 and a server 140.

[0059] The vehicle-mounted terminal 110 is connected to the server 140 via a wireless network. The vehicle-mounted terminal 110 includes a seat where passengers can sit. The vehicle-mounted terminal 110 includes various types of sensors that can acquire various information from inside and outside the vehicle. Correspondingly, the vehicle-mounted terminal 110 also includes a processor and a memory. The memory stores the information collected by the sensors, and the processor processes the information stored in the memory. The vehicle-mounted terminal 110 has a hazard warning application installed and running.

[0060] Server 140 is a standalone physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0061] After introducing the implementation environment provided by the embodiments of this application, the application scenarios of the embodiments of this application are described below. The technical solution provided by the embodiments of this application can be applied to any vehicle equipped with an in-vehicle terminal. Using the technical solution provided by the embodiments of this application, the in-vehicle terminal can identify passengers and obtain passenger information, including the passenger's actions and location. Based on the vehicle's form information and at least one of the passenger information and vehicle information, the in-vehicle terminal determines the risk status of the passenger riding in the vehicle. Based on the risk status, the in-vehicle terminal can issue a hazard warning, thereby simultaneously reminding both the passenger and the driver that there may be danger, allowing the passenger and the driver to intervene in a timely manner to avoid the danger from occurring.

[0062] After introducing the implementation environment and application scenarios of the embodiments of this application, the technical solutions provided by the embodiments of this application will be described below. (See also...) Figure 2 Taking a vehicle-mounted terminal as the executing entity as an example, the method includes the following steps.

[0063] 201. The vehicle terminal identifies the passenger in the smart cockpit and obtains the passenger information.

[0064] Passenger identification includes both action recognition and location recognition. Action recognition identifies the actions a passenger performs within the smart cockpit, such as extending their hand out of the window. Location recognition identifies the passenger's position within the smart cockpit, including whether they are actually inside, whether they are seated, and which seat they are in. This passenger information reflects their status within the smart cockpit.

[0065] 202. The vehicle terminal determines the risk status of a passenger riding in the vehicle based on at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information. The driving information is used to indicate the driving status of the vehicle, and the vehicle information is used to indicate the status of multiple functional components in the vehicle.

[0066] The vehicle's driving status includes stationary, straight-line, turning, climbing, and descending. Functional components within the vehicle include windows, doors, and sunroof; correspondingly, the status of these components includes the open / closed state of windows, the open / closed state of doors, the locked state of doors, and the open / closed state of the sunroof. The risk status of passengers riding in the vehicle indicates the risk associated with their travel.

[0067] 203. The vehicle terminal provides a hazard warning based on the passenger's risk status. This hazard warning is used to alert the passenger that there may be danger.

[0068] The technical solution provided in this application allows the in-vehicle terminal to directly identify passengers in the smart cockpit and obtain their information. Based on the vehicle's driving information, passenger information, and vehicle information, the in-vehicle terminal can determine the risk status of the vehicle. Based on this risk status, the in-vehicle terminal can promptly issue hazard warnings so that passengers or drivers can eliminate risks in a timely manner, thereby improving passenger safety.

[0069] Steps 201-203 above are a brief introduction to the technical solutions provided in the embodiments of this application. The technical solutions provided in the embodiments of this application will be explained more clearly below with some examples. See [link to relevant documentation]. Figure 3 Taking a vehicle-mounted terminal as the executing entity as an example, the method includes the following steps.

[0070] 301. The vehicle terminal performs motion recognition on passengers in the smart cockpit to obtain the passenger's actions.

[0071] The action recognition results identify the actions performed by passengers within the smart cockpit. For example, a passenger extending their hand out of the window is one action; extending their head out of the window is another; and throwing an object out of the window is yet another. After identifying these actions, the system can combine them with other information to comprehensively assess the risk status of the vehicle the passenger is traveling in, thereby promptly mitigating risks and ensuring passenger safety.

[0072] In one possible implementation, the in-vehicle terminal inputs passenger images captured in the smart cockpit into a motion recognition model. The model then extracts features from the passenger images to obtain their image features. Based on these image features, the in-vehicle terminal uses the motion recognition model to identify the passenger's actions within the smart cockpit.

[0073] The passenger image is captured within the smart cockpit, for example, through a camera installed within the smart cockpit. The action recognition model is a multi-classification model used to recognize the actions of objects in an input image. In this embodiment, the image refers to a passenger image, and the object refers to a passenger. During the training of the action recognition model, sample passenger images are input into the model, which then identifies the actions based on these images and outputs a predicted action corresponding to the sample passenger image. Based on the difference between the labeled action and the predicted action corresponding to the sample passenger image, the action recognition model undergoes one round of iterative training. Through multiple rounds of iterative training, a trained action recognition model is obtained. In the following description, the action recognition model refers to this trained action recognition model.

[0074] It should be noted that the action recognition model can be trained by either the vehicle terminal or the server, and this application embodiment does not limit this.

[0075] Through the above implementation method, the vehicle terminal can use the motion recognition model to perform motion recognition on passenger images, thereby obtaining the passenger's actions in the smart cockpit with high efficiency and accuracy.

[0076] To provide a clearer explanation of the above embodiments, the following description will be divided into several parts.

[0077] The first part involves the in-vehicle terminal inputting passenger images collected in the smart cockpit into the action recognition model.

[0078] The number of passenger images can be one or more, and this embodiment does not limit this. When there are multiple passenger images, the action recognition model can utilize the correlation between the images to perform action recognition, resulting in high accuracy. When there is only one passenger image, the action recognition model can quickly identify the passenger's actions, resulting in high efficiency. This embodiment uses a single passenger image as an example for explanation.

[0079] The second part involves the vehicle-mounted terminal extracting features from the passenger image using the motion recognition model to obtain the image features of the passenger image.

[0080] In one possible implementation, the vehicle terminal performs at least one convolution on the passenger image through the convolutional layer of the motion recognition model to obtain the image features of the passenger image.

[0081] For example, the vehicle-mounted terminal uses the convolutional layer of the action recognition model to slide multiple convolutional kernels across the pixel matrix of the passenger image, performing convolution operations during the sliding process to obtain feature maps corresponding to each convolutional kernel. The vehicle-mounted terminal then fuses these feature maps from the multiple convolutional kernels using the action recognition model to obtain the image features of the passenger image. The size of the convolutional kernels and the stride length are set by technicians according to actual conditions, and this embodiment does not limit these settings.

[0082] In one possible implementation, the vehicle terminal performs at least one full connection on the passenger image through the fully connected layer of the motion recognition model to obtain the image features of the passenger image.

[0083] For example, the vehicle terminal uses the fully connected layer of the action recognition model to multiply the pixel matrix of the passenger image with at least one fully connected matrix and then adds it to at least one bias matrix to obtain the image features of the passenger image.

[0084] In one possible implementation, the vehicle terminal performs attention encoding on the passenger image through the attention encoding layer of the action recognition model to obtain the image features of the passenger image.

[0085] For example, the in-vehicle terminal uses the attention encoding layer of the action recognition model to embed and encode multiple parts of the pixel matrix of the passenger image, obtaining the embedding features of each part of the pixel matrix. The in-vehicle terminal then uses the attention encoding layer of the action recognition model to encode the embedding features of these multiple parts based on an attention mechanism, obtaining the query vector, key vector, and value vector for each part. Finally, the in-vehicle terminal uses the attention encoding layer of the action recognition model to fuse the query vector, key vector, and value vector of these multiple parts to obtain the image features of the passenger image.

[0086] It should be noted that different feature extraction methods correspond to action recognition models with different structures. In the embodiments of this application, any type of action recognition model can be used, such as neural network models, deep learning models, convolutional neural networks, and residual neural networks. This application does not limit the types of action recognition models used.

[0087] Part Three: The vehicle-mounted terminal uses the motion recognition model to identify the actions of the passenger based on the image features of the passenger image, thereby obtaining the actions of the passenger in the vehicle-mounted terminal.

[0088] In one possible implementation, the vehicle-mounted terminal uses the action recognition model to perform fully connected and normalized image features on the passenger image, outputting the probability that the passenger image corresponds to multiple candidate actions. The vehicle-mounted terminal then identifies the candidate action with the highest probability among these multiple candidate actions as the passenger's action.

[0089] In some embodiments, the action recognition model outputs a probability set, which includes multiple probabilities, each probability corresponding to a candidate action.

[0090] 302. The vehicle terminal identifies the location of the passenger in the smart cockpit and obtains the passenger's location.

[0091] The result of location recognition is the passenger's position in the smart cockpit, such as whether the passenger is in the smart cockpit, whether the passenger is sitting in a seat in the smart cockpit, and which seat the passenger is sitting in.

[0092] In one possible implementation, the in-vehicle terminal identifies the passenger's location using sensors on the seat of the smart cockpit, thus obtaining the passenger's position within the smart cockpit.

[0093] For example, if any sensor on the seat of the smart cockpit detects a passenger, the onboard terminal determines the passenger's position as the sensor's position on the seat. If no sensor on the seat detects a passenger, the onboard terminal determines the passenger's position as not being seated.

[0094] In this implementation, the vehicle terminal can identify the passenger's location using sensors on the seat. The sensors have high recognition efficiency and low cost.

[0095] In some embodiments, the sensor is a pressure sensor installed beneath the seat in the smart cockpit. If the pressure value detected by any pressure sensor is greater than a preset pressure threshold, it is determined that the pressure sensor has detected a passenger. The on-board terminal identifies the seat above the pressure sensor as the passenger's location. If the pressure values ​​detected by all pressure sensors beneath the seat in the smart cockpit are less than or equal to the preset pressure threshold, it is determined that the pressure sensor has not detected a passenger. The on-board terminal identifies the passenger as not in a seat.

[0096] In some embodiments, the sensor is a laser sensor, mounted above the seat in the smart cockpit. The laser sensor identifies the passenger's location by sending a laser beam towards the seat; specifically, it identifies the passenger's location by detecting the time difference between the time the laser beam is emitted and the time the reflected laser beam is received. If the time difference detected by any laser sensor is less than or equal to a preset time difference threshold, it is determined that the laser sensor has detected a passenger. The in-vehicle terminal then identifies the seat below the laser sensor as the passenger's location. If the time differences detected by all laser sensors above the seat in the smart cockpit are greater than the preset time difference threshold, it is determined that the laser sensors have not detected a passenger. The in-vehicle terminal then determines that the passenger is not in a seat.

[0097] In one possible implementation, the vehicle terminal uses a location recognition model to identify the passenger's location based on the passenger image, thereby obtaining the passenger's location in the smart cockpit.

[0098] The location recognition model is used to identify the location of an object in an input image. In this embodiment, the image refers to a passenger image, and the object refers to a passenger. During the training of the location recognition model, a sample passenger image is input into the model, which then identifies the location based on the sample passenger image and outputs the predicted location corresponding to that sample passenger image. Based on the difference between the labeled location and the predicted location of the sample passenger image, the location recognition model undergoes one round of iterative training. Through multiple rounds of iterative training, a trained location recognition model is obtained. In the following description, the location recognition model refers to this trained location recognition model.

[0099] Through the above implementation method, the vehicle terminal can use the location as a recognition model to perform location recognition on passenger images, thereby obtaining the passenger's location in the smart cockpit with high efficiency and accuracy.

[0100] It should be noted that the location recognition model can be trained by either an in-vehicle terminal or a server; this application embodiment does not limit this. Furthermore, the location recognition model in the above embodiments can adopt any model structure, such as a neural network model, a deep learning model, a convolutional neural network, or a residual neural network; this application embodiment does not limit this.

[0101] For example, the in-vehicle terminal inputs passenger images collected in the smart cockpit into a location recognition model. The in-vehicle terminal then uses this location recognition model to extract features from the passenger images, obtaining their image features. Based on these image features, the in-vehicle terminal uses the location recognition model to determine the passenger's location within the smart cockpit.

[0102] It should be noted that steps 301 and 302 can be executed sequentially or simultaneously. When steps 301 and 302 are executed sequentially, steps 301 can be executed first and then steps 302, or steps 302 can be executed first and then steps 301. This application embodiment does not limit the execution order and timing of steps 301 and 302.

[0103] Optionally, before step 301, the vehicle terminal can first perform age recognition on the passengers in the smart cockpit. If the passenger's age is found to be less than the age threshold, that is, if the passenger is a child, then step 301 is performed. The vehicle terminal can use any method to perform age recognition, and this application embodiment does not limit this.

[0104] 303. The vehicle terminal determines the risk status of a passenger riding in the vehicle based on at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information. The driving information is used to indicate the driving status of the vehicle, the vehicle information is used to indicate the status of multiple functional components in the vehicle, and the passenger information includes the passenger's actions and location.

[0105] The vehicle's driving status includes stationary, straight-line, turning, climbing, and descending. Functional components within the vehicle include windows, doors, and a sunroof; correspondingly, the status of these components includes the open / closed state of windows, doors, and the sunroof. The risk status of a passenger riding in the vehicle indicates the risk associated with that passenger's journey. In some embodiments, the vehicle's driving information is acquired by an onboard terminal through the vehicle's driving sensors, such as tire sensors or steering wheel sensors. The tire sensors detect the tire rotation angle, and the steering wheel sensors detect the steering wheel rotation angle. Correspondingly, vehicle information is acquired through status sensors of multiple functional components within the vehicle. For example, for windows, a window sensor determines the window opening degree; for the sunroof, a sunroof sensor determines the sunroof opening degree.

[0106] The following four examples illustrate step 303 above.

[0107] Example 1: If the driving information indicates that the vehicle is going straight or turning, and the passenger information indicates that the passenger's action is any one of sticking their hand out of the window, leaning their body out of the window, or throwing an object out of the window, the on-board terminal determines the risk status of the passenger riding in the vehicle as the first candidate risk status. The first candidate risk status is used to indicate that the passenger has performed a dangerous action.

[0108] The first candidate risk state belongs to the plurality of candidate risk states, which are set by technicians according to the actual situation. This application embodiment does not limit this.

[0109] In this implementation, the in-vehicle terminal can quickly identify when a passenger performs a dangerous action in the smart cockpit, and then provide prompts based on the first candidate risk status to remind the passenger to stop the dangerous action and ensure the passenger's safety.

[0110] Example 2: If the vehicle's driving information indicates that it is going straight or turning, and the passenger information indicates that the passenger is inside the smart cockpit but not sitting in a seat in the smart cockpit, the vehicle terminal determines the risk status of the passenger riding in the vehicle as a second candidate risk status. The second candidate risk status is used to indicate that the passenger has left the seat.

[0111] The second candidate risk state belongs to these multiple candidate risk states. "Passenger leaving seat" refers to the passenger being unoccupied while the vehicle is in motion.

[0112] In this implementation, the in-vehicle terminal can quickly identify when a passenger leaves their seat in the smart cockpit, and then provide prompts based on this second candidate risk status to remind the passenger to remain in their seat and ensure passenger safety.

[0113] Example 3: When the vehicle's driving information indicates a turn, the vehicle's information indicates that the opening of the vehicle's windows is greater than a preset opening threshold, and the passenger's information indicates that the passenger is inside the smart cockpit, the vehicle terminal determines the risk status of the passenger riding in the vehicle as a third candidate risk status. This third candidate risk status is used to indicate that the passenger may leave the vehicle.

[0114] The third candidate risk state belongs to the multiple candidate risk states. The preset opening threshold is set by technicians according to the actual situation, and this application embodiment does not limit this. "Passengers may evacuate the vehicle" means that passengers may evacuate the vehicle through the window during the vehicle's turn.

[0115] In this implementation, the vehicle terminal can determine the window opening when the vehicle is turning. If the window opening is greater than a preset opening threshold, the passenger's risk status is determined to be a third risk status, indicating that the passenger is at risk of leaving the vehicle. This is especially true when the passenger is a child. In this way, the driver can be reminded to close the window or close it slightly to prevent the child from leaving the window and ensure the safety of the child when riding in the vehicle.

[0116] Example 4: If the vehicle's driving information indicates that it is going straight or turning, the vehicle information indicates that the vehicle's doors are not open during the current driving cycle, and the passenger information indicates that the passenger is not inside the smart cockpit, the on-board terminal determines the risk status corresponding to the passenger riding in the vehicle as the fourth candidate risk status. The fourth candidate risk status is used to indicate that the passenger has left the vehicle.

[0117] The fourth candidate risk state belongs to these multiple candidate risk states. "The passenger has left the vehicle" means the passenger is no longer inside the smart cabin.

[0118] In this implementation, the in-vehicle terminal can quickly identify that the passenger is not in the smart cockpit, and then provide a prompt based on the fourth candidate risk status to remind the passenger that they are no longer in the vehicle, so that the driver can take remedial measures in time.

[0119] In one possible implementation, the vehicle terminal inputs at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information into a risk status determination model. The risk status determination model then makes a prediction based on the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information, and outputs the risk status of the passenger riding in the vehicle.

[0120] For example, the risk status determination model predicts at least one of the following: the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information, to obtain the probability of multiple candidate risk statuses corresponding to the passenger's travel in the vehicle. The on-board terminal then determines the candidate risk status with the highest probability among these multiple candidate risk statuses as the risk status of the passenger's travel in the vehicle.

[0121] It should be noted that the risk status determination model in the above embodiments can adopt any model structure, such as neural network model, deep learning model, convolutional neural network and residual neural network, etc., and this application embodiment does not limit it.

[0122] In some embodiments, when the vehicle terminal determines the passenger's risk status, the vehicle terminal can first determine whether to execute step 304 based on the type of risk status. For example, if the determined risk status type is the target type, the vehicle terminal executes step 304; if the determined risk status type is not the target type, the vehicle terminal does not execute step 304. The target type is set by a technician according to the actual situation, and this application embodiment does not limit it.

[0123] 304. The vehicle terminal provides a hazard warning based on the passenger's risk status. This hazard warning is used to alert the passenger that there may be danger.

[0124] In one possible implementation, the in-vehicle terminal plays a hazard warning voice message based on the type of risk status of the passenger.

[0125] The smart cockpit includes a speaker capable of playing hazard warning voice messages. The target type is set by technicians according to actual circumstances; this application embodiment does not limit this.

[0126] In this implementation, the in-vehicle terminal can provide hazard warnings by playing voice prompts. Passengers and drivers can quickly receive the warnings and take appropriate measures to avoid danger, thus improving passenger safety.

[0127] For example, if the risk status is the first candidate risk status, the in-vehicle terminal will play a hazard warning voice message through the speaker. This hazard warning voice message is used to prompt passengers to stop performing dangerous actions. For example, the hazard warning voice message may be "Extending your body out of the window is dangerous behavior," "Stop performing dangerous actions," or "Do not extend your body out of the window," etc.

[0128] When the risk status is classified as the second candidate risk status, the in-vehicle terminal plays a hazard warning voice message through the speaker. This message is used to remind passengers to remain in their seats. For example, the warning voice message might say, "You are not seated, this is dangerous behavior," or "Please remain seated."

[0129] When the risk status is classified as the third candidate risk status, the in-vehicle terminal plays a hazard warning voice message through the speaker. This message is used to advise reducing the opening of the windows, meaning it is intended to alert the driver. For example, the message might say, "Please drive carefully, raise the windows to avoid danger to rear passengers." Alternatively, the message might be used to advise maintaining a proper seating position, meaning it is intended to alert passengers. For example, the message might say, "Please be prepared to avoid falling out of the vehicle."

[0130] When the risk status is the fourth candidate risk status, the vehicle terminal plays a hazard warning voice through the speaker. This hazard warning voice is used to inform the passenger that they have left the vehicle, that is, to inform the driver. For example, the hazard warning voice is "There are no passengers in the back seat of the current cabin".

[0131] In one possible implementation, when the risk status is a risk status of the target type, the vehicle terminal displays a danger warning text based on the risk status.

[0132] The smart cockpit includes a first display and a second display. The first display is used to alert the driver, and the second display is used to alert the passengers. The smart cockpit can display hazard warning text through the first display or the second display.

[0133] In this implementation, the in-vehicle terminal can display hazard warnings by showing text, allowing both passengers and drivers to quickly receive the warnings and take timely measures to avoid danger, thus improving passenger safety.

[0134] For example, if the risk status is the first candidate risk status, the in-vehicle terminal displays a hazard warning text on the second display screen. This hazard warning text is used to prompt the user to stop performing the dangerous action. For example, the hazard warning text could be "Stop performing the dangerous action" or "Do not stick your body out of the window."

[0135] When the risk status is the second candidate risk status, the in-vehicle terminal displays a hazard warning text on the second display. This hazard warning text is used to remind passengers to remain in their seats. For example, the hazard warning text may read "It is dangerous for you not to be seated" or "Please remain in your seat."

[0136] When the risk status is classified as the third candidate risk status, the in-vehicle terminal displays a hazard warning text on the first display screen. This hazard warning text is used to advise reducing the opening of the windows, that is, to advise the driver. For example, the hazard warning text could read, "Please drive carefully, raise the windows to avoid danger to rear passengers." Alternatively, the hazard warning text can be displayed on the second display screen to advise maintaining a proper seating position, that is, to advise passengers. For example, the hazard warning text could read, "Please be prepared to avoid falling out of the vehicle."

[0137] When the risk status is the fourth candidate risk status, the vehicle terminal displays a hazard warning text on the first display. This hazard warning text is used to inform the driver that the passenger has left the vehicle. For example, the hazard warning text could be "There are no passengers in the back seat of the current cabin".

[0138] In one possible implementation, when the risk state is a risk state of the target type, the vehicle terminal performs a hazard warning action based on the risk state, which is any one of seat vibration, steering wheel vibration, and dashboard flashing.

[0139] Among them, seat vibration, steering wheel vibration, and dashboard flashing are all used to remind the driver, who can promptly alert the passengers.

[0140] In this implementation, the vehicle terminal can trigger a hazard warning action to remind the driver to promptly alert passengers, thereby enabling timely measures to avoid danger and improving passenger safety.

[0141] It should be noted that the vehicle terminal can provide hazard warnings through any one of the above methods or a combination of any two or three of the above methods, and this application embodiment does not limit this.

[0142] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0143] The technical solution provided in this application allows the in-vehicle terminal to directly identify passengers in the smart cockpit and obtain their information. Based on the vehicle's driving information, passenger information, and vehicle information, the in-vehicle terminal can determine the risk status of the vehicle. Based on this risk status, the in-vehicle terminal can promptly issue hazard warnings so that passengers or drivers can eliminate risks in a timely manner, thereby improving passenger safety.

[0144] Figure 4 This is a schematic diagram of the structure of a hazard warning device provided in an embodiment of this application. See also... Figure 4 The device includes: an identification module 401, a risk status determination module 402, and a hazard warning module 403.

[0145] The identification module 401 is used to identify passengers in the smart cockpit and obtain their passenger information.

[0146] The risk status determination module 402 is used to determine the risk status of a passenger riding in the vehicle based on at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information. The driving information is used to indicate the driving status of the vehicle, and the vehicle information is used to indicate the status of multiple functional components in the vehicle.

[0147] The danger warning module 403 is used to provide a danger warning based on the passenger's risk status, and the danger warning is used to indicate that there may be danger to the passenger.

[0148] In one possible implementation, the identification module 401 is used to perform motion recognition on the passenger in the smart cockpit to obtain the passenger's motion. It also performs location recognition on the passenger in the smart cockpit to obtain the passenger's location. The passenger information includes the passenger's motion and location.

[0149] In one possible implementation, the recognition module 401 is used to input passenger images collected in the smart cockpit into a motion recognition model, extract features from the passenger images using the motion recognition model, and obtain image features of the passenger images. Based on the image features of the passenger images, the motion recognition model is used to identify the actions of the passengers in the smart cockpit.

[0150] In one possible implementation, the recognition module 401 is used to perform fully connected and normalized image features of the passenger image using the action recognition model, and output the probability that the passenger image corresponds to multiple candidate actions. The candidate action with the highest probability among the multiple candidate actions is determined as the passenger's action.

[0151] In one possible implementation, the identification module 401 is configured to perform any of the following:

[0152] The location of the passenger in the smart cockpit is determined by sensors on the seats.

[0153] The location of the passenger in the smart cockpit is obtained by using a location recognition model based on the passenger image.

[0154] In one possible implementation, the identification module 401 is configured to perform any of the following:

[0155] If any sensor on the seat of the smart cockpit detects a passenger, the passenger's position is determined as the position of the sensor on the seat of the smart cockpit.

[0156] If the sensors on the seats in the smart cockpit do not detect a passenger, it is determined that the passenger is not sitting in the seat.

[0157] In one possible implementation, the risk status determination module 402 is configured to perform any of the following:

[0158] If the driving information indicates that the vehicle is going straight or turning, and the passenger information indicates that the passenger's action is any one of sticking their hand out of the window, leaning their body out of the window, or throwing an object out of the window, then the risk status of the passenger riding in the vehicle is determined to be the first candidate risk status. The first candidate risk status is used to indicate that the passenger has performed a dangerous action.

[0159] If the vehicle's driving information indicates that it is going straight or turning, and the passenger's passenger information indicates that the passenger is inside the smart cockpit but not sitting in a seat in the smart cockpit, the risk status of the passenger riding in the vehicle is determined to be a second candidate risk status, which is used to indicate that the passenger has left the seat.

[0160] If the vehicle's driving information indicates a turn, the vehicle's information indicates that the vehicle's window opening is greater than a preset opening threshold, and the passenger's information indicates that the passenger is inside the smart cockpit, then the risk status of the passenger riding in the vehicle is determined to be a third candidate risk status. This third candidate risk status is used to indicate that the passenger may detach from the vehicle.

[0161] If the vehicle's driving information indicates that it is going straight or turning, the vehicle information indicates that the vehicle's doors are not open during the current driving cycle, and the passenger information indicates that the passenger is not inside the smart cockpit, then the risk status corresponding to riding in the vehicle is determined to be the fourth candidate risk status. The fourth candidate risk status is used to indicate that the passenger has left the vehicle.

[0162] In one possible implementation, the risk status determination module 402 is used to input at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information into a risk status determination model, and the risk status determination model makes a prediction based on the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information, and outputs the risk status of the passenger riding in the vehicle.

[0163] In one possible implementation, the risk state determination module 402 is used to predict, from at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information, to obtain the probability of multiple candidate risk states corresponding to the passenger riding in the vehicle. The candidate risk state with the highest probability among the multiple candidate risk states is determined as the risk state of the passenger riding in the vehicle.

[0164] In one possible implementation, the danger warning module 403 is configured to perform at least one of the following:

[0165] A warning message will be played based on the passenger's risk level.

[0166] A danger warning text is displayed based on the passenger's risk status.

[0167] A hazard warning action is executed based on the type of risk status of the passenger, which may be any one of seat vibration, steering wheel vibration, or dashboard flashing.

[0168] It should be noted that the danger warning device provided in the above embodiments is only illustrated by the division of the above functional modules when providing danger warnings. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the danger warning device and the danger warning method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0169] The technical solution provided in this application allows the in-vehicle terminal to directly identify passengers in the smart cockpit and obtain their information. Based on the vehicle's driving information, passenger information, and vehicle information, the in-vehicle terminal can determine the risk status of the vehicle. Based on this risk status, the in-vehicle terminal can promptly issue hazard warnings so that passengers or drivers can eliminate risks in a timely manner, thereby improving passenger safety.

[0170] This application provides a vehicle including an on-board terminal for performing the above-described method. The structure of the on-board terminal is described below:

[0171] Figure 5 This is a schematic diagram of the structure of an in-vehicle terminal provided in an embodiment of this application. Typically, the in-vehicle terminal 500 includes one or more processors 501 and one or more memories 502.

[0172] Processor 501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0173] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 502 are used to store at least one computer program, which is executed by the processor 501 to implement the hazard warning method provided in the method embodiments of this application for providing hazard warnings in an in-vehicle terminal.

[0174] In some embodiments, the vehicle terminal 500 may optionally include a peripheral device interface 503 and at least one peripheral device. The processor 501, memory 502, and peripheral device interface 503 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 504, a display screen 505, a camera assembly 506, an audio circuit 507, and a power supply 508.

[0175] Peripheral device interface 503 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 501 and memory 502. In some embodiments, processor 501, memory 502 and peripheral device interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 501, memory 502 and peripheral device interface 503 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0176] The radio frequency (RF) circuit 504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 504 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc.

[0177] Display screen 505 is used to display a user interface (UI). This UI may include graphics, text, icons, video, and any combination thereof. When display screen 505 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 501 for processing. In this case, display screen 505 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard.

[0178] The camera assembly 506 is used to acquire images or videos. Optionally, the camera assembly 506 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the vehicle terminal, and the rear-facing camera is located on the back of the vehicle terminal.

[0179] The audio circuit 507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 501 for processing, or input to the radio frequency circuit 504 to realize voice communication.

[0180] Power supply 508 is used to power the various components in vehicle terminal 500. Power supply 508 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery.

[0181] In some embodiments, the vehicle terminal 500 further includes one or more sensors 509. The one or more sensors 509 include, but are not limited to, an acceleration sensor 510, a gyroscope sensor 511, a pressure sensor 512, an optical sensor 513, and a proximity sensor 514.

[0182] Accelerometer 510 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by vehicle terminal 500.

[0183] The gyroscope sensor 511 can detect the body orientation and rotation angle of the vehicle terminal 500. The gyroscope sensor 511 can work in conjunction with the accelerometer sensor 510 to collect the user's 3D movements on the vehicle terminal 500.

[0184] The pressure sensor 512 can be installed on the side bezel of the vehicle terminal 500 and / or on the lower layer of the display screen 505. When the pressure sensor 512 is installed on the side bezel of the vehicle terminal 500, it can detect the user's grip signal on the vehicle terminal 500, and the processor 501 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 512. When the pressure sensor 512 is installed on the lower layer of the display screen 505, the processor 501 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 505.

[0185] An optical sensor 513 is used to collect ambient light intensity. In one embodiment, the processor 501 can control the display brightness of the display screen 505 based on the ambient light intensity collected by the optical sensor 513.

[0186] The proximity sensor 514 is used to detect the distance between the user and the front of the vehicle terminal 500.

[0187] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the vehicle terminal 500, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0188] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program that can be executed by a processor to perform the danger warning method described in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0189] In an exemplary embodiment, a computer program product or computer program is also provided, which includes program code stored in a computer-readable storage medium. A processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the aforementioned danger warning method.

[0190] In some embodiments, the computer program involved in the present application embodiments may be deployed and executed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0191] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0192] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for issuing a hazard warning, characterized in that, The method includes: The passenger in the smart cockpit is identified to obtain the passenger information, which is used to indicate the passenger's status in the smart cockpit; Based on the vehicle's driving information, and at least one of the passenger's passenger information and the vehicle's vehicle information, the risk status of the passenger riding in the vehicle is determined. The driving information is used to indicate the driving status of the vehicle, and the vehicle information is used to indicate the status of multiple functional components in the vehicle. The driving status includes stationary, straight, turning, climbing, and descending. Determining the risk status of a passenger riding in the vehicle based on at least one of the vehicle's driving information, passenger information, and vehicle information includes: The vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information are input into a risk state determination model. The risk state determination model predicts the probability of multiple candidate risk states corresponding to the passenger riding in the vehicle based on the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information. The candidate risk state with the highest probability among the multiple candidate risk states is determined as the risk state of the passenger riding in the vehicle. A hazard warning is issued based on the passenger's risk status, indicating that the passenger may be in danger.

2. The method according to claim 1, characterized in that, The process of identifying passengers in the smart cockpit and obtaining their passenger information includes: The actions of passengers in the smart cockpit are recognized to obtain the actions of the passengers in the smart cockpit; The location of passengers in the smart cockpit is identified to obtain the passenger's position; the passenger information includes the passenger's actions and position.

3. The method according to claim 2, characterized in that, The step of performing motion recognition on passengers in the smart cockpit to obtain their actions includes: Passenger images collected in the smart cockpit are input into the action recognition model, and the action recognition model is used to extract features from the passenger images to obtain the image features of the passenger images. The motion recognition model identifies the actions of passengers in the smart cockpit based on the image features of the passenger images.

4. The method according to claim 2, characterized in that, The step of identifying the location of passengers in the smart cockpit to obtain the location of passengers in the smart cockpit includes any of the following: The location of the passenger in the smart cockpit is obtained by using sensors on the seats of the smart cockpit to identify the passenger's position. The location of the passenger in the smart cockpit is obtained by identifying the passenger based on the passenger image using a location recognition model.

5. The method according to claim 1, characterized in that, Determining the risk status of a passenger riding in the vehicle based on at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information further includes any one of the following: If the driving information indicates that the vehicle is going straight or turning, and the passenger information indicates that the passenger's action is any one of sticking their hand out of the window, leaning their body out of the window, or throwing an object out of the window, then the risk status of the passenger riding in the vehicle is determined as a first candidate risk status. The first candidate risk status is used to indicate that the passenger has performed a dangerous action. If the vehicle's driving information indicates that it is going straight or turning, and the passenger information indicates that the passenger is inside the smart cockpit but not sitting in a seat in the smart cockpit, then the risk status of the passenger riding in the vehicle is determined as a second candidate risk status, which is used to indicate that the passenger has left the seat. If the vehicle's driving information indicates a turn, the vehicle's information indicates that the opening degree of the vehicle's windows is greater than a preset opening degree threshold, and the passenger's information indicates that the passenger is inside the smart cockpit, then the risk status of the passenger riding in the vehicle is determined to be a third candidate risk status, which indicates that the passenger may detach from the vehicle. If the vehicle's driving information indicates that it is going straight or turning, the vehicle's information indicates that the vehicle's doors are not open during the current driving cycle, and the passenger's information indicates that the passenger is not inside the smart cockpit, then the risk status corresponding to riding in the vehicle is determined to be the fourth candidate risk status. The fourth candidate risk status is used to indicate that the passenger has left the vehicle.

6. The method according to claim 1, characterized in that, The risk warning based on the passenger's risk status includes at least one of the following: Play a danger warning voice message based on the type of risk status of the passenger; Display a danger warning text based on the passenger's risk status; A hazard warning action is performed based on the type of risk status of the passenger, which is any one of seat vibration, steering wheel vibration, and dashboard flashing.

7. A hazard warning device, characterized in that, The device includes: The identification module is used to identify passengers in the smart cockpit and obtain passenger information of the passengers, which is used to indicate the status of the passengers in the smart cockpit. A risk status determination module is used to determine the risk status of a passenger riding in the vehicle based on at least one of the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information. The driving information is used to indicate the driving status of the vehicle, and the vehicle information is used to indicate the status of multiple functional components in the vehicle. The driving status includes stationary, straight, turning, climbing, and descending. The risk status determination module is further configured to input the vehicle's driving information, the passenger's passenger information, and the vehicle's vehicle information into the risk status determination model, and the risk status determination model predicts the probability of multiple candidate risk statuses corresponding to the passenger riding in the vehicle; and determines the candidate risk status with the highest probability among the multiple candidate risk statuses as the risk status of the passenger riding in the vehicle. The danger warning module is used to provide danger warnings based on the passenger's risk status, and the danger warnings are used to indicate that there may be danger to the passenger.

8. A vehicle, characterized in that, The vehicle includes an on-board terminal, which includes one or more processors and one or more memories. The one or more memories store at least one computer program, which is loaded and executed by the one or more processors to implement the hazard warning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the danger warning method as described in any one of claims 1 to 6.

Images