Control method and device for intelligent equipment of vehicle, intelligent control equipment and vehicle

Abstract

The embodiment of the invention provides a control method and device for intelligent equipment of a vehicle, intelligent control equipment and the vehicle. The method comprises the steps of obtaining at least one kind of information data of a vehicle in response to a first instruction sent by a controller in the vehicle; the information data represents a user state of a user in the vehicle; generating a second instruction according to the at least one information data; wherein the second instruction is used for indicating whether the intelligent equipment in the vehicle enters a dormant state or not; determining a score value of the second instruction; if it is determined that the score value of the second instruction is larger than or equal to the first threshold value, the second instruction is sent to the controller to be executed. The method is used for accurately determining whether the intelligent equipment in the vehicle needs to enter the dormant state or not.

Description

Technical Field

[0001] This application relates to the technical field of vehicles, and in particular, to a control method, device, intelligent control device and vehicle for an intelligent device of a vehicle. Background Art

[0002] With the continuous development of vehicle technology, intelligent devices are usually installed in vehicles for information interaction with users. For example, the intelligent device is equipped with a touch display screen, which can display vehicle information, navigation information and other data to the user, and can also control other devices in the vehicle in response to the user's operation. When the user controls the vehicle to turn off the engine, the controller in the vehicle controls each device in the vehicle to enter the sleep state.

[0003] In the prior art, when the user controls the vehicle to turn off the engine, the controller in the vehicle controls each device in the vehicle to enter the sleep state. Among them, the controller of the vehicle also controls the intelligent device in the vehicle to enter the sleep state. However, the user may still need to use the intelligent device in the vehicle, which may lead to the user being unable to use the intelligent device.

[0004] Furthermore, there is an urgent need for a solution that can accurately determine whether the intelligent device in the vehicle needs to enter the sleep state when the vehicle needs to turn off the engine. Summary of the Invention

[0005] The control method, device, intelligent control device and vehicle for an intelligent device of a vehicle provided by the embodiments of the present application are used to accurately determine whether the intelligent device in the vehicle needs to enter the sleep state.

[0006] In a first aspect, an embodiment of the present application provides a control method for an intelligent device of a vehicle, which is applied to a vehicle. The method includes:

[0007] Respond to a first instruction sent by a controller in the vehicle to obtain at least one piece of information data of the vehicle; wherein, the first instruction indicates whether the intelligent device in the vehicle enters the sleep state; the information data characterizes the user state of the user in the vehicle;

[0008] Generate a second instruction according to at least one piece of information data; wherein, the second instruction is used to indicate whether the intelligent device in the vehicle enters the sleep state; and determine the score value of the second instruction;

[0009] If it is determined that the score value of the second instruction is greater than or equal to a first threshold, send the second instruction to the controller for execution.

[0010] In a possible implementation manner, generating a second instruction according to at least one piece of information data includes:

[0011] Input at least one piece of information data into a preset intelligent recognition model to obtain a second instruction.

[0012] In a possible implementation, inputting at least one piece of information data into a preset intelligent recognition model to obtain a second instruction includes:

[0013] Performing feature extraction processing on the at least one piece of information data based on the preset intelligent recognition model to obtain user features corresponding to the at least one piece of information data;

[0014] Performing feature fusion processing on each user feature based on the cross-attention mechanism of the preset intelligent recognition model to obtain a fusion feature, and performing recognition on the fusion feature based on the preset intelligent recognition model to obtain a second instruction.

[0015] In a possible implementation, before inputting the at least one piece of information data into the preset intelligent recognition model, the method further includes:

[0016] Performing time series alignment processing on the at least one piece of information data so that the time among the pieces of information data is unified.

[0017] In a possible implementation, the method further includes:

[0018] Performing reinforcement learning processing on the preset intelligent recognition model according to the first instruction and the second instruction to update the parameters of the preset intelligent recognition model.

[0019] In a possible implementation, the method further includes:

[0020] Receiving a parameter configuration file sent by a server, and configuring the parameters of an initial model corresponding to the preset intelligent recognition model according to the parameter configuration file to obtain the preset intelligent recognition model;

[0021] Wherein, the parameter configuration file is obtained by the server based on the information data of multiple vehicles.

[0022] In a possible implementation, the at least one piece of information data includes: pupil data of a user in a vehicle, user image of a user in a vehicle, personnel distribution information of a user in a vehicle, seat pressure information in a vehicle, air flow state information in a vehicle, user voice of a user in a vehicle.

[0023] In a possible implementation, the second instruction has a confidence level, which is output by the preset intelligent recognition model that generates the second instruction; determining a score value of the second instruction includes:

[0024] Determining the delay time for collecting the at least one piece of information data; and determining the historical decision accuracy rate of the preset intelligent recognition model that generates the second instruction, where the historical decision accuracy rate indicates the accuracy rate of multiple historical second instructions;

[0025] Determine the score value of the second instruction according to the delay time, confidence level, and historical decision accuracy rate.

[0026] In a possible implementation manner, determining the score value of the second instruction according to the delay time, confidence level, and historical decision accuracy rate includes:

[0027] Perform a weighted sum of the delay time, confidence level, and historical decision accuracy rate to obtain the score value of the second instruction;

[0028] Among them, there is a negative correlation between the value of the delay time and the weight of the delay time, a positive correlation between the value of the confidence level and the weight of the confidence level, and a positive correlation between the value of the historical decision accuracy rate and the weight of the historical decision accuracy rate.

[0029] In a possible implementation manner, the method further includes:

[0030] If it is determined that the score value of the second instruction is less than the first threshold, determine the scenario information of the vehicle according to the driving state of the vehicle;

[0031] If it is determined that the scenario information of the vehicle represents that the vehicle is in a safe scenario, when the score value of the second instruction is greater than or equal to the second threshold, send the second instruction to the controller for execution; wherein, the second threshold is less than the first threshold;

[0032] If it is determined that the scenario information of the vehicle represents that the vehicle is in a safe scenario, when the score value of the second instruction is less than the second threshold, send a locking instruction to the controller, where the locking instruction represents that the controller needs to execute the first instruction.

[0033] In a possible implementation manner, the method further includes:

[0034] If it is determined that the scenario information of the vehicle represents that the vehicle is not in a safe scenario, send a locking instruction to the controller.

[0035] In a second aspect, an embodiment of the present application provides a control device for an intelligent device of a vehicle, including:

[0036] An acquisition module, configured to acquire at least one information data of the vehicle in response to a first instruction issued by a controller in the vehicle; wherein, the first instruction indicates whether the intelligent device in the vehicle enters a sleep state; the information data represents the user state of the user in the vehicle;

[0037] A processing module, configured to generate a second instruction according to at least one information data; wherein, the second instruction is used to indicate whether the intelligent device in the vehicle enters a sleep state; and determine the score value of the second instruction;

[0038] A sending module, configured to send the second instruction to a controller for execution if it is determined that the score value of the second instruction is greater than or equal to a first threshold.

[0039] In a possible implementation manner, according to at least one piece of information data, a second instruction is generated, and the processing module is configured to:

[0040] Input the at least one piece of information data into a preset intelligent recognition model to obtain a second instruction.

[0041] In a possible implementation manner, input the at least one piece of information data into a preset intelligent recognition model to obtain a second instruction, and the processing module is configured to:

[0042] Perform feature extraction processing on the at least one piece of information data based on the preset intelligent recognition model to obtain user features corresponding to the at least one piece of information data;

[0043] Perform feature fusion processing on each user feature based on the cross-attention mechanism of the preset intelligent recognition model to obtain a fusion feature, and perform recognition on the fusion feature based on the preset intelligent recognition model to obtain a second instruction.

[0044] In a possible implementation manner, before inputting the at least one piece of information data into the preset intelligent recognition model, the acquisition module is further configured to:

[0045] Perform time series alignment processing on the at least one piece of information data so that the time between each piece of information data is unified.

[0046] In a possible implementation manner, the processing module is further configured to:

[0047] Perform reinforcement learning processing on the preset intelligent recognition model according to the first instruction and the second instruction to update the parameters of the preset intelligent recognition model.

[0048] In a possible implementation manner, the processing module is further configured to:

[0049] Receive a parameter configuration file sent by a server, and configure the parameters of an initial model corresponding to the preset intelligent recognition model according to the parameter configuration file to obtain the preset intelligent recognition model;

[0050] Wherein, the parameter configuration file is obtained by the server based on the information data of multiple vehicles.

[0051] In a possible implementation manner, the at least one piece of information data includes: pupil data of a user in a vehicle, user image of a user in a vehicle, personnel distribution information of a user in a vehicle, seat pressure information in a vehicle, air flow state information in a vehicle, user voice of a user in a vehicle.

[0052] In a possible implementation, the second instruction has a confidence level, which is output by a preset intelligent recognition model for generating the second instruction; determining a score value of the second instruction, the processing module is configured to:

[0053] Determine the delay time for collecting at least one piece of information data; and determine the historical decision-making accuracy rate of the preset intelligent recognition model for generating the second instruction, where the historical decision-making accuracy rate indicates the accuracy rate of multiple historical second instructions;

[0054] Determine the score value of the second instruction according to the delay time, the confidence level, and the historical decision-making accuracy rate.

[0055] In a possible implementation, according to the delay time, the confidence level, and the historical decision-making accuracy rate, determining the score value of the second instruction, the processing module is configured to:

[0056] Perform a weighted sum of the delay time, the confidence level, and the historical decision-making accuracy rate to obtain the score value of the second instruction;

[0057] Wherein, there is a negative correlation between the value of the delay time and the weight of the delay time, a positive correlation between the value of the confidence level and the weight of the confidence level, and a positive correlation between the value of the historical decision-making accuracy rate and the weight of the historical decision-making accuracy rate.

[0058] In a possible implementation, the sending module is further configured to:

[0059] If it is determined that the score value of the second instruction is less than the first threshold, then determine the scenario information of the vehicle according to the driving state of the vehicle;

[0060] If it is determined that the scenario information of the vehicle indicates that the vehicle is in a safe scenario, then when the score value of the second instruction is greater than or equal to the second threshold, send the second instruction to the controller for execution; wherein, the second threshold is less than the first threshold;

[0061] If it is determined that the scenario information of the vehicle indicates that the vehicle is in a safe scenario, then when the score value of the second instruction is less than the second threshold, send a locking instruction to the controller, where the locking instruction indicates that the controller needs to execute the first instruction.

[0062] In a possible implementation, the sending module is further configured to:

[0063] If it is determined that the scenario information of the vehicle indicates that the vehicle is not in a safe scenario, then send a locking instruction to the controller.

[0064] In a third aspect, an embodiment of the present application provides an intelligent control device, including: a memory, a processor;

[0065] The memory stores computer execution instructions;

[0066] The processor executes the computer-executable instructions stored in the memory, enabling the processor to execute the above first aspect and / or various possible implementations of the first aspect.

[0067] In a fourth aspect, an embodiment of the present application provides a vehicle, in which an intelligent control device provided in the above third aspect is provided.

[0068] In a possible implementation, the vehicle further includes: a controller and an intelligent device;

[0069] The controller of the vehicle is respectively connected to the intelligent control device and the intelligent device;

[0070] The controller of the vehicle is configured to execute a first instruction or a second instruction sent by the intelligent control device, thereby controlling whether the intelligent device enters a sleep state.

[0071] In a possible implementation, the controller of the vehicle and the intelligent device are respectively connected through a first channel and a second channel;

[0072] When the controller of the vehicle executes the first instruction, it sends the first instruction to the intelligent device through the first channel to control whether the intelligent device enters a sleep state; wherein, the controller of the vehicle executes the first instruction based on the lock instruction sent by the intelligent control device;

[0073] When the controller of the vehicle executes the second instruction sent by the intelligent control device, it sends the second instruction to the intelligent device through the second channel to control whether the intelligent device enters a sleep state.

[0074] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementations of the first aspect.

[0075] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above first aspect and / or various possible implementations of the first aspect.

[0076] The control method, device, intelligent control device and vehicle of the intelligent device of the vehicle provided by the embodiments of the present application, when the controller of the vehicle generates a first instruction indicating whether the intelligent device enters the sleep state, in response to the first instruction, obtains various information data of the vehicle, and these information data are data related to the user state collected by devices such as sensors of the vehicle; a second instruction indicating whether the intelligent device should enter the sleep state can be generated according to these information data; and a score value representing the credibility of the second instruction is determined; if the score value is greater than a preset first threshold, it indicates that the credibility of the current second instruction is high, and then the second instruction is executed to control whether the intelligent device enters the sleep state, achieving the effect of accurately determining whether the intelligent device in the vehicle needs to enter the sleep state. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The drawings herein are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0078] Figure 1 Flow schematic of the control method of the intelligent device of the vehicle provided by the present application Figure 1 ;

[0079] Figure 2 Flow schematic of the control method of the intelligent device of the vehicle provided by the present application Figure 2 ;

[0080] Figure 3 Flow schematic of the control method of the intelligent device of the vehicle provided by the present application Figure 3 ;

[0081] Figure 4 Structural schematic diagram of the control device of the intelligent device of the vehicle provided by the present application;

[0082] Figure 5 Structural schematic diagram of the intelligent control device provided by the present application;

[0083] Figure 6 Structural schematic of an exemplary vehicle Figure 1 ;

[0084] Figure 7 Structural schematic of an exemplary vehicle Figure 2 .

[0085] Through the above drawings, the clear embodiments of the present application have been shown, and there will be more detailed descriptions later. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0086] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0087] First, the terms related to the present application will be explained:

[0088] First instruction: It refers to a control instruction generated by a controller in a vehicle according to a user's operation, and this first instruction is used to control whether each vehicle device (including intelligent devices) in the vehicle enters a sleep state.

[0089] Information data: It refers to the data related to the user status inside the vehicle collected, which can indicate whether the user is currently using an intelligent device.

[0090] With the continuous development of vehicle technology, various devices are configured in the vehicle, which may include intelligent devices. These intelligent devices are usually arranged on the console between the driver's seat and the front passenger seat in the cockpit. The intelligent device can interact with the user for information. For example, the intelligent device is equipped with a touch display screen, which can display data such as vehicle information and navigation information to the user, and can also control other devices inside the vehicle in response to the user's operation.

[0091] Combined with practical applications, when the user controls the vehicle to turn off the engine with the car key, the controller in the vehicle detects the engine-off signal of the car key, generates a sleep instruction, and thus controls each device in the vehicle to enter the sleep state.

[0092] In some embodiments, when the user controls the vehicle to turn off the engine, the controller in the vehicle controls each device in the vehicle to enter the sleep state. Among them, the controller of the vehicle also controls the intelligent device in the vehicle to enter the sleep state. However, it is possible that the user still needs to use the intelligent device in the vehicle, which may lead to the user being unable to use the intelligent device.

[0093] For example, when the user controls the vehicle to turn off the engine, the user is still using the intelligent device to query navigation information. If the intelligent device is controlled to enter the sleep state at this time, it will interrupt the user's query of navigation information. The intention of the user to use the intelligent device cannot be correctly recognized.

[0094] Combined with the above scenarios, in the prior art, directly controlling the sleep state of the intelligent device by the vehicle controller cannot correctly recognize the user's intention to use the intelligent device.

[0095] The control method for the intelligent device of a vehicle provided by this application, when the vehicle's controller generates a first instruction indicating whether the intelligent device enters the sleep state, in response to this first instruction, obtains various information data of the vehicle, and these information data are data related to the user state collected by devices such as the vehicle's sensors; a second instruction indicating whether the intelligent device should enter the sleep state can be generated based on these information data; and a score value representing the credibility of the second instruction is determined; if this score value is greater than a preset first threshold, it indicates that the current second instruction has a high credibility, then the second instruction is executed to control whether the intelligent device enters the sleep state, so as to accurately identify the user's intention, represent whether the intelligent device is being used according to the user's intention, and further accurately control the sleep state of the intelligent device.

[0096] The following uses specific embodiments to elaborate in detail on the technical solution of this application and how the technical solution of this application solves the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0097] Figure 1 Flow schematic of the control method for the intelligent device of a vehicle provided by this application Figure 1 , as Figure 1 shown, this method is applied to a vehicle and specifically includes:

[0098] Step 101. In response to a first instruction issued by the controller in the vehicle, obtain at least one information data of the vehicle.

[0099] Among them, the first instruction indicates whether the intelligent device in the vehicle enters the sleep state; the information data represents the user state of the user in the vehicle.

[0100] Exemplarily, the controller in the vehicle, also known as the vehicle's microcontroller unit (MCU for short), will generate a first instruction in response to the user's operation. This first instruction is used to indicate whether the intelligent device in the vehicle enters the sleep state and can also be called a control instruction. For example, when the user controls the vehicle to turn off the engine and turns the key to the "OFF" gear, the controller in the vehicle will generate a first instruction based on the user's control of turning off the vehicle, and this first instruction indicates that the intelligent device in the vehicle enters the sleep state.

[0101] However, in some cases, the user may still need to use the intelligent device after controlling the vehicle to turn off the engine. If the controller executes the first instruction indicating that the intelligent device in the vehicle enters the sleep state, it will cause the user to be unable to continue using the intelligent device. Therefore, in response to the first instruction issued by the controller in the vehicle, obtain at least one information data of the vehicle.

[0102] These information data characterize the user status of the user in the vehicle. For example, these information data can characterize whether the user in the vehicle is in a state of using a smart device. For example, these information data can be obtained by sensors installed in the vehicle, or can be audio data or image data collected by cameras or microphones installed in the vehicle.

[0103] Step 102. Generate a second instruction based on at least one piece of information data; and determine the score value of the second instruction.

[0104] Wherein, the second instruction is used to indicate whether the smart device in the vehicle enters the sleep state.

[0105] Exemplarily, based on the collected information data that can characterize the user status of the user in the vehicle, a second instruction is generated. Wherein, the second instruction is used to indicate whether the smart device in the vehicle enters the sleep state. It can be understood that since the second instruction is generated based on the information data, and the information data is data that can characterize the user status. Therefore, in the case where the information data characterizes that the current user is using the smart device, the second instruction should indicate that the smart device in the vehicle does not enter the sleep state; in the case where the information data characterizes that the current user is not using the smart device, the second instruction should indicate that the smart device in the vehicle enters the sleep state.

[0106] Furthermore, the second instruction can be scored to determine the score value of the second instruction. Wherein, the score value of the second instruction can characterize the credibility of the second instruction. It can be understood that the higher the score value of the second instruction, the more accurately the second instruction can combine the user status to determine whether the smart device should enter the sleep state.

[0107] Optionally, the score value of the second instruction can be determined based on multiple factors. For example, the score value of the second instruction can be calculated based on the reliability score of the device that collected the information data, or can be calculated based on the order of magnitude of the information data. This embodiment does not make specific limitations on how to determine the score value of the second instruction.

[0108] Step 103. If it is determined that the score value of the second instruction is greater than or equal to the first threshold, send the second instruction to the controller for execution.

[0109] Exemplarily, compare the score value of the second instruction with the first threshold. If it is determined that the score value of the second instruction is greater than or equal to the first threshold, it indicates that the current second instruction has a high credibility, that is, the second instruction can accurately combine the current usage status of the user and indicate whether the current smart device should enter the sleep state. When the score value of the second instruction is greater than or equal to the first threshold, send the second instruction to the vehicle controller so that the vehicle controller controls the smart device to enter the sleep state or not to enter the sleep state based on this second instruction.

[0110] Optionally, the score value of the second instruction can be set in the form of a percentage. Correspondingly, the first threshold can be set to 99%. Wherein, the score value of the second instruction can also be set in the form of a real number, or other forms; and the specific value of the first threshold in this embodiment is not limited.

[0111] The control method of the smart device of the vehicle provided by the embodiment of the present application, when the vehicle controller generates a first instruction indicating whether the smart device enters the sleep state, in response to this first instruction, obtain various information data of the vehicle, and these information data are data related to the user state collected by devices such as vehicle sensors; according to these information data, a second instruction indicating whether the smart device should enter the sleep state can be generated. Based on these information data that can characterize the user state, a second instruction that can characterize the user state of whether the smart device is being used can be determined. It can accurately determine whether the user is in the state of using the smart device, so that it can accurately control whether the smart device enters the sleep state.

[0112] In addition, determine a score value used to characterize the credibility of the second instruction; if this score value is greater than a preset first threshold, it indicates that the current second instruction has a high credibility, and then execute this second instruction to control whether the smart device enters the sleep state. Based on the score value of the second instruction, the accuracy of controlling the sleep state of the smart device can be further improved.

[0113] Combined with the foregoing embodiments, it can be seen that the second instruction is obtained based on at least one piece of information data. Optionally, through model processing, the second instruction can be obtained based on at least one piece of information data.

[0114] On the basis of the foregoing embodiments, in one example, inputting at least one piece of information data into a preset intelligent recognition model to obtain a second instruction includes:

[0115] Input at least one piece of information data into a preset intelligent recognition model to obtain a second instruction.

[0116] Exemplarily, it can be processed through a model. Input at least one piece of information data into a preset intelligent recognition model, and after being processed by the preset intelligent recognition model, output to obtain a second instruction.

[0117] Optionally, the preset intelligent recognition model may be a two-stream deep learning network. In this two-stream deep learning network, it includes a state perception stream and a control policy stream. Based on the state perception stream, through a Convolutional Neural Networks (abbreviated as CNN), it can perform feature extraction according to at least one piece of information data to obtain spatio-temporal features. Based on the control policy stream, through a Long Short-Term Memory (abbreviated as LSTM), it can perform temporal modeling to obtain the law of historical policies. According to the spatio-temporal features and the law of historical policies, a second instruction is generated.

[0118] Specifically, during the process of model processing, different features can be obtained according to at least one piece of information data, and a second instruction capable of characterizing the intention of the user to use the intelligent device is generated based on the features.

[0119] Figure 2 Flow schematic of the control method for the intelligent device of the vehicle provided by this application Figure 2 , such as Figure 2 As shown, on the basis of the foregoing example, this embodiment details the step of inputting at least one piece of information data into a preset intelligent recognition model to obtain a second instruction. The method includes:

[0120] Step 201. Perform feature extraction processing on at least one piece of information data based on a preset intelligent recognition model to obtain user features corresponding to the at least one piece of information data.

[0121] Exemplarily, based on a preset intelligent recognition model, feature extraction can be performed on at least one piece of information data. Among them, combined with the foregoing example, it can be known that the preset intelligent recognition model may be a two-stream deep learning network model. Based on the state perception stream, feature extraction processing can be performed on the information data to obtain spatio-temporal features. These spatio-temporal features can characterize the joint dynamic features of the vehicle user in terms of time and space. For example, it can characterize the position of the user in the vehicle or the staying time in the vehicle, etc.

[0122] Based on the control policy stream, it can combine the current information data and historical data to perform feature extraction processing to obtain device temporal features. These device temporal features can characterize the intention features of the vehicle user to use the intelligent device.

[0123] It can be understood that the above-mentioned intention features and spatio-temporal features are both user features related to the vehicle user. And this user feature is the user feature corresponding to at least one piece of information data input into the preset intelligent recognition model.

[0124] Step 202. Based on the cross-attention mechanism of the preset intelligent recognition model, perform feature fusion processing on each user feature to obtain a fused feature, and identify the fused feature based on the preset intelligent recognition model to obtain a second instruction.

[0125] Exemplarily, the preset intelligent recognition model may have a cross-attention mechanism, which can allow the preset intelligent recognition model to establish a focus relationship based on different sequences or different data sources, thereby achieving information fusion.

[0126] Therefore, based on the cross-attention mechanism of the preset intelligent recognition model, perform fusion processing on each user feature to obtain a fused feature.

[0127] Furthermore, based on the preset intelligent recognition model, identify the obtained fused feature to determine the second instruction corresponding to the fused feature.

[0128] The second instruction is used to indicate whether the intelligent device in the vehicle enters the sleep state. Optionally, the second instruction may be the power action space of the intelligent device. For example, the power action space includes: sleep action, wake-up action, enter the energy-saving mode, etc.

[0129] In the above embodiments, through the preset intelligent recognition model, feature extraction is performed on at least one piece of information data, and through feature fusion and feature recognition, a second instruction is obtained. It can improve the recognition accuracy of the true intention of the user in the information data, and further ensure that whether the intelligent device indicated in the generated second instruction enters the sleep state is consistent with whether the current user uses the intelligent device in the vehicle.

[0130] Combined with the foregoing examples, at least one piece of information data may be collected by different devices, and the data collected by these different devices may have time deviations. Therefore, before inputting into the preset intelligent recognition model for processing, it is also necessary to perform unification in the time dimension.

[0131] In one example, before inputting at least one piece of information data into the preset intelligent recognition model, the method further includes:

[0132] Perform timing alignment processing on at least one piece of information data so that the time between each piece of information data is unified.

[0133] Exemplarily, perform timing alignment on at least one piece of information data. Among them, at least one piece of information data may be collected from different devices. Therefore, perform timing alignment processing on each piece of information data.

[0134] Specifically, the vehicle controller is set with a clock signal. Based on the clock signal of the controller, a unified timestamp is assigned to each type of information data. Based on the unified timestamp, the deviation of the physical clocks between different devices can be eliminated.

[0135] In a possible scenario, the starting times of different devices for data collection are different, which may cause some devices to start collecting information data earlier while some start later. In this case, for the information data collected later, there are data gaps between the different starting times of data collection.

[0136] Optionally, after assigning a unified timestamp to each type of information data, data compensation can also be performed. The adaptive Kalman filtering algorithm is used to fuse the noise covariance matrix of the device that collects the information data, predict and compensate for the missing data. It can achieve the compensation of the missing data and improve the smoothness of the information data.

[0137] In the above example, by assigning a unified timestamp to the information data, it is possible to achieve the time sequence alignment of different types of information data. Based on the information data with unified time, it can be ensured that between different types of information data, the state of the user at the same moment can be reflected. Thus, the second instruction generated by the preset intelligent recognition model can more accurately fit the current state of the user. Furthermore, it can more accurately control the sleep state of the intelligent device based on the user's intention.

[0138] Combined with the foregoing example, it can be seen that the second instruction is obtained through the preset intelligent recognition model. Therefore, for the preset intelligent recognition model, it can be trained through the way of federated learning; and after each generation of the second instruction, the preset intelligent recognition model can be feedback-optimized.

[0139] In one example, the method further includes:

[0140] Receiving a parameter configuration file sent by the server, and configuring the parameters of the initial model corresponding to the preset intelligent recognition model according to the parameter configuration file to obtain the preset intelligent recognition model.

[0141] Wherein, the parameter configuration file is obtained by the server based on the information data of multiple vehicles.

[0142] Exemplarily, an initial model is set on the server, and the server is respectively communicatively connected to multiple vehicles. The information data collected by the multiple vehicles is sent to the server, and the server performs training based on the initial model on the server to obtain the parameter configuration file. This parameter configuration file represents the parameter information of the preset intelligent recognition model obtained after training.

[0143] Further, the server sends the parameter configuration file to the vehicle. The vehicle also has the same initial model as the server, and this initial model is the initial model corresponding to the preset intelligent recognition model. The vehicle receives the parameter configuration file and configures the initial model according to the parameter configuration file, so as to obtain the preset intelligent recognition model.

[0144] In the above example, by training the initial model on the server, a parameter configuration file representing the parameter information of the preset intelligent recognition model is obtained, and this parameter configuration file is sent to the vehicle. The vehicle configures the parameters of the initial model corresponding to the preset intelligent recognition model according to this parameter configuration file, so as to obtain the intelligent recognition model. By training in the cloud, the computing load on the vehicle side is reduced, and the efficiency of model training is improved.

[0145] In one example, the method further includes:

[0146] Perform reinforcement learning on the preset intelligent recognition model according to the first instruction and the second instruction to update the parameters of the preset intelligent recognition model.

[0147] Exemplarily, the preset intelligent recognition model can generate a second instruction for indicating whether the intelligent device in the vehicle enters the sleep state; and the first instruction is generated by the controller of the vehicle for indicating whether the intelligent device in the vehicle enters the sleep state.

[0148] In a possible situation, according to the second instruction generated by the preset intelligent recognition model and the first instruction generated by the controller of the vehicle, the parameters of the preset intelligent recognition model are updated by means of reinforcement learning.

[0149] Specifically, a reinforcement learning environment can be defined, including actions, states, and rewards. Among them, the action is the power action in the second instruction; the fused feature after feature extraction and feature fusion processing of the information data can be used as the state; and a reward function for multi-objective optimization can be set.

[0150] Further, reinforcement learning can be performed according to the Proximal Policy Optimization (PPO) algorithm to update and iterate the parameters of the preset intelligent recognition model.

[0151] In the above example, through the generated second instruction and the first instruction, the parameters of the preset intelligent recognition model can be updated and iterated by means of reinforcement learning, so as to realize the further optimization of the preset intelligent recognition model. By using the PPO algorithm, the exploration and optimization process of reinforcement learning can be balanced, and the efficiency of the reinforcement learning process can be improved.

[0152] As can be seen from the foregoing embodiments, the second instruction is obtained based on information data. In practical applications, for example, for at least one type of information data for generating the second instruction, it may include various types of information data collected inside the vehicle and related to the user.

[0153] In one example, the at least one type of information data includes: pupil data of the user in the vehicle, user image of the user in the vehicle, personnel distribution information of the user in the vehicle, seat pressure information in the vehicle, air flow state information in the vehicle, and user voice of the user in the vehicle.

[0154] Exemplarily, the vehicle is provided with an infrared camera to collect infrared image data related to the user; and based on a pupil tracking algorithm, the pupil data of the user in the vehicle is determined according to the infrared image data.

[0155] Exemplarily, the vehicle is provided with a camera to collect the user image of the user in the vehicle.

[0156] Exemplarily, the vehicle is provided with a capacitive pressure sensor that can sense the personnel distribution information of the user in the vehicle and the seat pressure information in the vehicle.

[0157] Exemplarily, the vehicle is provided with a temperature and humidity composite sensor that can determine the air flow state information in the vehicle.

[0158] Exemplarily, a microphone is provided inside the vehicle to collect the user voice of the user in the vehicle. Optionally, a microphone is also provided outside the vehicle, which can perform noise separation processing on the user voice based on the user voice collected by the microphone inside the vehicle and the ambient noise collected by the microphone outside the vehicle, improve the clarity of the user voice, and thus improve the recognition accuracy of the user intention.

[0159] In the above examples, by collecting various types of information data, the true intention of the user can be comprehensively reflected from multi-dimensional data, so as to more accurately determine whether the user is using the intelligent device; this lays a foundation for accurately controlling the sleep state of the intelligent device.

[0160] As can be seen from the foregoing embodiments, the score value of the second instruction can be determined, and this score value can reflect whether the second instruction is credible currently. Based on the foregoing embodiments, this embodiment specifically describes how to determine the score value of the second instruction.

[0161] In one example, the second instruction has a confidence level, and the confidence level is output by a preset intelligent recognition model for generating the second instruction.

[0162] Exemplarily, when the preset intelligent recognition model generates the second instruction, it will simultaneously output the confidence corresponding to the second instruction. The confidence characterizes the credibility of the intelligent recognition model in the process of generating the second instruction based on at least one piece of information data.

[0163] Figure 3 Schematic flow of the control method for the intelligent device of the vehicle provided by the present application Figure 3 As Figure 3 shown, specifically, determining the score value of the second instruction includes:

[0164] Step 301. Determine the delay time for collecting at least one piece of information data; and determine the historical decision accuracy rate of the preset intelligent recognition model for generating the second instruction.

[0165] Among them, the historical decision accuracy rate indicates the accuracy rate of multiple historical second instructions.

[0166] Exemplarily, the difference obtained by subtracting the time when the information data is acquired from the time represented by the time stamp of the information data collection can be used as the delay time for collecting the information data.

[0167] Exemplarily, if it is determined that the execution logic between whether the second instruction indicates that the intelligent device enters the sleep state and whether the actual intelligent device enters the sleep state is consistent, then the second instruction is considered correct. Based on the above method, it can be determined whether multiple historical second instructions are correct, and based on the number of correct second instructions and the number of historical second instructions, the historical decision accuracy rate can be determined.

[0168] Step 302. Determine the score value of the second instruction according to the delay time, the confidence, and the historical decision accuracy rate.

[0169] Exemplarily, the score value of the second instruction can be determined according to the delay time, the confidence, and the historical decision accuracy rate.

[0170] Specifically, in the process of determining the score value of the second instruction, it can be carried out by means of weighted summation.

[0171] In one example, a weighted sum of the delay time, the confidence, and the historical decision accuracy rate is obtained to get the score value of the second instruction.

[0172] Among them, the value of the delay time has a negative correlation with the weight of the delay time, the value of the confidence has a positive correlation with the weight of the confidence, and the value of the historical decision accuracy rate has a positive correlation with the weight of the historical decision accuracy rate.

[0173] Exemplarily, the delay time corresponds to a weight value, the confidence level corresponds to a weight value, and the historical decision accuracy rate corresponds to a weight value. Multiply the delay time by the corresponding weight value, multiply the confidence level by the corresponding weight value, and multiply the historical decision accuracy rate by the corresponding weight value; then sum them up to obtain the score value of the second instruction.

[0174] It should be noted that since the larger the delay time, the worse the real-time performance of the information data, the weight value corresponding to the delay time can be set as a real number less than 0.

[0175] Furthermore, the weight value corresponding to the delay time can be adjusted according to the value of the delay time. Specifically, the value of the delay time and the weight value corresponding to the delay time can show a negative correlation. For example, if it is determined that the delay time is greater than 50 milliseconds, the weight value corresponding to the delay time is reduced to 70% of the original weight value.

[0176] Similarly, the weight value corresponding to the confidence level can be adjusted according to the value of the confidence level. Specifically, the value of the confidence level and the weight value corresponding to the confidence level can show a positive correlation. For example, if it is determined that the confidence level is less than the preset confidence threshold, and at this time the second instruction is a low-confidence decision, the weight value corresponding to the confidence level is changed to 0.

[0177] Optionally, the preset intelligent recognition model can also output the output probability distribution entropy value of the second instruction. If the output probability distribution entropy value is greater than 2.5, the second instruction can also be determined as a low-confidence decision, and the weight value corresponding to the confidence level is changed to 0.

[0178] Similarly, the weight value of the historical decision accuracy rate can be adjusted according to the value of the historical decision accuracy rate. Specifically, the value of the historical decision accuracy rate and the weight value of the historical decision accuracy rate can show a positive correlation.

[0179] It should be noted that when determining the score value of the second instruction by weighted summation according to the delay time, confidence level, and historical decision accuracy rate, the weight value corresponding to the historical decision accuracy rate can be set to be the largest among the three weight values.

[0180] In the above example, through the method of weighted summation, the score value of the second instruction can be determined, and during the process of weighted summation, the corresponding weight values can be adjusted in real time according to the delay time, confidence level, and historical decision accuracy rate. It can accurately determine the score value of the second instruction, laying a foundation for accurately fitting the actual intention of the user and controlling the sleep state of the intelligent device based on the score value.

[0181] In the above embodiments, the delay time of the information data, the confidence level of the second instruction, and the historical decision-making accuracy rate of the second instruction are first determined. Then, based on these values, a weighted sum is performed to obtain the score value of the second instruction. Considering the freshness of the information data and the reliability of the second instruction, the score value of the second instruction can more accurately reflect whether the second instruction reflects the user's true usage intention of the intelligent device.

[0182] Combined with the foregoing embodiments, when the score value of the second instruction is greater than or equal to the first threshold, it indicates that the current second instruction has a high credibility, and then the second instruction is executed. However, if the score value of the second instruction is less than the first threshold, it indicates that the current second instruction does not have sufficient credibility to be directly executed. Therefore, based on any of the foregoing embodiments, this embodiment specifically describes the case where the score value of the second instruction is less than the first threshold.

[0183] In one example, the method further includes:

[0184] If it is determined that the score value of the second instruction is less than the first threshold, the scenario information of the vehicle is determined according to the driving state of the vehicle.

[0185] Exemplarily, if it is determined that the score value of the second instruction is less than the first threshold, it indicates that the current second instruction does not have sufficient credibility to be directly executed. Further, the scenario information of the vehicle is determined according to the driving state of the vehicle. Among them, the scenario information of the vehicle represents different vehicle states. For example, the scenario information of the vehicle represents that the vehicle is in a parked state, the vehicle is in a low-speed driving state, or the vehicle is in a high-speed driving state. Among them, the parked state means that the vehicle is stationary and its vehicle speed is 0.

[0186] Specifically, the low-speed driving state means that the vehicle speed of the vehicle is less than or equal to the preset driving speed; the high-speed driving state means that the vehicle speed of the vehicle is greater than the preset driving speed.

[0187] Optionally, in addition to judging the scenario information of the vehicle based on the vehicle speed, the operation state of the vehicle's motor, the gear position of the vehicle key, the navigation working state, the vehicle driving gear, the throttle and brake pedal state, etc. can also be combined to jointly judge the scenario information of the vehicle.

[0188] In one example, if it is determined that the scenario information of the vehicle represents a safe scenario, when the score value of the second instruction is greater than or equal to the second threshold, the second instruction is sent to the controller for execution. Among them, the second threshold is less than the first threshold.

[0189] Among them, the safe scenario means that the vehicle state is that the vehicle is in a parked state or the vehicle is in a low-speed driving state. If it is determined that the vehicle is in a parked state or the vehicle is in a low-speed driving state, then the magnitude relationship between the score value of the second instruction and the second threshold is further determined. If it is determined that the score value of the second instruction is greater than or equal to the second threshold, the second instruction is sent to the vehicle controller, so that the vehicle controller controls the intelligent device to enter the sleep state or not enter the sleep state based on the second instruction.

[0190] Exemplarily, the score value of the second instruction can be in the form of a percentage. The first threshold can be set to 99%, and the second threshold should be less than the first threshold. Optionally, the second threshold can be set to 90%.

[0191] In one example, if it is determined that the vehicle scenario information indicates that the vehicle is in a safe scenario, then when the score value of the second instruction is less than the second threshold, a locking instruction is sent to the controller, where the locking instruction indicates that the controller needs to execute the first instruction.

[0192] If it is determined that the vehicle is in a parked state or the vehicle is in a low-speed driving state, then the magnitude relationship between the score value of the second instruction and the second threshold is further determined. If it is determined that the score value of the second instruction is less than the second threshold, it indicates that the current credibility of the second instruction is low, that is, whether the intelligent device indicated by the second instruction enters the sleep state deviates greatly from the actual usage intention of the user.

[0193] Therefore, a locking instruction is sent to the controller. The locking instruction is used to instruct the controller to execute the first instruction. The first instruction indicates whether the intelligent device in the vehicle enters the sleep state. After sending the locking instruction to the controller, the controller responds to the locking instruction, locks the first instruction, and executes the first instruction to control the sleep state of the intelligent device to be the state of the intelligent device indicated by the first instruction.

[0194] In one example, if it is determined that the vehicle scenario information indicates that the vehicle is not in a safe scenario, a locking instruction is sent to the controller.

[0195] Not being in a safe scenario means that the vehicle is in a high-speed driving state. If it is determined that the vehicle is in a high-speed driving state, a locking instruction is directly sent to the controller at this time. The controller responds to the locking instruction, locks the first instruction, and executes the first instruction to control the sleep state of the intelligent device to be the state of the intelligent device indicated by the first instruction.

[0196] Taking an actual application example, when the user is driving a vehicle normally, the first instruction generated by the controller indicates that the intelligent device does not enter the sleep state. If the second instruction generated by the preset intelligent recognition model indicates that the intelligent device enters the sleep state at this time, even if the score value of the second instruction is relatively high, the intelligent device cannot be put into sleep when the user is driving the vehicle normally. Therefore, when it is determined that the vehicle is in a high-speed driving state, the first instruction generated by the controller should be executed.

[0197] In the above embodiment, when the score value of the second instruction is less than the first threshold, in combination with the scenario information of the vehicle, if the scenario information indicates that the vehicle is in a safe state, the size relationship between the score value of the second instruction and the second threshold is further judged; if the scenario information indicates that the vehicle is not in a safe state, a locking instruction is directly sent so that the controller executes the first instruction. It is possible to further determine whether to execute the first instruction or the second instruction in combination with the state of the vehicle represented by the scenario information of the vehicle. It is possible to further consider the state of the vehicle. When it is not suitable to enter the sleep state, the second instruction generated by the model is not considered, and the first instruction generated by the controller is directly used for execution. This improves the safety of vehicle driving from the side.

[0198] In the control method of the intelligent device of the vehicle provided by the embodiment of the present application, when the controller of the vehicle generates a first instruction indicating whether the intelligent device enters the sleep state, in response to the first instruction, a variety of information data of the vehicle are obtained, and based on these information data, a second instruction indicating whether the intelligent device should enter the sleep state can be generated. Among them, the information data is data related to the user state collected by devices such as sensors of the vehicle. Based on this information data that can characterize the user state, a second instruction that can characterize whether the user state is in the state of using the intelligent device can be determined. It is possible to accurately determine whether the user is in the state of using the intelligent device, so as to accurately control whether the intelligent device enters the sleep state.

[0199] In addition, the second instruction can have a score value. According to the relationship between the score value and the threshold, it is judged whether to execute the second instruction or the first instruction originally generated by the controller. When the score value is relatively high, the controller executes the second instruction; if the score value indicates that the credibility of the second instruction is not sufficient to directly execute the second instruction, the scenario information of the vehicle is further judged. If the scenario information indicates that the vehicle is in a safe state, it is further judged whether the score value is greater than the second threshold. If the score value is greater than or equal to the second threshold, the controller executes the second instruction; otherwise, the controller executes the first instruction. If the scenario information indicates that the vehicle is not in a safe state, the controller directly executes the first instruction. It can ensure that the sleep state of the intelligent device not only accurately conforms to the user's true usage intention, but also takes into account the state of the vehicle to ensure the safety of vehicle driving.

[0200] Figure 4 The structural schematic diagram of the control device for the intelligent device of the vehicle provided by this application is as Figure 4 shown. The control device 40 for the intelligent device of the vehicle provided in this embodiment includes:

[0201] An acquisition module 401, configured to acquire at least one piece of information data of the vehicle in response to a first instruction issued by a controller in the vehicle; wherein, the first instruction indicates whether the intelligent device in the vehicle enters a sleep state; the information data characterizes the user state of the user in the vehicle;

[0202] A processing module 402, configured to generate a second instruction according to at least one piece of information data; wherein, the second instruction is used to indicate whether the intelligent device in the vehicle enters a sleep state; and determine the score value of the second instruction;

[0203] A sending module 403, configured to send the second instruction to the controller for execution if it is determined that the score value of the second instruction is greater than or equal to a first threshold.

[0204] In a possible implementation manner, to generate a second instruction according to at least one piece of information data, the processing module 402 is configured to:

[0205] Input at least one piece of information data into a preset intelligent recognition model to obtain a second instruction.

[0206] In a possible implementation manner, to input at least one piece of information data into a preset intelligent recognition model to obtain a second instruction, the processing module 402 is configured to:

[0207] Perform feature extraction processing on at least one piece of information data based on the preset intelligent recognition model to obtain user features corresponding to at least one piece of information data;

[0208] Perform feature fusion processing on each user feature based on the cross-attention mechanism of the preset intelligent recognition model to obtain a fusion feature, and perform recognition on the fusion feature based on the preset intelligent recognition model to obtain a second instruction.

[0209] In a possible implementation manner, before inputting at least one piece of information data into the preset intelligent recognition model, the acquisition module 401 is further configured to:

[0210] Perform time series alignment processing on at least one piece of information data so that the time between each piece of information data is unified.

[0211] In a possible implementation manner, the processing module 402 is further configured to:

[0212] Perform reinforcement learning processing on the preset intelligent recognition model according to the first instruction and the second instruction to update the parameters of the preset intelligent recognition model.

[0213] In a possible implementation, the processing module 402 is further configured to:

[0214] Receive the parameter configuration file sent by the server, and configure the parameters of the initial model corresponding to the preset intelligent recognition model according to the parameter configuration file to obtain the preset intelligent recognition model;

[0215] Wherein, the parameter configuration file is obtained by the server based on the information data of multiple vehicles.

[0216] In a possible implementation, at least one type of information data includes: pupil data of the user in the vehicle, user image of the user in the vehicle, personnel distribution information of the user in the vehicle, seat pressure information in the vehicle, air flow state information in the vehicle, and user voice of the user in the vehicle.

[0217] In a possible implementation, the second instruction has a confidence level, which is output by the preset intelligent recognition model that generates the second instruction; determine the score value of the second instruction, and the processing module 402 is configured to:

[0218] Determine the delay time for collecting at least one type of information data; and determine the historical decision accuracy rate of the preset intelligent recognition model that generates the second instruction, where the historical decision accuracy rate indicates the accuracy rate of multiple historical second instructions;

[0219] Determine the score value of the second instruction according to the delay time, confidence level, and historical decision accuracy rate.

[0220] In a possible implementation, to determine the score value of the second instruction according to the delay time, confidence level, and historical decision accuracy rate, the processing module 402 is configured to:

[0221] Perform a weighted sum of the delay time, confidence level, and historical decision accuracy rate to obtain the score value of the second instruction;

[0222] Wherein, there is a negative correlation between the value of the delay time and the weight of the delay time, a positive correlation between the value of the confidence level and the weight of the confidence level, and a positive correlation between the value of the historical decision accuracy rate and the weight of the historical decision accuracy rate.

[0223] In a possible implementation, the sending module 403 is further configured to:

[0224] If it is determined that the score value of the second instruction is less than the first threshold, determine the scene information of the vehicle according to the driving state of the vehicle;

[0225] If it is determined that the scenario information of the vehicle indicates that the vehicle is in a safe scenario, when the score value of the second instruction is greater than or equal to the second threshold, the second instruction is sent to the controller for execution; wherein, the second threshold is less than the first threshold.

[0226] If it is determined that the scenario information of the vehicle indicates that the vehicle is in a safe scenario, when the score value of the second instruction is less than the second threshold, a locking instruction is sent to the controller, wherein the locking instruction indicates that the controller needs to execute the first instruction.

[0227] In a possible implementation manner, the sending module 403 is further configured to:

[0228] If it is determined that the scenario information of the vehicle indicates that the vehicle is not in a safe scenario, a locking instruction is sent to the controller.

[0229] The control device of the intelligent device of the vehicle provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, and will not be elaborated here in this embodiment.

[0230] Figure 5 It is a schematic structural diagram of the intelligent control device provided in this application. As Figure 5 shown, the intelligent control device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the intelligent control device 50 further includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus 504.

[0231] In a specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0232] The specific implementation process of the processor 501 can refer to the above method embodiment, and its implementation principle and technical effect are similar, and will not be elaborated here in this embodiment.

[0233] In the above embodiment, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, abbreviated: CPU), and may also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, abbreviated: DSP), application specific integrated circuits (English: Application Specific Integrated Circuit, abbreviated: ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0234] The memory may include a random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0235] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of easy representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0236] This application also provides a vehicle, in which an intelligent control device as provided in the foregoing embodiment is provided.

[0237] Figure 6 Schematic diagram of the structure of an exemplary vehicle Figure 1 , such as Figure 6 shown, in a possible implementation manner, the vehicle 60 is provided with an intelligent control device 601, and the vehicle further includes: a controller 602 and an intelligent device 603.

[0238] The controller 602 of the vehicle 60 is respectively connected to the intelligent control device 601 and the intelligent device 603;

[0239] The controller 602 of the vehicle 60 is used to execute a first instruction or execute a second instruction sent by the intelligent control device 601, so as to control whether the intelligent device 603 enters a sleep state.

[0240] Exemplarily, the controller is respectively connected to the intelligent control device and the intelligent device through communication wires. The controller can execute a first instruction generated by the controller, or execute a second instruction sent by the intelligent control device, so as to control whether the intelligent device enters a sleep state through the communication wires.

[0241] Furthermore, Figure 7 Schematic diagram of the structure of an exemplary vehicle Figure 2 , such as Figure 7 shown, the controller 602 of the vehicle 70 and the intelligent device 603 are respectively connected through a first channel 701 and a second channel 702.

[0242] When the controller 602 of the vehicle 70 executes the first instruction, it sends the first instruction to the smart device 603 through the first channel 701 to control whether the smart device 603 enters the sleep state; wherein, the controller 602 of the vehicle 70 executes the first instruction based on the lock instruction sent by the intelligent control device 601;

[0243] When the controller 602 of the vehicle 70 executes the second instruction sent by the intelligent control device 601, it sends the second instruction to the smart device 603 through the second channel 702 to control whether the smart device 603 enters the sleep state.

[0244] Exemplarily, the controller sends the first instruction to the smart device through the first channel, and the controller sends the second instruction to the smart device through the second channel, thereby controlling whether the smart device enters the sleep state.

[0245] Combined with the foregoing embodiments, if the intelligent control device sends a second instruction to the controller, the controller further sends the second instruction to the smart device through the second channel; if the intelligent control device sends a lock instruction to the controller, the controller further sends the first instruction to the smart device through the first channel.

[0246] Optionally, the second channel can transmit the second instruction and control the sleep state of the smart device, while the first channel can not only transmit the first instruction, control the sleep state of the smart device, but also control other devices in the vehicle. Other devices may include, but are not limited to: Advanced Driver Assistance Systems (ADAS), devices with an Automotive Safety Integrity Level D (ASIL-D) in the automotive electronic and electrical system.

[0247] In the above example, independent first and second channels are set up to separate the transmission channels of the first and second instructions, realizing hierarchical sleep management, which can avoid signal interference during the transmission of the first and second instructions. And it ensures that the second instruction generated by the model will not affect the sleep state of the core devices for vehicle driving.

[0248] The vehicle provided in this application is provided with an intelligent control device, which can execute the method provided in the above method embodiment. The implementation principle and technical effect are similar, and will not be elaborated here in this embodiment.

[0249] This application also provides a computer program product, including a computer program, which when executed by a processor implements the above method.

[0250] The present application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above method.

[0251] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0252] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0253] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0254] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0255] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0256] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, and other various media that can store program codes.

[0257] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: ROMs, RAMs, magnetic disks, or optical discs, and other various media that can store program codes.

[0258] Finally, it should be noted that: After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily think of other implementation manners of the present invention. The present invention is intended to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A control method for an intelligent device of a vehicle, characterized in that, Applied to a vehicle, including: In response to a first instruction issued by a controller in the vehicle, obtaining at least one piece of information data of the vehicle; wherein, the first instruction indicates whether the intelligent device in the vehicle enters a sleep state; the information data characterizes the user state of the user in the vehicle; Generating a second instruction according to the at least one piece of information data; wherein, the second instruction is used to indicate whether the intelligent device in the vehicle enters a sleep state; and determining a score value of the second instruction; If it is determined that the score value of the second instruction is greater than or equal to a first threshold, sending the second instruction to the controller for execution.

2. The method according to claim 1, wherein Generating a second instruction according to the at least one piece of information data, including: Inputting the at least one piece of information data into a preset intelligent recognition model to obtain the second instruction.

3. The method according to claim 2, characterized in that, Inputting the at least one piece of information data into a preset intelligent recognition model to obtain the second instruction, including: Performing feature extraction processing on the at least one piece of information data based on the preset intelligent recognition model to obtain user features corresponding to the at least one piece of information data; Performing feature fusion processing on each of the user features based on the cross-attention mechanism of the preset intelligent recognition model to obtain a fusion feature, and performing recognition on the fusion feature based on the preset intelligent recognition model to obtain the second instruction.

4. The method according to claim 2, wherein Before inputting the at least one piece of information data into a preset intelligent recognition model, the method further includes: Performing time series alignment processing on the at least one piece of information data so that the time between each piece of information data is unified.

5. The method according to claim 2, wherein The method further includes: Performing reinforcement learning processing on the preset intelligent recognition model according to the first instruction and the second instruction to update the parameters of the preset intelligent recognition model.

6. The method according to claim 2, wherein The method further includes: Receiving a parameter configuration file sent by a server, and configuring the parameters of an initial model corresponding to the preset intelligent recognition model according to the parameter configuration file to obtain the preset intelligent recognition model; Wherein, the parameter configuration file is obtained by the server based on the information data of multiple vehicles.

7. The method according to claim 1, wherein The at least one piece of information data includes: pupil data of the user in the vehicle, user image of the user in the vehicle, personnel distribution information of the user in the vehicle, seat pressure information in the vehicle, air flow state information in the vehicle, user voice of the user in the vehicle.

8. The method according to claim 1, characterized in that, The second instruction has a confidence level, which is output by the preset intelligent recognition model that generates the second instruction; Determining the score value of the second instruction, including: Determining the delay time for collecting the at least one piece of information data; and determining the historical decision accuracy rate of the preset intelligent recognition model that generates the second instruction, wherein the historical decision accuracy rate indicates the accuracy rate of multiple historical second instructions; Determining the score value of the second instruction according to the delay time, the confidence level, and the historical decision accuracy rate.

9. The method according to claim 8, wherein Determining the score value of the second instruction according to the delay time, the confidence level, and the historical decision accuracy rate, including: Perform a weighted sum of the delay time, the confidence level, and the historical decision accuracy rate to obtain the score value of the second instruction; Among them, there is a negative correlation between the value of the delay time and the weight of the delay time, a positive correlation between the value of the confidence level and the weight of the confidence level, and a positive correlation between the value of the historical decision accuracy rate and the weight of the historical decision accuracy rate.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: If it is determined that the score value of the second instruction is less than the first threshold, determine the scenario information of the vehicle according to the driving state of the vehicle; If it is determined that the scenario information of the vehicle indicates that the vehicle is in a safe scenario, when the score value of the second instruction is greater than or equal to the second threshold, send the second instruction to the controller for execution; where the second threshold is less than the first threshold; If it is determined that the scenario information of the vehicle indicates that the vehicle is in a safe scenario, when the score value of the second instruction is less than the second threshold, send a locking instruction to the controller, where the locking instruction indicates that the controller needs to execute the first instruction.

11. The method according to claim 10, wherein The method further includes: If it is determined that the scenario information of the vehicle indicates that the vehicle is not in a safe scenario, send the locking instruction to the controller.

12. A control device for an intelligent device of a vehicle, characterized in that, Includes: An acquisition module, configured to acquire at least one information data of the vehicle in response to a first instruction issued by a controller in the vehicle; where the first instruction instructs whether an intelligent device in the vehicle enters a sleep state; the information data characterizes the user state of a user in the vehicle; A processing module, configured to generate a second instruction according to the at least one information data; where the second instruction is used to instruct whether an intelligent device in the vehicle enters a sleep state; and determine the score value of the second instruction; A sending module, configured to send the second instruction to the controller for execution if it is determined that the score value of the second instruction is greater than or equal to the first threshold.

13. An intelligent control device, characterized in that, Includes: A memory, a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-11.

14. A vehicle, characterized in that, An intelligent control device as described in claim 13 is provided in the vehicle.

15. The vehicle according to claim 14, wherein, The vehicle further includes: a controller and an intelligent device; The controller of the vehicle is respectively connected to the intelligent control device and the intelligent device; The controller of the vehicle is configured to execute the first instruction or execute the second instruction sent by the intelligent control device, so as to control whether the intelligent device enters a sleep state.

16. The vehicle according to claim 15, characterized in that The controller of the vehicle and the intelligent device are respectively connected through a first channel and a second channel; When the controller of the vehicle executes the first instruction, the first instruction is sent to the intelligent device through the first channel to control whether the intelligent device enters a sleep state; where the controller of the vehicle executes the first instruction based on the locking instruction sent by the intelligent control device; When the controller of the vehicle executes the second instruction sent by the intelligent control device, the second instruction is sent to the intelligent device through the second channel to control whether the intelligent device enters the sleep state.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1-11 when executed by a processor.

18. A computer program product, characterized in that, It includes a computer program, which implements the method according to any one of claims 1-11 when executed by a processor.

Images