Vehicle cabin control method and device and storage medium

Abstract

The invention discloses a vehicle cabin control method and device and a storage medium. The method comprises the steps that environment associated data of a current vehicle cabin and a currently displayed vehicle cabin environment model are obtained; adjusting the vehicle cabin environment model by using the environment associated data to obtain an adjusted vehicle cabin environment model; and controlling the vehicle cabin by using the adjusted vehicle cabin environment model. By means of the mode, the convenience and accuracy of controlling the environment in the vehicle by a driver can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a cabin control method, device and storage medium. Background Technology

[0002] With the development of automotive intelligence, drivers often need to observe the in-vehicle environment through in-vehicle video images and then call the corresponding control switches to control the cabin based on the observation results. However, the control switches for different objects in the cabin are often scattered, which means that after the driver completes the observation and judgment based on the in-vehicle video images, he still has to find the corresponding switch in order to make the corresponding control. This results in low cabin control efficiency and is prone to operational errors due to the driver's distraction. Summary of the Invention

[0003] The main technical problem addressed by this application is to provide a vehicle cabin control method, device, and storage medium that can improve the convenience and accuracy of the driver's control over the in-vehicle environment.

[0004] To solve the above-mentioned technical problems, one technical solution adopted in this application is: to provide a vehicle cabin control method, the method comprising: acquiring current vehicle cabin environmental correlation data and currently displayed vehicle cabin environmental model; adjusting the vehicle cabin environmental model using the environmental correlation data to obtain an adjusted vehicle cabin environmental model; and controlling the vehicle cabin using the adjusted vehicle cabin environmental model.

[0005] The process of adjusting the cabin environment model using environmental correlation data to obtain the adjusted cabin environment model includes: performing entity recognition using environmental correlation data to obtain the current entity recognition result; and adjusting the cabin environment model using the current entity recognition result to obtain the adjusted cabin environment model.

[0006] The current entity recognition result includes: sub-recognition results for each current entity; sub-recognition results include entity type and entity parameters; adjusting the cabin environment model using the current entity recognition result includes: using the current entity recognition result and historical entity recognition results from previous times to determine whether each current entity is a newly added entity; historical entities are entities that existed in the cabin at previous times, and historical entity recognition results include sub-recognition results for each historical entity; newly added entities are entities for which there is no corresponding historical entity in each current entity; the cabin environment model includes the entity model corresponding to each historical entity; for each newly added entity, the reference entity model of the newly added entity is called from the model library to the cabin environment model, and the model parameters of the reference entity model are adjusted until they are consistent with the entity parameters of the newly added entity; wherein, the reference entity model is a preset base model, or a reference model corresponding to an entity with the same type as the newly added entity; and for each non-new entity, the model parameters of the entity model corresponding to its matching historical entity are adjusted until they are consistent with the entity parameters of the non-new entity; both model parameters and entity parameters include at least one of the following: pose, size, and shape.

[0007] The process includes, before or during the process of adjusting the cabin environment model using the current entity recognition results to obtain the adjusted cabin environment model, the following steps: using at least one of the environmental association data and the current entity recognition results to perform scene recognition, and obtaining scene recognition results for at least one preset scene; the scene recognition results are used to characterize whether there are any anomalies in the corresponding preset scene; and adjusting the cabin environment model using the scene recognition results of each preset scene.

[0008] The process involves adjusting the vehicle cabin environment model using the scene recognition results of each preset scenario. This includes: for each preset scenario, in response to the scene recognition results indicating an anomaly in the preset scenario, adding a prompt icon corresponding to the preset scenario to the vehicle cabin environment model; wherein the prompt icon is used to alert the user that an anomaly exists in the preset scenario; the prompt level varies for different preset scenarios.

[0009] The environmental association data includes at least one of the following: basic structural data of the vehicle cabin, multimodal environmental acquisition data, and cabin feedback data; the basic structural data of the vehicle cabin includes: spatial distribution data of seats; the cabin feedback data includes at least one of the following: spatial feedback data of seats and environmental feedback data of the vehicle cabin; the current entity recognition result is obtained by detecting the multimodal environmental acquisition data, and the current entity recognition result includes: the sub-recognition result of the passenger; the sub-recognition result includes at least one of the following: passenger type, passenger physiological representation data; scene recognition is performed using at least one of the environmental association data and the current entity recognition result to obtain a scene recognition result for at least one preset scene, including: responding to at least one preset scene. The system includes a passenger space recognition scenario. Using seat spatial distribution data and spatial feedback data, the system determines the passenger's current passenger space value. Based on a comparison between the current passenger space value and a suitable space threshold corresponding to the passenger type, the system determines the scenario recognition result for the passenger space recognition scenario. The scenario recognition result indicates whether the current passenger space value is less than the suitable space threshold. In response to at least one preset scenario, including an interference recognition scenario, the system uses physiological characterization data to identify whether a sleeping passenger exists in the cabin. In response to the presence of a sleeping passenger, the system determines the scenario recognition result corresponding to the interference recognition scenario based on cabin environmental feedback data. The scenario recognition result indicates whether there is an interfering environment that disrupts the passenger's sleep.

[0010] The environmental feedback data includes at least one of the following: feedback data on the opening and closing of vehicle windows, feedback data on the operation of lights, feedback data on the adjustment of player volume, and feedback data on air conditioning direction; physiological characterization data includes at least one of heart rate, respiratory data, posture, and eye closure degree; multimodal environmental acquisition data includes image data and point cloud data.

[0011] The vehicle cabin environment model includes sub-models corresponding to each entity, and each sub-model integrates at least one functional control item; controlling the vehicle cabin using the adjusted vehicle cabin environment model includes: responding to receiving a user's operation command on the functional control item of any sub-model, controlling the entity corresponding to the sub-model.

[0012] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor coupled to each other, wherein the memory stores program instructions; and the processor is used to execute the program instructions stored in the memory to implement the above-mentioned method.

[0013] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing program instructions that can be executed to implement the above-mentioned method.

[0014] The above-described solution, after acquiring the current cabin environment data and the currently displayed cabin environment model, adjusts the cabin environment model in real time based on the current cabin environment data, and then uses the adjusted cabin environment model to control the cabin. It is evident that the cabin environment model of this application remains synchronized with the actual cabin environment. Compared to the method of controlling the cabin by finding corresponding switches after observing the driver's image, this application's method of controlling the cabin using a dynamically adjusted cabin environment model can reduce the operational delay caused by the driver being distracted by switches, as well as operational errors caused by distracted attention, thereby improving the convenience and accuracy of control. Attached Figure Description

[0015] Figure 1 This is a schematic flowchart of an embodiment of the cabin control method provided in this application; Figure 2 This is a flowchart illustrating an embodiment of the vehicle cabin environment model provided in this application; Figure 3 This is a flowchart illustrating another embodiment of the vehicle cabin environment model provided in this application; Figure 4 This is a schematic diagram of the framework of an embodiment of the cabin control device provided in this application; Figure 5 This is a schematic diagram of the framework of an embodiment of the electronic device provided in this application; Figure 6 This is a schematic diagram of the framework of the computer-readable storage medium provided in this application; Figure 7 This is a schematic diagram of an embodiment of the vehicle cabin environment model provided in this application. Detailed Implementation

[0016] To make the purpose, technical solution and effects of this application clearer and more explicit, the following describes this application in further detail with reference to the accompanying drawings and embodiments.

[0017] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0018] Please see Figure 1 , Figure 1This is a schematic flowchart of an embodiment of the vehicle cabin control method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes: S11: Obtain the current cabin environment data and the currently displayed cabin environment model.

[0019] like Figure 7 As shown, the currently displayed vehicle cabin environment model is a three-dimensional model (i.e., 3D model) of the vehicle cabin environment. This vehicle cabin environment model includes: a fixed frame model of the vehicle cabin and sub-models corresponding to each non-fixed entity outside the fixed frame; wherein, each fixed entity sub-model in the fixed frame model of the vehicle cabin integrates at least one functional control item, and the driver can operate the functional control items on each sub-model, thereby controlling the entity corresponding to the sub-model.

[0020] The fixed frame model of the vehicle cabin is a basic model constructed based on the vehicle body size and the entity parameter information of the fixed entities in the vehicle cabin. The fixed frame model of the vehicle cabin includes the vehicle cabin model, seat model, steering wheel model, window model, etc. The fixed entities are such as the vehicle cabin body, seats, steering wheel, windows, etc. The non-fixed entities outside the fixed frame are such as passengers, drivers, water cups, backpacks, etc.

[0021] In this embodiment, the vehicle cabin environment model is displayed on an in-vehicle display device, such as a central control screen or a HUD (Head-Up Display). Each sub-model within the vehicle cabin environment model is a 3D model. The driver can adjust the viewing angle of each sub-model through the in-vehicle display device, thereby gaining a clear view of the entire interior environment of the vehicle cabin and reducing blind spots.

[0022] In one embodiment, the environmental data associated with the vehicle cabin includes at least one of the following: basic structural data of the vehicle cabin, multimodal environmental acquisition data, and cabin feedback data.

[0023] The basic structural data of the vehicle cabin includes: the three-dimensional dimensions of the vehicle cabin, the spatial layout data of the vehicle cabin, and the spatial distribution data of the seats (such as seat geometric parameters and installation positions). The spatial layout data of the vehicle cabin includes the spatial distribution data of other basic / fixed entities, such as the steering wheel, air conditioning, ambient lighting, interior structure, and various installed sensors. The basic structural data of the vehicle cabin provides a reference coordinate system for subsequent data processing.

[0024] The multimodal environment data collected includes at least two of the following: image data, point cloud data, and temperature distribution data inside the vehicle cabin. Among them, image data is two-dimensional data acquired using cameras, point cloud data is three-dimensional data acquired using radar, and temperature distribution data inside the vehicle cabin is spatial temperature spectrum data acquired using infrared thermal imaging cameras.

[0025] The cabin feedback data includes at least one of the following: seat space feedback data and cabin environment feedback data. Seat space feedback data refers to the seat space data after adjustment (e.g., backrest angle, fore-and-aft movement distance, etc.). Cabin environment feedback data includes at least one of the following: window opening / closing status feedback data, light operation status feedback data, player volume adjustment feedback data, and air conditioning direction feedback data. Each feedback data point represents the status data of the corresponding fixed entity, fed back to the cabin by the controller of that fixed entity after control is applied. For example, window opening / closing status feedback data is the window status data fed back to the cabin by the window controller after window opening / closing control is applied; air conditioning direction feedback data is the air conditioning direction data fed back to the cabin by the air conditioning controller after air conditioning direction adjustment is applied.

[0026] S12: Adjust the cabin environment model using environmental correlation data to obtain the adjusted cabin environment model.

[0027] In one embodiment, entity recognition can be performed in real time using environmental correlation data to obtain the current entity recognition result; then, the cabin environment model can be adjusted using the current entity recognition result to obtain the adjusted cabin environment model.

[0028] In one specific embodiment, the current entity recognition result includes: sub-recognition results for each current entity; the sub-recognition results include entity type and entity parameters; wherein, each current entity is a non-fixed entity; for example, a passenger, a backpack, a mobile phone, or other entities that do not belong to the fixed entities in the cabin.

[0029] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of adjusting a vehicle cabin environment model provided in this application. In this embodiment, adjusting the vehicle cabin environment model using the current entity recognition results includes: S21: Using the current entity recognition results and the historical entity recognition results at historical moments, determine whether each current entity is a newly added entity; historical entities are entities that existed in the cabin at historical moments, and the historical entity recognition results include the sub-recognition results of each historical entity; newly added entities are entities that do not have a corresponding historical entity in each current entity; the cabin environment model includes the entity model corresponding to each historical entity.

[0030] S22: For each newly added entity, retrieve the reference entity model of the newly added entity from the model library and transfer it to the cabin environment model, and adjust the model parameters of the reference entity model until they are consistent with the entity parameters of the newly added entity; wherein, the reference entity model is a preset base model or a reference model corresponding to an entity of the same type as the newly added entity; and, for each non-new entity, adjust the model parameters of the entity model corresponding to its matching historical entity until they are consistent with the entity parameters of the non-new entity.

[0031] This embodiment uses the current entity recognition results and historical entity recognition results from previous times to determine whether each current entity is a newly added entity; then, it distinguishes between newly added and non-new entities and adjusts the cabin environment model accordingly. Specifically: for newly added entities, the reference entity model of the newly added entity is first retrieved from the model library and added to the cabin environment model, and then the reference entity model is adjusted to match the entity parameters of the newly added entity. For non-new entities, since the currently displayed cabin environment model already contains the entity model of the corresponding matching historical entity, the entity model can be adjusted based on the entity model of the matching historical entity until it matches the entity parameters of the corresponding non-new entity.

[0032] The aforementioned model parameters and entity parameters each include at least one of the following: pose, size, and shape. Taking the addition of a new entity as an example, the model pose of the corresponding reference entity model can be adjusted to match the pose of the new entity based on the pose; the model pose of the corresponding reference entity model can be adjusted to match the shape of the new entity based on the shape; and the model pose of the corresponding reference entity model can be adjusted to match the size of the new entity based on the size.

[0033] Furthermore, given the limited storage capacity of the model library, it cannot cover the entity models of all entities. Therefore, entity models of common entities can be stored in the model library first. For some uncommon entities, or entities whose corresponding types cannot be identified, a preset basic model can be determined for them, such as a sphere model or a cube model. If the corresponding type of entity cannot be identified, or if there is no entity model of the corresponding type in the model library, the preset basic model can be called from the model library as the reference entity model for the newly added entity, and then adjusted based on the entity parameters of the newly added entity.

[0034] Of course, subsequent iterations and optimizations can be made to add more entity models, or to refine the modeling of existing entity models and add more detailed features.

[0035] In one specific embodiment, the current vehicle cabin environment-related data includes: image data and point cloud data of the vehicle interior environment; the current entity recognition results are obtained by recognizing the image data and point cloud data of the vehicle interior environment; wherein, the image data is two-dimensional data collected using cameras, etc., and the point cloud data is three-dimensional point cloud data collected using radar. Recognizing the image data yields the corresponding entity type recognition result; recognizing the point cloud data yields the three-dimensional size information of the corresponding entity, i.e., the entity parameter information corresponding to the entity.

[0036] It should be noted that, in one embodiment, a target tracking algorithm can be used to associate the current entity recognition result of the current frame with the historical entity recognition result of the historical frames collected at historical moments, and determine whether each current entity is a newly added entity by matching the spatiotemporal features of the entities; in another embodiment, an image feature matching algorithm can also be used to extract the visual features of the entities in the current frame and the entities in the historical frames and perform similarity calculation, and distinguish between newly added entities and non-new entities in the current frame based on the matching results.

[0037] In summary, Figure 2 The embodiment shown is used to adjust the cabin environment model using the current entity recognition results, so that the adjusted cabin environment model keeps the entities in the actual vehicle interior environment synchronized.

[0038] In another embodiment, the temperature distribution data inside the vehicle cabin can also be used to adjust the vehicle cabin environment model; for example, the collected temperature distribution data inside the vehicle cabin can be superimposed onto the vehicle cabin environment model in real time, so that the driver can intuitively see the temperature conditions in each area.

[0039] In another embodiment, before or during the process of adjusting the cabin environment model using the current entity recognition results to obtain the adjusted cabin environment model, scene recognition can be performed using at least one of the environmental association data and the current entity recognition results to obtain scene recognition results for at least one preset scene; then, the cabin environment model is adjusted using the scene recognition results of each preset scene. The scene recognition results of the preset scenes are used to characterize whether there are any anomalies in the corresponding preset scene.

[0040] Specifically, please refer to Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of the vehicle cabin environment model provided in this application. This embodiment includes: S31: Use at least one of the environmental association data and the current entity recognition result to perform scene recognition and obtain a scene recognition result for at least one preset scene; the scene recognition result is used to characterize whether there is an anomaly in the corresponding preset scene.

[0041] In this embodiment, the environmental correlation data includes at least one of the following: basic structural data of the vehicle cabin, multimodal environmental acquisition data, and cabin feedback data; the basic structural data of the vehicle cabin includes: spatial distribution data of seats; the cabin feedback data includes at least one of the following: spatial feedback data of seats and environmental feedback data of the vehicle cabin.

[0042] The current entity recognition result is obtained by detecting data collected from the multimodal environment. The current entity recognition result includes: the passenger sub-recognition result; the sub-recognition result includes at least one of the following: passenger type, passenger physiological representation data.

[0043] In this embodiment, at least one preset scenario includes at least one of the following: a passenger space recognition scenario and an interference recognition scenario. Of course, multiple scenarios can be preset according to actual needs, which will not be listed here.

[0044] In one implementation scenario, at least one preset scenario includes a passenger space recognition scenario. In this scenario, the current passenger space value can be determined by first using the spatial distribution data and spatial feedback data of the seats. Based on the comparison between the current passenger space value and the appropriate space threshold corresponding to the passenger type, the scenario recognition result of the passenger space recognition scenario is determined. The scenario recognition result is used to characterize whether the current passenger space value is less than the appropriate space threshold.

[0045] The spatial distribution data of the seats includes seat geometric parameters and setting positions, while the seat space feedback data consists of seat space data after adjustment (e.g., backrest angle, fore-and-aft movement distance). Based on these two data points, the passenger's current seating space value can be calculated. Then, a suitable space threshold is obtained according to the passenger type (e.g., male / female, child / elderly / adult). It is then determined whether the current seating space value is less than the suitable space threshold. If it is less than the suitable space threshold, the passenger's seating space is determined to be narrow, and the recognition result for the seating space identification scenario is abnormal.

[0046] In another implementation scenario, at least one preset scenario includes an interference identification scenario. In this implementation scenario, physiological characterization data can be used to identify whether there are sleeping passengers in the cabin. If there are sleeping passengers, the scene identification result corresponding to the interference identification scenario is determined based on the environmental feedback data of the cabin. The scene identification result is used to characterize whether there is an interfering environment that disturbs passengers' sleep.

[0047] The physiological data includes at least one of the following: heart rate, respiratory data, posture, and eye closure degree; the environmental feedback data includes at least one of the following: feedback data on window opening / closing status, lighting operation status, player volume adjustment, and air conditioning vent direction. Heart rate and respiratory data can be detected using millimeter-wave radar installed in the vehicle cabin, while posture and eye closure degree are obtained by image recognition of the collected in-vehicle environment image data using relevant recognition models; the method for obtaining environmental feedback data is described above.

[0048] In this implementation scenario, physiological data such as heart rate, respiration, posture, and eye closure can be used to determine whether a passenger is asleep (i.e., whether a sleeping passenger exists). If a sleeping passenger is present, data from the windows, lights, media player, and air conditioning can be used to determine if there is an environment that might disturb the passenger's sleep. For example, if environmental feedback data indicates that the passenger's side window is open or closed by more than 50%, the media player volume is too high, the rear lights are on, or the air conditioning is blowing directly on the sleeping passenger, then an environment that might disturb the passenger's sleep is identified, and the scene identification result corresponding to the interference identification scenario is considered abnormal.

[0049] S32: Adjust the cabin environment model using the scene recognition results of each preset scene.

[0050] For each preset scenario, if the scenario recognition result indicates that the preset scenario is abnormal, a prompt icon corresponding to the preset scenario is added to the vehicle cabin environment model; the added prompt icon is used to notify the user that the preset scenario is abnormal.

[0051] In one implementation scenario, the prompt icon corresponding to the preset scenario can be a graphic, such as a red circle or triangle. This graphic can be pre-stored in a model library, and when an abnormal preset scenario is detected, the corresponding graphic is retrieved from the model library and applied to the vehicle cabin environment model. It should be noted that different preset scenarios correspond to different graphic prompts so that users can understand which preset scenario in the current vehicle cabin environment has encountered an anomaly through the specific graphic.

[0052] In another implementation scenario, the prompt icon corresponding to the preset scenario can also be a prompt text. When an abnormal preset scenario is detected, the corresponding prompt text is added to the displayed cabin environment model to prompt the user (driver) that the preset scenario is abnormal. The prompt text is, for example, "The passenger has fallen asleep and the volume can be turned down appropriately."

[0053] In some implementation scenarios, different preset scenarios correspond to different levels of anomalies. For example, some scenario anomalies do not involve safety, such as the temperature and humidity inside the car deviating from the preset comfort range, or the space between the front and rear seats being less than the preset comfort threshold. However, some scenario anomalies do involve safety, such as a child in the rear seat accidentally touching the door unlock button or the window lift button, or rear passengers (especially children and the elderly) experiencing limb convulsions or sudden fainting.

[0054] In this implementation scenario, different levels of prompts can be provided based on the safety level corresponding to different preset scenarios. For example, for preset scenarios that do not involve safety, prompts can be provided using text or light-colored graphics; for preset scenarios that involve safety, both text and prominent graphics can be added to the vehicle cabin environment model.

[0055] In some embodiments, feedback data from the air conditioning system and feedback data from the opening and closing of the windows can be further integrated to predict wind field data in the cabin environment. The wind field data can then be visualized and overlaid onto the cabin environment model to allow the driver to intuitively observe the wind field data in the cabin environment, thereby facilitating accurate decision-making.

[0056] S13: Control the cabin using the adjusted cabin environment model.

[0057] In this embodiment, the cabin environment model includes sub-models corresponding to each entity, and each sub-model integrates at least one functional control item. The entities in this embodiment mainly refer to the fixed entities mentioned above.

[0058] In one implementation scenario, a user can observe the in-vehicle environment through the adjusted cabin environment model, and after observing an anomaly, operate on at least one functional control item on the sub-model of the corresponding entity to control the corresponding entity.

[0059] In one implementation scenario, users can also operate on at least one functional control item on the sub-model of the corresponding entity based on the prompt text and / or prompt graphics in the cabin environment model. After receiving the user's operation instruction on any functional control item on the sub-model, the system can respond to the operation instruction and control the entity corresponding to the operated sub-model.

[0060] The functional control items of each sub-model can be displayed on the same page or on different pages. For example, after a user clicks on the functional control item on the physical model corresponding to the air conditioner, they can enter the specific air conditioner control page, which includes multiple specific functional control items.

[0061] The above-described solution, after acquiring the current cabin environment data and the currently displayed cabin environment model, adjusts the cabin environment model in real time based on the current cabin environment data, and then uses the adjusted cabin environment model to control the cabin. It is evident that the cabin environment model of this application remains synchronized with the actual cabin environment. Compared to the method of controlling the cabin by finding corresponding switches after observing the driver's image, this application's method of controlling the cabin using a dynamically adjusted cabin environment model can reduce the operational delay caused by the driver being distracted by switches, as well as operational errors caused by distracted attention, thereby improving the convenience and accuracy of control.

[0062] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the vehicle cabin control device provided in this application. In this embodiment, the vehicle cabin control device 40 includes an acquisition module 41, an adjustment module 42, and a control module 43. The acquisition module 41 is used to acquire the current environmental association data of the vehicle cabin and the currently displayed vehicle cabin environment model; the adjustment module 42 is used to adjust the vehicle cabin environment model using the environmental association data to obtain the adjusted vehicle cabin environment model; the control module 43 is used to control the vehicle cabin using the adjusted vehicle cabin environment model.

[0063] In some embodiments, the adjustment module 42 adjusts the cabin environment model using environmental association data to obtain an adjusted cabin environment model, including: performing entity recognition using environmental association data to obtain the current entity recognition result; and adjusting the cabin environment model using the current entity recognition result to obtain the adjusted cabin environment model.

[0064] In some embodiments, the current entity recognition result includes: sub-recognition results of each current entity; the sub-recognition results include entity type and entity parameters; adjusting the cabin environment model using the current entity recognition result includes: determining whether each current entity is a new entity using the current entity recognition result and the historical entity recognition result at a historical time; historical entities are entities that existed in the cabin at a historical time, the historical entity recognition result includes the sub-recognition results of each historical entity, and new entities are entities that do not have a corresponding historical entity in each current entity; the cabin environment model includes the entity model corresponding to each historical entity; for each new entity, calling the reference entity model of the new entity from the model library to the cabin environment model, and adjusting the model parameters of the reference entity model until they are consistent with the entity parameters of the new entity; wherein, the reference entity model is a preset basic model, or a reference model corresponding to an entity with the same type as the new entity; and for each non-new entity, adjusting the model parameters of the entity model corresponding to its matching historical entity until they are consistent with the entity parameters of the non-new entity.

[0065] In some embodiments, before or during the process of adjusting the cabin environment model using the current entity recognition result to obtain the adjusted cabin environment model, the method further includes: performing scene recognition using at least one of the environmental association data and the current entity recognition result to obtain a scene recognition result for at least one preset scene; the scene recognition result is used to characterize whether there is an anomaly in the corresponding preset scene; and adjusting the cabin environment model using the scene recognition results of each preset scene.

[0066] In some embodiments, the cabin environment model is adjusted using the scene recognition results of each preset scenario, including: for each preset scenario, in response to the scene recognition results of the preset scenario indicating that there is an anomaly in the preset scenario, adding a prompt icon corresponding to the preset scenario to the cabin environment model; wherein, the prompt icon is used to prompt the user that there is an anomaly in the preset scenario; the prompt level is different for different preset scenarios.

[0067] In some embodiments, the environmental association data acquired by the acquisition module 41 includes at least one of the following: basic structural data of the vehicle cabin, multimodal environmental acquisition data, and cabin feedback data; the basic structural data of the vehicle cabin includes: spatial distribution data of seats; the cabin feedback data includes at least one of the following: spatial feedback data of seats and environmental feedback data of the vehicle cabin; the current entity recognition result is obtained by detecting the multimodal environmental acquisition data, and the current entity recognition result includes: the sub-recognition result of the passenger; the sub-recognition result includes at least one of the following: passenger type, passenger physiological representation data; scene recognition is performed using at least one of the environmental association data and the current entity recognition result to obtain a scene recognition result for at least one preset scene, including: In response to at least one preset scenario including a passenger space recognition scenario, the system utilizes seat spatial distribution data and spatial feedback data to determine the passenger's current passenger space value. Based on the comparison between the current passenger space value and a suitable space threshold corresponding to the passenger type, the system determines the scenario recognition result for the passenger space recognition scenario. The scenario recognition result is used to characterize whether the current passenger space value is less than the suitable space threshold. In response to at least one preset scenario including an interference recognition scenario, the system utilizes physiological characterization data to identify whether there is a sleeping passenger in the cabin. In response to the presence of a sleeping passenger, the system determines the scenario recognition result corresponding to the interference recognition scenario based on cabin environmental feedback data. The scenario recognition result is used to characterize whether there is an interfering environment that disturbs the passenger's sleep.

[0068] In some embodiments, environmental feedback data includes at least one of the following: window opening / closing status feedback data, light operation status feedback data, player volume adjustment feedback data, and air conditioning vent direction feedback data; physiological characterization data includes at least one of heart rate, respiratory data, posture, and eye closure degree; multimodal environmental acquisition data includes image data and point cloud data.

[0069] In some embodiments, the cabin environment model includes sub-models corresponding to each entity, and each sub-model integrates at least one functional control item; the control module 43 controls the cabin using the adjusted cabin environment model, including: in response to receiving an operation instruction from the user on the functional control item of any sub-model, controlling the entity corresponding to the sub-model.

[0070] Please see Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the electronic device provided in this application. In this embodiment, the electronic device 50 includes a memory 51 and a processor 52 coupled to each other.

[0071] The memory 51 stores program instructions, and the processor 52 executes the program instructions stored in the memory 51 to implement the steps of any of the above-described method implementations. In a specific implementation scenario, the electronic device 50 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 50 may also include mobile devices such as laptops and tablets, which are not limited here.

[0072] Specifically, processor 52 controls itself and memory 51 to implement the steps of any of the above embodiments. Processor 52 may also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 52 may be implemented using integrated circuit chips.

[0073] Please see Figure 6 , Figure 6This is a schematic diagram of the framework of the computer-readable storage medium provided in this application. The computer-readable storage medium 60 of this application embodiment stores program instructions 61, which, when executed, implement the methods provided in any embodiment or any non-conflicting combination of the above-described methods. The program instructions 61 can form a program file and be stored in the computer-readable storage medium 60 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned computer-readable storage medium 60 includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0074] The above-described solution, after acquiring the current cabin environment data and the currently displayed cabin environment model, adjusts the cabin environment model in real time based on the current cabin environment data, and then uses the adjusted cabin environment model to control the cabin. It is evident that the cabin environment model of this application remains synchronized with the actual cabin environment. Compared to the method of controlling the cabin by finding corresponding switches after observing the driver's image, this application's method of controlling the cabin using a dynamically adjusted cabin environment model can reduce the operational delay caused by the driver being distracted by switches, as well as operational errors caused by distracted attention, thereby improving the convenience and accuracy of control.

[0075] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0076] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0079] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0082] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A vehicle cabin control method, characterized in that, The method includes: Obtain the current cabin environment data and the currently displayed cabin environment model; The vehicle cabin environment model is adjusted using the environmental correlation data to obtain the adjusted vehicle cabin environment model; The cabin is controlled using the adjusted cabin environment model.

2. The method according to claim 1, characterized in that, The step of adjusting the vehicle cabin environment model using the environmental correlation data to obtain the adjusted vehicle cabin environment model includes: Entity recognition is performed using the aforementioned environmental association data to obtain the current entity recognition result; The vehicle cabin environment model is adjusted using the current entity recognition results to obtain the adjusted vehicle cabin environment model.

3. The method according to claim 2, characterized in that, The current entity recognition result includes: sub-recognition results for each current entity; the sub-recognition results include entity type and entity parameters; The step of adjusting the vehicle cabin environment model using the current entity recognition result includes: Using the current entity recognition result and the historical entity recognition result at a historical moment, it is determined whether each current entity is a newly added entity; the historical entity is an entity that existed in the cabin at a historical moment, the historical entity recognition result includes the sub-recognition result of each historical entity, and the newly added entity is an entity that does not have a corresponding historical entity in each of the current entities; the cabin environment model includes the entity model corresponding to each historical entity; For each newly added entity, a reference entity model of the newly added entity is retrieved from the model library and applied to the vehicle cabin environment model. The model parameters of the reference entity model are then adjusted until they match the entity parameters of the newly added entity. The reference entity model is either a preset base model or a reference model corresponding to an entity of the same type as the newly added entity. For each non-new entity, adjust the model parameters of the entity model corresponding to the historical entity until they are consistent with the entity parameters of the non-new entity; the model parameters and the entity parameters each include at least one of the following: pose, size, and shape.

4. The method according to claim 2, characterized in that, Before or during the process of adjusting the vehicle cabin environment model using the current entity recognition result to obtain the adjusted vehicle cabin environment model, the method further includes: Scene recognition is performed using at least one of the environmental association data and the current entity recognition result to obtain a scene recognition result for at least one preset scene; the scene recognition result is used to characterize whether there is an anomaly in the corresponding preset scene; The cabin environment model is adjusted using the scene recognition results of each preset scenario.

5. The method according to claim 4, characterized in that, The step of adjusting the vehicle cabin environment model using the scene recognition results of each preset scenario includes: For each of the preset scenarios, in response to the scenario recognition result indicating that the preset scenario is abnormal, a prompt icon corresponding to the preset scenario is added to the cabin environment model; wherein, the prompt icon is used to prompt the user that the preset scenario is abnormal; the prompt level is different for different preset scenarios.

6. The method according to claim 4, characterized in that, The environmental data includes at least one of the following: basic structural data of the vehicle cabin, multimodal environmental acquisition data, and cabin feedback data; the basic structural data of the vehicle cabin includes: spatial distribution data of seats; the cabin feedback data includes at least one of the following: spatial feedback data of seats and environmental feedback data of the vehicle cabin. The current entity recognition result is obtained by detecting the multimodal environment data. The current entity recognition result includes: passenger sub-recognition result; the sub-recognition result includes at least one of the following: passenger type, passenger physiological representation data; The step of using at least one of the environmental association data and the current entity recognition result to perform scene recognition and obtain a scene recognition result for at least one preset scene includes: In response to at least one preset scenario including a passenger space recognition scenario, the current passenger space value is determined using seat spatial distribution data and the spatial feedback data. Based on the comparison between the current passenger space value and the appropriate space threshold corresponding to the passenger type, the scenario recognition result of the passenger space recognition scenario is determined. The scenario recognition result is used to characterize whether the current passenger space value is less than the appropriate space threshold. In response to at least one preset scenario including an interference identification scenario, the physiological characterization data is used for identification to determine whether there is a sleeping passenger in the cabin; in response to the presence of a sleeping passenger, the scene identification result corresponding to the interference identification scenario is determined based on the environmental feedback data of the cabin; the scene identification result is used to characterize whether there is an interfering environment that disturbs the passenger's sleep.

7. The method according to claim 6, characterized in that, The environmental feedback data includes at least one of the following: feedback data on the opening and closing of vehicle windows, feedback data on the operation of lights, feedback data on the adjustment of player volume, and feedback data on air conditioning direction. The physiological data include at least one of heart rate, respiratory data, posture, and eye closure. The multimodal environment acquisition data includes image data and point cloud data.

8. The method according to claim 1, characterized in that, The vehicle cabin environment model includes sub-models corresponding to each entity, and each sub-model integrates at least one functional control item; The method of controlling the vehicle cabin using the adjusted vehicle cabin environment model includes: In response to receiving a user's operation command for a function control item on any sub-model, the entity corresponding to the sub-model is controlled.

9. An electronic device, characterized in that, Including interconnected memory and processor, The memory stores program instructions; The processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the method of any one of claims 1-8.

Images