Vehicle-mounted fragrance control method and device, vehicle machine system and storage medium

Abstract

The application provides a vehicle-mounted fragrance control method and device, a vehicle machine system and a storage medium, and belongs to the technical field of vehicle control. The method provided in the application embodiment can obtain the first emotion of the driver by collecting the facial image of the driver, predict the second emotion of the driver through the first emotion, prepare for emitting the corresponding odor, match the target odor type corresponding to the second emotion in the matching relationship table of odor types and emotion information types, the odor of the target odor type can specifically affect the second emotion of the driver, then output an odor switching prompt to prompt the driver to switch the odor type, and in response to the confirmation switching operation on the target odor type, switch the current odor type to the target odor type, thereby affecting the emotion of the driver, improving the intelligent degree of the vehicle-mounted fragrance system, and further improving the fragrance experience of the driver.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, in particular to a vehicle-mounted fragrance control method and device, a car system and a storage medium. BACKGROUND

[0002] With the development of the vehicle industry, the demand for personalization of vehicle-mounted systems is gradually increasing. As one of the vehicle-mounted systems, the vehicle-mounted fragrance system can provide an odor environment in the vehicle. Research shows that odor and human emotion can influence each other, and different odors can affect different emotions. Therefore, how to control the emission of odor based on the vehicle-mounted fragrance system to affect the emotions of the driver and improve the fragrance experience of the driver is the focus of research in the field.

[0003] At present, the common way to control the emission of odor by the vehicle-mounted fragrance system is that the vehicle-mounted fragrance system emits odor based on the odor type selected by the user.

[0004] However, the above technical solution can only change the odor type in the vehicle based on the selection operation of the user, and the degree of intelligence is not high, which affects the fragrance experience of the driver. SUMMARY

[0005] The embodiments of the present application provide a vehicle-mounted fragrance control method, device, car system and storage medium, which are used to control the emission of odor based on the vehicle-mounted fragrance system, improve the degree of intelligence of the vehicle-mounted fragrance system, and further improve the fragrance experience of the driver. The technical solution is as follows:

[0006] On the one hand, a vehicle-mounted fragrance control method is provided, which comprises the following steps:

[0007] Collecting a facial image of a driver, obtaining at least one first emotional information of the driver based on the facial image, the at least one first emotional information being used to represent the emotion of the driver in a first time period;

[0008] Based on the at least one first emotional information, predicting second emotional information, the second emotional information being used to represent the emotion of the driver in a second time period, the second time period being a time period after the first time period;

[0009] Based on the second emotional information, obtaining a target odor type in a matching relationship table of odor types and emotional information types, the target odor type matching the type of the second emotional information;

[0010] Outputting an odor switching prompt, the odor switching prompt being used to prompt to switch the current odor type to the target odor type;

[0011] In response to a confirmation switching operation on the target odor type, switching the current odor type to the target odor type.

[0012] In another aspect, a vehicle-mounted fragrance control device is provided, the device comprising:

[0013] an emotion information acquisition module, configured to capture a facial image of a driver and acquire at least one first emotion information of the driver based on the facial image, the at least one first emotion information being used to represent the driver's emotion within a first time period;

[0014] a prediction module, configured to predict second emotion information based on at least one first emotion information, where the second emotion information is used to represent the driver's emotion in a second time period, where the second time period is a time period after the first time period;

[0015] a target odor type acquisition module, configured to acquire a target odor type from a matching relationship table of odor types and emotion information types based on the second emotion information, wherein the target odor type matches the type of the second emotion information;

[0016] a prompt module, configured to output an odor switching prompt, wherein the odor switching prompt is used to prompt the user to switch the current odor type to the target odor type;

[0017] The switching module is configured to switch the current odor type to the target odor type in response to a confirmation switching operation on the target odor type.

[0018] In some embodiments, the prediction module is configured to:

[0019] Determine the number of occurrences of the first emotion information in each sub-time period within the first time period, and determine the duration ratio of the first emotion information in the first time period based on the first emotion information and the number of sub-time periods within the first time period;

[0020] The first emotion information whose duration ratio is higher than the target duration ratio is determined as the second emotion information.

[0021] In some embodiments, the apparatus further comprises:

[0022] An emotion information type acquisition module is used to acquire multiple emotion information types from an emotion type library, where the emotion type library includes multiple different emotion information types;

[0023] an odor type acquisition module, configured to acquire multiple odor types possessed by multiple fragrance diffusion devices in a vehicle based on radio frequency signals emitted by the multiple fragrance diffusion devices, wherein the fragrance diffusion devices are configured to emit odors of a single odor type;

[0024] A display module, used to display multiple types of emotion information and multiple types of smells in the form of an interface;

[0025] The generating module is configured to generate a matching relationship table between the odor type and the emotion information type in response to a driver's matching operation on the multiple emotion information types and the multiple odor types.

[0026] In some embodiments, the switching module is configured to:

[0027] Locating a target type of fragrance diffusion device, where the target type is a type of fragrance diffusion device capable of emitting a target odor type;

[0028] If the odor type in the vehicle is not the target odor type within the first time period, controlling the currently working fragrance diffusion device to stop emitting the odor and controlling the fragrance diffusion device to emit the target odor type;

[0029] If no odor emission is detected within the first time period, the target type of fragrance diffusion device is controlled to emit the target odor type of odor.

[0030] On the other hand, a computer device is provided, which includes a processor and a memory, the memory being used to store at least one computer program, and the at least one computer program being loaded and executed by the processor to implement the operations performed by the in-vehicle fragrance control method in the embodiment of the present application.

[0031] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the vehicle fragrance control method in the embodiment of the present application.

[0032] On the other hand, a computer program product or computer program is provided, which includes computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the in-vehicle fragrance control method provided in various optional implementations of any of the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 Schematic diagram of an implementation environment of a vehicle-mounted fragrance control method provided in accordance with an embodiment of the present application;

[0035] Figure 2 This is a flow chart of a vehicle-mounted fragrance control method provided according to an embodiment of the present application;

[0036] Figure 3This is a flow chart of a vehicle-mounted fragrance control method provided in accordance with an embodiment of the present application;

[0037] Figure 4 This is a flow chart of a vehicle-mounted fragrance control method provided according to an embodiment of the present application;

[0038] Figure 5 This is a flow chart of a vehicle-mounted fragrance control method provided in accordance with an embodiment of the present application;

[0039] Figure 6 This is a flow chart of a vehicle-mounted fragrance control method provided in accordance with an embodiment of the present application;

[0040] Figure 7 is a block diagram of a vehicle-mounted fragrance control device provided according to an embodiment of the present application;

[0041] Figure 8 is a structural diagram of a computer device provided according to an embodiment of the present application;

[0042] Figure 9 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0044] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0045] In the present application, the term "at least one" means one or more, and the term "plurality" means two or more.

[0046] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the facial images involved in this application were obtained with full authorization.

[0047] Figure 1 Schematic diagram of the implementation environment of a vehicle fragrance control method provided in accordance with an embodiment of the present application. Figure 1 , the implementation environment is the vehicle system.

[0048] Among them, the vehicle system includes a computer device 101. In some embodiments, the computer device is installed and runs an application that supports data processing. The application is used to process emotional information data and odor type data. Schematically, the computer device 101 is a computer device used by the user. The user can trigger the data processing function on the computer device 101 to determine the target odor type, thereby realizing the control of the odor emission in the car by the vehicle fragrance system.

[0049] In some embodiments, the vehicle system includes a DMS (Driver Monitoring System) and a vehicle-mounted fragrance system. The DMS is mainly responsible for collecting the driver's facial image and predicting emotional information. The vehicle-mounted fragrance system is mainly responsible for obtaining the odor type and controlling the emission of odors in the vehicle. The vehicle-mounted fragrance system includes a CLM (Climate Module, air conditioning controller), a fragrance controller and a fragrance diffusion device. The air conditioning controller detects the in-vehicle environment through sensors, including the in-vehicle temperature, etc. The fragrance controller is mainly responsible for controlling the fragrance diffusion device to emit odors of different types. The fragrance diffusion device is mainly responsible for emitting odors of different types. The air conditioning controller, fragrance controller and fragrance diffusion device work together to implement the in-vehicle fragrance control method provided in the embodiments of the present application.

[0050] In some embodiments, the vehicle system can download data from the server 102, wherein the server 102 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), big data and artificial intelligence platforms. The server is used to provide background services for the vehicle system. In some embodiments, the server undertakes the main computing work and the vehicle system undertakes the secondary computing work; or, the server undertakes the secondary computing work and the vehicle system undertakes the main computing work; or, the server and the vehicle system adopt a distributed computing architecture for collaborative computing.

[0051] Among them, the vehicle system and the server can be connected directly or indirectly through wired or wireless communication, and this application does not impose any restrictions here.

[0052] In some embodiments, the vehicle system can also be connected to a terminal, which can be a smart phone, a tablet computer, a laptop computer, etc., but is not limited to this. The terminal has a functional module that supports facial recognition, and the functional module that supports facial recognition can be a social application and a multimedia application, etc.

[0053] For example, the application that supports facial recognition is a social application. It first shoots a video and then sends the shot video to a server. After receiving the video, the server uses the in-vehicle fragrance control method provided in the embodiment of the present application to determine the facial expression characteristics of the characters in the video, thereby determining the emotional information of the characters in the video. Optionally, the emotional information of the characters is displayed on the GUI (Graphical User Interface) interface in the vehicle system.

[0054] In some embodiments, the wireless or wired network described above uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network, or any combination of a virtual private network. In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the above-mentioned data communication technologies.

[0055] Figure 2 This is a flow chart of a vehicle-mounted fragrance control method provided in an embodiment of the present application. Figure 2 As shown, in the embodiment of the present application, the vehicle system is used as an example for description. The vehicle fragrance control method includes the following steps:

[0056] 201. A DMS in a vehicle-mounted system collects a facial image of a driver, and obtains at least one first emotion information of the driver based on the facial image, where the at least one first emotion information is used to represent the driver's emotion in a first time period.

[0057] In some embodiments, the collection of the facial image of the driver is based on a DMS implementation, the DMS collects an image or a video stream containing a human face based on a camera in a shooting module, and then the DMS transmits the image or the video stream to a detection module, detects a human face in the image and the video stream based on the detection module, thereby collecting the facial image of the driver, and obtains the first emotional information of the driver through face feature extraction and other operations on the facial image of the driver, the first emotional information includes the emotion of the driver in a first time period, for example, happy, angry, sad, etc.

[0058] The first time period is a certain target time period in which the DMS detects a human face after the DMS is started, and the length of the target time period can be set to ten minutes or other lengths, which is not limited in the embodiments of the application.

[0059] 202. The DMS in the car machine system predicts second emotional information based on the at least one first emotional information, the second emotional information is used to represent the emotion of the driver in a second time period, and the second time period is a time period after the first time period.

[0060] In some embodiments, the DMS in the car machine system and the vehicle-mounted fragrance control system perform data transmission through CAN (Controller Area Network) bus technology, the DMS transmits the predicted second emotional information data to the vehicle-mounted fragrance system, and the vehicle-mounted fragrance system further controls the smell through the received second emotional information data, thereby realizing the linkage of the two systems and realizing the vehicle-mounted fragrance control method of the embodiments of the application.

[0061] In order to ensure the real-time performance of the vehicle-mounted fragrance system for smell control, the DMS linked with the vehicle-mounted fragrance system periodically collects the facial image of the driver, and predicts the emotional information of the next time period after the first time period after collecting the emotional information of the first time period, thereby specifically controlling the emission of the smell.

[0062] The second emotional information includes the emotion of the driver in the time period after the first time period, and the process of obtaining the first emotional information is the same, for example, collecting the facial image of the driver in the time period after the first time period, and obtaining the second emotional information of the driver through face feature extraction and other operations on the facial image of the driver.

[0063] 203. The vehicle-mounted fragrance system in the car machine system obtains a target smell type in a matching relationship table of smell type and emotional information type based on the second emotional information, and the target smell type matches the type of the second emotional information.

[0064] In some embodiments, the odor type and emotion information type matching relationship table includes matching relationships between multiple odor types and multiple emotion information types, wherein the smells of different odor types can affect the emotions of different emotion information types. For example, when a driver is depressed, the smell of fresh apples or reeds can affect the driver's mood and make him feel good.

[0065] Among them, the matching relationship table between the smell type and the emotion information type is customized by the user. The user can register a user ID in the car fragrance system. Different user IDs correspond to different matching relationship tables between the smell type and the emotion information type, and the matching relationship table between the smell type and the emotion information type is stored with the user ID. Among them, the user ID corresponds to the user's facial image. The user enters his or her facial image in advance, and the car system associates the facial image with the user's user ID. After the car fragrance system is started, facial recognition is performed through the DMS linked to the car fragrance system to identify the user's facial image to determine the user's user ID, and then the matching relationship table between the smell type and the emotion information type corresponding to the user is obtained based on the user ID.

[0066] 204. The vehicle-mounted fragrance system in the vehicle system outputs a scent switching prompt, where the scent switching prompt is used to prompt a switch from a current scent type to a target scent type.

[0067] In some embodiments, the in-vehicle fragrance system interacts with the user in the form of an interface. After the in-vehicle fragrance system obtains the target odor type, it outputs an odor switching prompt on the interface, wherein the odor switching prompt may include the odor type of the odor, the odor concentration of the odor, and the odor emission rate of the odor. The odor type here is the target odor type. The user can confirm the odor type, odor concentration, and odor emission rate of the odor to be emitted by selecting on the interface.

[0068] For example, the interface displays "Do you want to change the odor type to reed odor?". After the user clicks the "Confirm" button, multiple buttons with different functions appear, including the "Adjust odor concentration" button and the "Adjust odor emission rate" button. The user can click the "Adjust odor concentration" button, and then slide the slider displayed in the interface to adjust the odor concentration, and click the "Save" button to save the odor concentration at the current concentration value position of the slider, or click the "Adjust odor emission rate" button, slide the above slider to adjust the odor emission rate, and click the "Save" button to save the odor concentration at the current concentration value position of the slider.

[0069] 205 . The vehicle-mounted fragrance system in the vehicle system switches the current scent type to the target scent type in response to the confirmation switching operation on the target scent type.

[0070] In some embodiments, a control instruction is generated based on the user's confirmation switching operation, which can enable the CLM in the vehicle fragrance system to control the switching of the scent type, so that the vehicle fragrance system emits the scent of the target scent type.

[0071] In some embodiments, the vehicle fragrance control method is implemented through a variety of hardware linkages, referring to Figure 3 , Figure 3 This is a hardware topology diagram involved in a vehicle-mounted fragrance control method provided according to an embodiment of the present application. In the figure, the vehicle computer is mainly used to provide a system interface to realize the interaction between the user and the system. The vehicle computer exchanges data with the DMS and CLM through CAN bus technology. The DMS and CLM also exchange data through CAN. The DMS is mainly used to collect and predict the driver's facial image in the driving state. The CLM is mainly used to control the switching of the smell type. Among them, the CLM and the fragrance controller exchange data through LIN (Local Interconnect Network). The fragrance controller is mainly used to emit odors.

[0072] The method provided in the embodiment of the present application can obtain the driver's first emotion by collecting the driver's facial image, and then predict the driver's second emotion based on the first emotion to prepare for the emission of the corresponding smell, and then match the target smell type corresponding to the second emotion in the matching relationship table of smell type and emotion information type. The smell of the target smell type can specifically affect the driver's second emotion, and then output an smell switching prompt to prompt the driver to switch the smell type. Then, in response to the confirmation switching operation of the target smell type, the current smell type is switched to the target smell type, thereby affecting the driver's emotions, improving the intelligence level of the in-vehicle fragrance system, and thus improving the driver's fragrance experience.

[0073] Figure 4 This is a flow chart of a vehicle-mounted fragrance control method provided in an embodiment of the present application. Figure 4 As shown, in the embodiment of the present application, the vehicle system is used as an example for explanation. The method includes the following steps:

[0074] 401. The DMS in the vehicle system collects a facial image of the driver, and obtains at least one facial expression feature of the driver within a first time period based on the facial image of the driver.

[0075] In some embodiments, when the DMS extracts facial features from the driver's facial image, it extracts the driver's facial features, such as the relative position of the driver's mouth in the entire face and the curvature of the driver's eye corners. These multiple facial features can represent the driver's facial expression characteristics.

[0076] There are multiple types of facial expression features, and different types of facial expression features can represent different emotional information of the driver.

[0077] 402. The DMS in the vehicle system obtains at least one first emotion information of the driver from a matching table of relationships between expression features and emotion information based on the at least one expression feature.

[0078] In some embodiments, the relationship matching table between facial expression features and emotional information stores a large amount of relationship data between facial expression features and emotional information, wherein the relationship between facial expression features and emotional information includes one-to-one, many-to-one and many-to-many. For example, a one-to-one relationship means that one facial expression feature can correspond to one emotional information.

[0079] In some embodiments, the DMS periodically obtains the driver's facial expression features within the first time period, and thus can obtain the facial expression features of each sub-time period within the first time period. For each facial expression feature, whenever the facial expression feature within the sub-time period is obtained, step 402 is executed based on the facial expression feature to obtain the first emotion information of each sub-time period within the first time period.

[0080] 403. The DMS in the vehicle system predicts second emotion information based on the at least one first emotion information, where the second emotion information is used to represent the driver's emotion in a second time period, where the second time period is a time period after the first time period.

[0081] In some embodiments, the prediction process is implemented in a DMS, wherein the prediction process includes the following steps 403A and 403B:

[0082] 403A: The DMS determines the number of occurrences of the first emotion information in each sub-time period within the first time period, and determines the duration ratio of the first emotion information in the first time period based on the first emotion information and the number of sub-time periods within the first time period.

[0083] In some embodiments, the number of occurrences of the first emotion information can reflect the user's emotions in the first time period. Based on the statistics of the number of occurrences, it can be determined which emotion is the main emotion in the first time period, and the main emotion may continue to the next time period. Therefore, the first emotion information determined based on the number of occurrences can more accurately predict the second emotion information to improve the accuracy of fragrance control.

[0084] For example, emotions are extracted 10 times in the first time period. Accordingly, the first emotion information indicating fatigue appears 8 times in the first time period, and the duration of the first emotion information in the first time period accounts for 80%.

[0085] 403B: The DMS determines the first emotion information whose duration ratio is higher than the target duration ratio as the second emotion information.

[0086] In some embodiments, after determining the duration ratios of multiple first emotion information within a first time period, the multiple duration ratios are compared with the target duration ratio, and the first emotion information whose duration ratio is higher than the target duration ratio is determined as the second emotion information, wherein the target duration ratio can be 70% or other values, and the embodiments of the present application do not limit this.

[0087] 404. The vehicle-mounted fragrance system in the vehicle system obtains a target odor type from a matching relationship table of odor types and emotion information types based on the second emotion information, where the target odor type matches the type of the second emotion information.

[0088] In some embodiments, the matching relationship table between the odor type and the emotion information type is generated during the system initialization phase of the vehicle fragrance system. The generation process can be referred to Figure 5 , Figure 5 This is a flow chart of a method for controlling in-vehicle fragrances according to an embodiment of the present application. As shown in the figure, the method first obtains the emotion information types supported by the DMS, then obtains the scent types available in the in-vehicle fragrance system through the CLM, and then enters the fragrance selection interface. Based on this fragrance selection interface, the user matches the scent type and the emotion information type, that is, creates a matching relationship table between the scent type and the emotion information type in the in-vehicle fragrance system. The in-vehicle fragrance system then stores this matching relationship table between the scent type and the emotion information type. The process of generating this matching relationship table between the scent type and the emotion information type includes the following steps 404A-404D:

[0089] 404A: The vehicle computer system obtains multiple emotion information types from the emotion type library, where the emotion type library includes multiple different emotion information types.

[0090] In some embodiments, the emotion type library is formed based on a large amount of emotion information data and stored in the DMS. The emotion type library includes emotion information types such as fatigue, happiness, and sadness. The in-vehicle fragrance system obtains a variety of different emotion information types in the emotion type library of the DMS based on CAN, thereby preparing for the vehicle system to display the various different emotion information types.

[0091] 404B: The vehicle fragrance system obtains multiple scent types of the multiple scent diffusion devices based on radio frequency signals emitted by the multiple scent diffusion devices in the vehicle, where the scent diffusion devices are used to emit scent of one scent type.

[0092] The scent device is controlled by the fragrance controller. In the vehicle-mounted fragrance system, there are multiple different scent devices, that is, multiple different smell types can be emitted. The vehicle-mounted fragrance system converts the radio frequency signal emitted by the scent device into a smell type by receiving the radio frequency signal, thereby obtaining multiple smell types included in the vehicle-mounted fragrance system and storing the data of the multiple smell types in the CLM, in preparation for displaying the multiple smell types.

[0093] In some embodiments, based on the radio frequency signal of the scent device received by the vehicle-mounted fragrance system, a prompt function can also be realized. When the fragrance concentration in the scent device is lower than a fixed value, it means that the fragrance in the scent device is insufficient to emit the smell, and the vehicle system sends a prompt in the form of an interface to replace the scent device, thereby ensuring the emission of the smell in the vehicle-mounted fragrance system at any time.

[0094] 404C: The vehicle system displays the multiple emotion information types and the multiple smell types in the form of an interface.

[0095] In some embodiments, the vehicle-mounted fragrance system obtains multiple emotion information types from the DMS based on CAN, obtains multiple smell types from the CLM, and displays the multiple emotion information types and the multiple smell types in the form of an interface for the user to create a matching relationship table of the smell types and the emotion information types.

[0096] The matching operation is self-defined and can be the same or different for different users. For example, user A matches the reed smell type with the sad emotion information type, but user B matches the reed smell type with the tired emotion information type.

[0097] 404D: The vehicle-mounted fragrance system generates the matching relationship table of the smell types and the emotion information types in response to the matching operation of the driver on the multiple emotion information types and the multiple smell types.

[0098] In some embodiments, the vehicle-mounted fragrance system generates the matching relationship table of the smell types and the emotion information types in the form of a data table. As in step 404C, the matching relationship table of the smell types and the emotion information types of different users can be different. In order to facilitate distinction, the multiple matching relationship tables of the smell types and the emotion information types are stored in correspondence with the user identifiers of the users.

[0099] 405, The vehicle-mounted fragrance system in the vehicle system outputs a smell switching prompt, which is used to prompt switching the current smell type to a target smell type.

[0100] The output smell switching prompt step is the same as step 204.

[0101] In some embodiments, before the output smell switching prompt, the air conditioner controller in the vehicle fragrance system generates an emotion control strategy based on environmental factors in the vehicle and system requirements, the emotion control strategy including smell emission strategies set for different emotion information types. In some embodiments, the emotion control strategy includes emitting different smell types of different styles at different time periods, or emitting the same smell type at different concentrations at different time periods. For example, the emotion control strategy is to emit different smell types at different time periods, based on the DMS system confirming that the second emotion information of the driver is joy, and the air conditioner controller obtaining that the temperature in the vehicle is higher than the preset temperature, the generated emotion control strategy is to emit the smell type of citrus corresponding to joy in the first half of the second time period, and emit the smell type of mint for high temperature in the vehicle in the second half, the content of the smell switching prompt corresponding to the emotion control strategy is still to switch the current smell type to the target smell type; or the emotion control strategy is to emit the same smell type at different concentrations at different time periods, based on the DMS system confirming that the second emotion information of the driver is joy, and the air conditioner controller obtaining that the temperature in the vehicle is the same as the preset temperature, the generated emotion control strategy is to emit the citrus smell type at a concentration of 50% in the first half of the second time period, and increase the concentration of the citrus smell type at a frequency of 10% per minute in the second half.

[0102] In yet some embodiments, the emotion control strategy can also include emitting different smell types of the same style at different time periods. For example, based on the DMS system confirming that the second emotion information of the driver is joy, and the air conditioner controller obtaining that the temperature in the vehicle is the same as the preset temperature, the generated emotion control strategy is to emit the smell type of citrus in the first half of the second time period, and emit the smell type of grapefruit in the second half. The smell type of citrus and the smell type of grapefruit are the same style corresponding to the smell type of emotion information of joy. The emotion control strategy enables the vehicle fragrance system to control the emission of the smell in a manner that is adjusted in real time based on the environment in the vehicle and the emotion information of the driver, making the emission of the smell in the vehicle more flexible, thereby further improving the intelligent degree of the vehicle fragrance system for the emission of the smell in the vehicle.

[0103] 406. The vehicle fragrance system in the vehicle infotainment system switches the current smell type to the target smell type in response to the confirmation switching operation of the target smell type.

[0104] In some embodiments, the target type of scent needs to be based on the fragrance controller to control the corresponding scent device to emit, therefore, after the vehicle-mounted fragrance system receives the confirmation switching operation of the target scent type, a corresponding fragrance control instruction is generated, which can instruct the fragrance controller to control the scent device to switch the current scent type to the target scent type, wherein the step of switching the current scent type to the target scent type includes steps 406A-406C:

[0105] 406A: Locate the target type of scent device, which is the type of scent device that can emit the target scent type of scent.

[0106] Wherein, based on the fragrance control instruction, the identification information of the scent device emitting the target scent type can be obtained, and the fragrance controller locates the scent device emitting the target scent type in the vehicle-mounted fragrance system based on the identification information, thereby further controlling the target type of scent device to emit scent.

[0107] 406B: If the scent type in the vehicle is not the target scent type within the first time period, control the currently working scent device to stop emitting the scent, and control the scent device to emit the target scent type.

[0108] Wherein, before controlling the scent device to emit scent, the CLM in the vehicle-mounted fragrance system first samples the air in the vehicle to detect whether there is scent emission in the vehicle, and the fragrance controller in the vehicle-mounted fragrance system also detects the type of scent device currently in working state.

[0109] In some embodiments, if the CLM detects that there is scent emission in the vehicle and the fragrance controller detects that the type of scent device in working state within the first time period is not the target scent type, the fragrance controller controls to turn off the scent device emitting the current scent type of scent, and then the fragrance controller locates the target type of scent device based on the fragrance control instruction and controls the target type of scent device to start working, thereby emitting the target scent type of scent.

[0110] 406C: If no scent emission is detected within the first time period, control the target type of scent device to emit the target scent type of scent.

[0111] Wherein, if the CLM detects that there is no scent emission in the vehicle within the first time, the CLM transmits the detection result to the fragrance controller, and the fragrance controller locates the target type of scent device based on the fragrance control instruction and controls the target type of scent device to emit the target scent type of scent.

[0112] In some embodiments, the vehicle fragrance control method is implemented in the vehicle machine system, and the overall process of the method can be seen from Figure 6 , Figure 6 is a flowchart of a vehicle fragrance control method in an embodiment of the present application. As shown in the figure, first, the face image of the driver in the first time period is collected based on the DMS in the vehicle machine system, the emotion of the driver in the first time period is obtained, and then the emotion of the driver in the next time period is predicted based on the DMS. The DMS transmits the predicted emotion of the driver in the next time period to the vehicle fragrance system, obtains the target odor type based on the matching relationship table of the odor type and the emotion information type in the vehicle fragrance system, and then the fragrance controller in the vehicle fragrance system controls the switching of the odor type in the vehicle to the target odor type.

[0113] The method provided in the embodiments of the present application can obtain the first emotion of the driver by collecting the face image of the driver, predict the second emotion of the driver based on the first emotion, prepare to emit the corresponding odor, match the target odor type corresponding to the second emotion in the matching relationship table of the odor type and the emotion information type, the odor of the target odor type can specifically affect the second emotion of the driver, then output the odor switching prompt to prompt the driver to switch the odor type, and in response to the confirmation switching operation of the target odor type, switch the current odor type to the target odor type, thereby affecting the emotion of the driver, improving the intelligent degree of the vehicle fragrance system, and further improving the fragrance experience of the driver. Further, the method also provides an emotion control strategy, so that the vehicle fragrance system controls the odor emission in a more flexible manner.

[0114] Figure 7 is a block diagram of a vehicle fragrance control device according to an embodiment of the present application. The device is used to execute the steps of the vehicle fragrance control method described above. Referring to Figure 7 , the device includes:

[0115] The emotion information acquisition module 701 is configured to collect the face image of the driver, and obtain at least one first emotion information of the driver based on the face image, the at least one first emotion information being used to represent the emotion of the driver in the first time period.

[0116] The prediction module 702 is configured to predict the second emotion information based on the at least one first emotion information, the second emotion information being used to represent the emotion of the driver in the second time period, the second time period being a time period after the first time period.

[0117] The target odor type acquisition module 703 is configured to obtain the target odor type in the matching relationship table of the odor type and the emotion information type based on the second emotion information, the target odor type matching the type of the second emotion information.

[0118] A prompt module 704 is configured to output an odor switching prompt, wherein the odor switching prompt is configured to prompt the user to switch the current odor type to the target odor type.

[0119] The switching module 705 is configured to switch the current odor type to the target odor type in response to a confirmation switching operation on the target odor type.

[0120] In some embodiments, the emotion information acquisition module 701 is used to:

[0121] Acquiring facial features of the driver during a first time period based on the driver's facial image;

[0122] Based on the facial expression features, the first emotional information of the driver is obtained from a relationship matching table between facial expression features and emotional information.

[0123] In some embodiments, the prediction module 702 is configured to:

[0124] Determine the number of occurrences of the first emotion information in each sub-time period within the first time period, and determine the duration ratio of the first emotion information in the first time period based on the first emotion information and the number of sub-time periods within the first time period;

[0125] The first emotion information whose duration ratio is higher than the target duration ratio is determined as the second emotion information.

[0126] In some embodiments, the apparatus further comprises:

[0127] An emotion information type acquisition module is used to acquire multiple emotion information types from an emotion type library, where the emotion type library includes multiple different emotion information types;

[0128] an odor type acquisition module, configured to acquire multiple odor types possessed by multiple fragrance diffusion devices in a vehicle based on radio frequency signals emitted by the multiple fragrance diffusion devices, wherein the fragrance diffusion devices are configured to emit odors of a single odor type;

[0129] A display module, used for displaying multiple types of emotion information and multiple types of smells in the form of an interface;

[0130] The generating module is configured to generate a matching relationship table between the odor types and the emotion information types in response to a driver's matching operation on the multiple emotion information types and the multiple odor types.

[0131] In some embodiments, the switching module 705 is configured to:

[0132] Locating a target type of fragrance diffusion device, where the target type is a type of fragrance diffusion device capable of emitting a target odor type;

[0133] If the odor type in the vehicle is not the target odor type within the first time period, controlling the currently working fragrance diffusion device to stop emitting the odor and controlling the fragrance diffusion device to emit the target odor type;

[0134] If no odor emission is detected within the first time period, the target type of fragrance diffusion device is controlled to emit the target odor type of odor.

[0135] It should be noted that the above-described embodiments of the vehicle fragrance control device, when used to control the release of odors within a vehicle, illustrate the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, i.e., the internal structure of the device can be divided into different functional modules to perform all or part of the functions described above. Furthermore, the vehicle fragrance control device and the vehicle fragrance control method embodiments provided in the above-described embodiments share the same concept. The specific implementation process is detailed in the method embodiments and will not be further elaborated here.

[0136] In the embodiments of the present application, the computer device can be configured as a terminal or a server. When the computer device is configured as a terminal, the terminal can be used as the execution subject to implement the technical solution provided in the embodiments of the present application. When the computer device is configured as a server, the server can be used as the execution subject to implement the technical solution provided in the embodiments of the present application. The technical solution provided in the present application can also be implemented through interaction between the terminal and the server. The embodiments of the present application do not limit this.

[0137] Figure 8 The following is a block diagram of a computer device 800 according to an embodiment of the present application. The computer device 800 may be a portable mobile terminal, such as a smartphone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The computer device 800 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other similar names.

[0138] Typically, the computer device 800 includes a processor 801 and a memory 802 .

[0139] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0140] Memory 802 may include one or more computer-readable storage media, which may be non-transitory. Memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 802 is used to store at least one computer program, which is executed by processor 801 to implement the in-vehicle fragrance control method provided in the method embodiments of this application.

[0141] In some embodiments, computer device 800 may optionally include a peripheral device interface 803 and at least one peripheral device. Processor 801, memory 802, and peripheral device interface 803 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 803 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, and a power supply 808.

[0142] The peripheral interface 803 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802 and the peripheral interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802 and the peripheral interface 803 can be implemented on a separate chip or circuit board, and the present embodiments are not limited in this regard.

[0143] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. In some embodiments, the radio frequency circuit 804 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 can also include NFC (Near Field Communication) related circuitry, and the present application is not limited in this regard.

[0144] The display screen 805 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 is further configured to capture touch signals on or above the surface of the display screen 805. The touch signals can be input to the processor 801 as control signals for processing. In this case, the display screen 805 can also be configured to provide virtual buttons and / or virtual keyboard, also known as soft buttons and / or soft keyboard. In some embodiments, the display screen 805 can be one, disposed on the front panel of the computer device 800; in other embodiments, the display screen 805 can be at least two, respectively disposed on different surfaces of the computer device 800 or in a folding design; in other embodiments, the display screen 805 can be a flexible display screen, disposed on a curved surface or a folding surface of the computer device 800. Even, the display screen 805 can also be disposed in an irregular shape, i.e., a special-shaped screen. The display screen 805 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0145] The camera assembly 806 is configured to capture images or videos. In some embodiments, the camera assembly 806 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, to realize the background blur function by fusing the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function by fusing the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera assembly 806 can further include a flash. The flash can be a single-color-temperature flash or a dual-color-temperature flash. The dual-color-temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0146] The audio circuit 807 can include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor 801 for processing, or input to the radio frequency circuit 804 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, respectively arranged at different parts of the computer device 800. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker can be a conventional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can it convert electrical signals into sound waves that humans can hear, but it can also convert electrical signals into sound waves that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 807 can also include a headphone jack.

[0147] The power supply 808 is used to supply power to various components in the computer device 800. The power supply 808 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 808 includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery charged through a wired line, and the wireless charging battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0148] In some embodiments, the computer device 800 further includes one or more sensors 809. The one or more sensors 809 include, but are not limited to, an acceleration sensor 810, a gyroscope sensor 811, a pressure sensor 812, an optical sensor 813, and a proximity sensor 814.

[0149] The acceleration sensor 810 can detect the acceleration in three coordinate axes of the coordinate system established by the computer device 800. For example, the acceleration sensor 810 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 810. The acceleration sensor 810 can also be used for game or user motion data collection.

[0150] The gyroscope sensor 811 can detect the body direction and rotation angle of the computer device 800, and the gyroscope sensor 811 can cooperate with the acceleration sensor 810 to collect 3D actions of the user on the computer device 800. The processor 801 can realize the following functions according to the data collected by the gyroscope sensor 811: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0151] The pressure sensor 812 can be arranged at the side frame of the computer device 800 and / or the lower layer of the display screen 805. When the pressure sensor 812 is arranged at the side frame of the computer device 800, the holding signal of the user to the computer device 800 can be detected, and the left-hand or right-hand recognition or shortcut operation can be performed by the processor 801 according to the holding signal collected by the pressure sensor 812. When the pressure sensor 812 is arranged at the lower layer of the display screen 805, the operability control on the UI interface can be controlled by the processor 801 according to the pressure operation of the user to the display screen 805. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0152] The optical sensor 813 is used to collect the ambient light intensity. In an embodiment, the processor 801 can control the display brightness of the display screen 805 according to the ambient light intensity collected by the optical sensor 813. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is decreased. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 according to the ambient light intensity collected by the optical sensor 813.

[0153] The proximity sensor 814, also referred to as a distance sensor, is usually arranged at the front panel of the computer device 800. The proximity sensor 814 is used to collect the distance between the user and the front of the computer device 800. In an embodiment, when the proximity sensor 814 detects that the distance between the user and the front of the computer device 800 gradually decreases, the display screen 805 is switched from the bright screen state to the off-screen state by the processor 801; when the proximity sensor 814 detects that the distance between the user and the front of the computer device 800 gradually increases, the display screen 805 is switched from the off-screen state to the bright screen state by the processor 801.

[0154] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the computer device 800, and the computer device 800 can include more or fewer components than those shown in the figure, or combine certain components, or adopt a different component arrangement. Figure 8 Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the computer device 800, and the computer device 800 can include more or fewer components than those shown in the figure, or combine certain components, or adopt a different component arrangement.

[0155] Figure 9A structural schematic diagram of a server is provided according to an embodiment of the present application. The server 900 can have great differences due to different configurations or performances, and can include one or more processors (Central Processing Units, CPUs) 901 and one or more memories 902. The memory 902 stores at least one computer program, which is loaded and executed by the processor 901 to implement the vehicle-mounted fragrance control method provided by each method embodiment. Of course, the server can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for realizing device functions, and the like, so as to perform input and output. The server can also include other components for realizing device functions, which are not described herein.

[0156] The present application also provides a computer readable storage medium, which stores at least one computer program. The at least one computer program is loaded and executed by a processor of an in-vehicle infotainment system to implement the operations performed by the in-vehicle infotainment system in the vehicle-mounted fragrance control method of the above embodiments. For example, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0157] The present application also provides a computer program product or a computer program, which includes computer program code stored in a computer readable storage medium. The computer program code is read by a processor of an in-vehicle infotainment system from the computer readable storage medium. The processor executes the computer program code, so that the in-vehicle infotainment system performs the vehicle-mounted fragrance control method provided in the various optional implementation manners.

[0158] Those of ordinary skill in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by a program instructing relevant hardware, and the program can be stored in a computer readable storage medium, such as a Read-Only Memory, a magnetic disk or an optical disk.

[0159] The above is only an optional embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle-mounted fragrance control method, characterized in that: The method comprises: Acquiring a facial image of a driver, and obtaining at least one first piece of emotion information of the driver based on the facial image, wherein the at least one first piece of emotion information is used to represent the driver's emotion within a first time period; Determine the number of occurrences of the first emotion information in each sub-time period within the first time period, and determine the duration ratio of the first emotion information in the first time period based on the first emotion information and the number of sub-time periods within the first time period; Determining the first emotion information in which the duration ratio is higher than the target duration ratio as second emotion information, where the second emotion information is used to represent the driver's emotion in a second time period, where the second time period is a time period after the first time period; Based on the second emotion information, obtaining a target odor type from a matching relationship table of odor types and emotion information types, wherein the target odor type matches the type of the second emotion information; Outputting an odor switching prompt, wherein the odor switching prompt is used to prompt a switch from a current odor type to the target odor type; In response to a confirmation switching operation on the target odor type, the current odor type is switched to the target odor type.

2. The method according to claim 1, characterized in that The collecting of the driver's facial image and obtaining at least one first emotion information of the driver based on the facial image includes: Acquiring facial features of the driver during the first time period based on the driver's facial image; Based on the facial expression feature, the first emotional information of the driver is obtained from a relationship matching table between facial expression features and emotional information.

3. The method according to claim 1, characterized in that The method further comprises: Acquire multiple emotion information types from an emotion type library, wherein the emotion type library includes multiple different emotion information types; Acquiring, based on radio frequency signals emitted by a plurality of fragrance diffusion devices in a vehicle, a plurality of scent types possessed by the plurality of fragrance diffusion devices, wherein the fragrance diffusion devices are configured to emit scent of a single scent type; Displaying the multiple emotion information types and the multiple smell types in the form of an interface; In response to the driver's matching operation on the multiple emotion information types and the multiple smell types, a matching relationship table between the smell types and the emotion information types is generated.

4. The method according to claim 1, wherein The step of switching the current odor type to the target odor type in response to the confirmation switching operation on the target odor type includes: Locating a target type of fragrance diffusion device, the target type being a type of fragrance diffusion device capable of emitting the target odor type; If the odor type in the vehicle is not the target odor type during the first time period, controlling the currently working fragrance diffusion device to stop emitting the odor and controlling the fragrance diffusion device to emit the target odor type; If no odor emission is detected within the first time period, the fragrance diffusion device of the target type is controlled to emit the odor of the target odor type.

5. A vehicle-mounted fragrance control device, characterized in that: The device comprises: an emotion information acquisition module, configured to capture a facial image of a driver and acquire at least one first piece of emotion information of the driver based on the facial image, wherein the at least one first piece of emotion information is used to represent the driver's emotion within a first time period; a prediction module, configured to determine a number of occurrences of first emotion information within each sub-time period within the first time period, and determine a duration ratio of the first emotion information within the first time period based on the first emotion information and the number of sub-time periods within the first time period; and determine the first emotion information having a duration ratio greater than a target duration ratio as second emotion information, where the second emotion information is used to represent the driver's emotion within a second time period, the second time period being a time period after the first time period; a target odor type acquisition module, configured to acquire, based on the second emotion information, a target odor type from a matching relationship table of odor types and emotion information types, wherein the target odor type matches the type of the second emotion information; a prompt module, configured to output an odor switching prompt, wherein the odor switching prompt is used to prompt a user to switch the current odor type to the target odor type; A switching module is configured to switch the current odor type to the target odor type in response to a confirmation switching operation on the target odor type.

6. The device according to claim 5, characterized in that The emotion information acquisition module is used to: Acquiring facial features of the driver during the first time period based on the driver's facial image; Based on the facial expression feature, the first emotional information of the driver is obtained from a relationship matching table between facial expression features and emotional information.

7. A vehicle computer system, characterized in that: The vehicle system includes a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the vehicle fragrance control method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store at least one computer program, and the at least one computer program is used to execute the vehicle-mounted fragrance control method according to any one of claims 1 to 4.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle-mounted fragrance control method according to any one of claims 1 to 4 is implemented.

Images