Cabin atmosphere theme generation method, device, equipment, medium and product

Abstract

The embodiment of the invention provides a cabin atmosphere theme generation method and device, equipment, a medium and a product. The method comprises the following steps: acquiring a cabin scene image related to a cabin; according to the cockpit scene image, generating cockpit scene features which describe the cockpit by adopting a natural language; and generating a cabin atmosphere theme of the cabin according to the cabin scene features and a theme generation model. By using the method, a cabin scene image related to a cabin is obtained, perception and understanding of a cabin scene are realized by processing the cabin scene image, and cabin scene features described by natural languages are generated; and then cabin scene features are processed based on a theme generation model, and a cabin atmosphere theme is generated in a generative mode. According to the method, the current overall driving scene of the vehicle can be automatically identified and understood, so that the suitable cabin atmosphere theme is dynamically generated, the method does not depend on a fixed preset atmosphere theme, and the adaptability, the individuation degree and the flexibility of cabin atmosphere theme generation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart cockpits, and in particular to a method, device, equipment, medium and product for generating a cockpit atmosphere theme. Background Art

[0002] With the continuous development of technologies such as electrification and autonomous driving, the interior of future cars will no longer be just a simple cockpit, but a smart cockpit that integrates infotainment, office, and leisure. The smart cockpit provides car owners with a personalized experience and creates a perfect atmosphere. Drivers and passengers can personalize the interior of the cockpit according to their preferences, such as adjusting the seat angle, lighting atmosphere, music playback, etc. However, the existing cockpit atmosphere adjustment often requires manual settings and adjustments by the driver and passengers, and the adjustment strategy is preset and relatively simple, which cannot adapt to new scenarios, limiting the adaptability and personalization of the cockpit atmosphere theme and lacking flexibility. Summary of the invention

[0003] The embodiments of the present invention provide a method, device, equipment, medium and product for generating a cabin atmosphere theme, which can automatically identify and understand the current overall driving scene of the vehicle, thereby dynamically generating a suitable cabin atmosphere theme, thereby improving the adaptability, personalization and flexibility of the cabin atmosphere theme generation.

[0004] In a first aspect, this embodiment provides a method for generating a cabin atmosphere theme, the method comprising:

[0005] Acquire cockpit scene images related to the cockpit;

[0006] generating, according to the cockpit scene image, a cockpit scene feature that describes the cockpit in natural language;

[0007] A cabin atmosphere theme of the cabin is generated according to the cabin scene features and a theme generation model, wherein the theme generation model is a model for generating a cabin atmosphere theme.

[0008] In a second aspect, this embodiment provides a device for generating a cabin atmosphere theme, the device comprising:

[0009] An image acquisition module, used to acquire cockpit scene images related to the cockpit;

[0010] A feature description module, used to generate a cockpit scene feature describing the cockpit in natural language according to the cockpit scene image;

[0011] A theme generation module is used to generate a cabin atmosphere theme of the cabin according to the cabin scene characteristics and a theme generation model, wherein the theme generation model is a model for generating a cabin atmosphere theme.

[0012] In a third aspect, this embodiment provides an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the cabin atmosphere theme generating method described in any embodiment of the present invention.

[0016] In a fourth aspect, this embodiment provides a computer-readable storage medium, and the computer program is executed by the at least one processor to enable the at least one processor to execute the cabin atmosphere theme generating method described in any embodiment of the present invention.

[0017] In a fifth aspect, an embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for generating a cabin atmosphere theme as described in any embodiment of the present invention.

[0018] The embodiment of the present invention provides a method, device, equipment, medium and product for generating a cabin atmosphere theme, the method comprising: obtaining a cabin scene image related to the cabin; generating a cabin scene feature that describes the cabin in natural language according to the cabin scene image; generating a cabin atmosphere theme of the cabin according to the cabin scene feature and a theme generation model, wherein the theme generation model is a model for generating a cabin atmosphere theme. In the above technical solution, a cabin scene image related to the cabin is obtained, and the cabin scene is perceived and understood by processing the cabin scene image, and a cabin scene feature that describes the cabin in natural language is generated; then the cabin scene feature is processed based on the theme generation model, and a cabin atmosphere theme is generated in a generative manner. The system realizes the automatic recognition and understanding of the current overall driving scene of the vehicle, thereby dynamically generating a suitable cabin atmosphere theme, which is independent of a fixed preset atmosphere theme, and improves the adaptability, personalization and flexibility of the cabin atmosphere theme generation. By perceiving scene changes in real time, the cabin atmosphere can be quickly adjusted to ensure that the driver and passengers are always in a comfortable and safe environment, thereby improving the comfort and safety of the driving process.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a method for generating a cabin atmosphere theme provided in the first embodiment of the present invention;

[0022] Figure 2 An example diagram of generating cabin scene information in the execution of a cabin atmosphere theme generation method provided in the first embodiment of the present invention;

[0023] Figure 3 This is a flowchart of a method for generating a cabin atmosphere theme provided in the first embodiment of the present invention;

[0024] Figure 4 A schematic diagram of a flow chart of another method for generating a cabin atmosphere theme provided in the second embodiment of the present invention;

[0025] Figure 5 An example diagram of a process of generating cabin scene features in the execution of a method for generating a cabin atmosphere theme provided in the second embodiment of the present invention;

[0026] Figure 6 This is a flowchart of a method for adjusting a cabin atmosphere theme in a certain application scenario provided by the second embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a cabin atmosphere theme generating device provided in Embodiment 3 of the present invention;

[0028] Figure 8 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It is understandable that before using the technical solutions disclosed in the embodiments of the present invention, the type, scope of use, usage scenarios, etc. of the personal information involved in the present invention should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0032] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present invention according to the prompt message.

[0033] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0034] It is understandable that the above notification and user authorization process is only illustrative and does not constitute a limitation on the implementation of the present invention. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present invention.

[0035] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0036] It should be clear that the existing cockpit atmosphere adjustment has the following problems: 1) Passive adjustment: The cockpit atmosphere adjustment (such as sound, temperature, lighting, etc.) is usually passive and requires the driver to manually set and adjust. 2) Fixed strategy: The adjustment strategy is usually preset, unable to understand and trigger in real time according to emerging scenarios, and lacks flexibility. 3) Lack of intelligent scene perception model: The existing system cannot dynamically identify scene changes, which limits the adaptability and personalization of the cabin atmosphere theme. 4) Insufficient utilization of extracabin perception systems: The existing extracabin perception systems mainly serve intelligent driving, not intelligent cockpits, and therefore lack the comprehensive use of the external environment and cockpit scenarios. In summary, a method is needed to solve the above problems.

[0037] Embodiment 1

[0038] Figure 1 A flowchart of a method for generating a cabin atmosphere theme is provided in accordance with a first embodiment of the present invention. The method is applicable to the case where a cabin atmosphere theme is automatically generated. The method can be executed by a cabin atmosphere theme generating device. The cabin atmosphere theme generating device can be implemented in the form of hardware and / or software and is generally integrated in an electronic device.

[0039] like Figure 1 As shown, a method for generating a cabin atmosphere theme provided in the first embodiment may specifically include the following steps:

[0040] S101: Acquire a cockpit scene image related to the cockpit.

[0041] In this embodiment, the existing in-cabin and out-cabin perception modules in the vehicle are used to perceive and understand the overall driving scene, and generate corresponding cabin atmosphere themes to adjust the cabin atmosphere based on the cabin atmosphere theme. This step is used to obtain an image of the overall driving scene related to the cabin, which is recorded as a cabin scene image. The cabin scene image may include an in-cabin situation image and an out-cabin environment image. Specifically, in combination with the current image acquisition equipment inside and outside the vehicle, multimodal data such as external environment images and scene images inside the cabin are obtained. For example, the external environment image can be captured based on sensors such as the vehicle's external camera and laser radar. The scene image inside the cabin can be captured based on sensors such as the vehicle's internal camera. Among them, the information contained in the external environment image includes but is not limited to: external road conditions, external weather, external landscape types (such as urban landscapes, natural landscapes, etc.), external lighting, etc.; the information contained in the scene image inside the cabin includes but is not limited to: the number of passengers, the passenger's seating position, the passenger's emotional state, etc. It is clear that the out-of-cabin sensing module originally used for intelligent driving can be extended to the cabin atmosphere adjustment without adding additional equipment. Through data sharing and analysis, out-of-cabin information can serve the cabin and enhance the driving experience. Effective integration of in-cabin and out-of-cabin data resources makes the cabin atmosphere adjustment not only based on the in-car status, but also linked with the external environment in real time to optimize the cabin experience.

[0042] S102: Generate a cockpit scene feature that describes the cockpit in natural language according to the cockpit scene image.

[0043] In this embodiment, after obtaining the cockpit-related cockpit scene image as described above, the scene understanding stage is entered. First, the cockpit scene image needs to be understood, and image features of the cockpit scene image need to be extracted. Then, a natural language description of the overall scene information is generated through natural language processing (Natural Language Processing, NLP), and the image features are converted into information that describes the entire driving scene in natural language, including the scene inside the cabin and the scene outside the cabin. The information that describes the entire driving scene in natural language is recorded as cockpit scene information. The cockpit scene information simultaneously integrates the situational features inside the cabin and the environmental features outside the cabin. By semantically recognizing the information that describes the entire driving scene in natural language, the cockpit scene features that describe the cockpit in natural language can be obtained.

[0044] For example, Figure 2 This is an example diagram of generating cabin scene information in the execution of a cabin atmosphere theme generation method provided in the first embodiment of the present invention, such as Figure 2 As shown, the outdoor environment image includes weather information, road conditions, and landscape types, and the in-cabin situation image includes the number of occupants, occupant distribution, and occupant emotional state. Based on the in-cabin situation image and the outdoor scene image, the overall cabin scene information described in natural language is generated.

[0045] Below are two examples of cockpit scene information that use natural language to describe the entire driving scenario.

[0046] Example 1: In a forest in the thick of autumn, a family car is driving on a winding path surrounded by colorful leaves. Outside the car, the sun shines through the branches and leaves, sprinkles on the road, and outlines a warm atmosphere, as if the whole forest is welcoming this journey. Inside the car, the family is laughing and having fun. Two children are sitting in the back seat, interacting with the adults, with happy smiles on their faces. The display screen in the front row plays various entertainment content, adding to the fun of the journey. Through the panoramic sunroof on the roof, everyone in the car can feel the beauty of nature, as if they are integrated with the autumn forest. The whole scene presents a warm and harmonious family travel atmosphere, which makes people immersed in the comfort of autumn and the warmth of family affection.

[0047] Example 2: On a rainy highway, a car is driving on a slippery road. The electronic sign in front of the lane displays a variety of warning symbols, reminding the driver that the road may be dangerous. The sky is gray and visibility is low. The vehicles on the road are turning on their lights, which are reflected on the wet ground, making driving more difficult. At the same time, the driver in the car looks nervous, holding the steering wheel tightly with both hands, and his eyes are anxious and uneasy. The driver is obviously facing a certain amount of pressure, either due to the poor weather and road conditions, or due to other factors during driving. The rain, low visibility and warning signs are intertwined, adding a sense of oppression and tension to the whole scene, reflecting the driver's high alert in bad weather.

[0048] S103: Generate a cockpit atmosphere theme of the cockpit according to cockpit scene characteristics and a theme generation model.

[0049] In this embodiment, the topic generation model can be specifically understood as a model for generating cabin atmosphere themes. The input of the topic generation model is the cabin scene features, and the output of the topic generation model is the cabin atmosphere theme. The topic generation model can be considered to be obtained by training a basic model based on a training sample set of cabin scene features-cabin atmosphere themes. For example, the basic model can use Transformer, which is a sequence model based on the attention mechanism. Using the topic generation model to process scene descriptions and generate cabin atmosphere themes that adapt to specific emotions and scenes can ensure the personalization and accuracy of lighting, music and fragrance configurations.

[0050] Specifically, the theme generation model generates a personalized cabin atmosphere theme based on different cabin scene characteristics such as occupant emotions, weather, road conditions, etc. In this embodiment, there is no specific restriction on which atmosphere parameters are included in the cabin atmosphere theme, and they can be set according to actual needs. For example, the atmosphere parameters may include light color, light brightness, music style, and fragrance type. For example, a "pleasant and relaxed" scene may generate a combination of "soft warm light + light music + fresh fragrance".

[0051] It is important to know that the topic generation model not only supports diversified topic generation, but also has adaptive learning capabilities, and can continuously optimize the topic generation effect through data accumulation and user feedback. It can also dynamically generate atmosphere themes based on different emotional keywords and scene features, providing richer and more flexible options than traditional preset atmospheres.

[0052] Below are two examples of cabin ambience themes.

[0053] Cabin atmosphere theme 1: Warm autumn family travel

[0054] Mood lighting: Warm orange and gold tones are used to simulate the light and shadow effects of autumn sunlight in the forest with a soft gradient effect, adding a sense of warmth. The lighting brightness is moderate to create a comfortable visual experience, especially in the panoramic sunroof, where soft light is selected to enhance the effect of natural light.

[0055] Music: Playing light music or family-friendly pop music can make the atmosphere in the car more harmonious and energetic, suitable for family interaction. The music selection can include natural sounds (such as rustling leaves and birdsong) as background to enhance the sense of connection with the autumn forest environment.

[0056] Fragrance: Use warm woody scents such as cedar or sandalwood, with a hint of fresh orange or cinnamon to evoke the scent of an autumn forest. Keep the concentration of the fragrance moderate to bring a fresh and comforting feeling, and avoid being too strong to interfere with the comfort of the car.

[0057] Cockpit atmosphere theme 2: safe and focused driving

[0058] Mood Lighting: Uses a cool, soft blue light to reduce driver anxiety and promote calmness. Reduces overall interior brightness to prevent distractions in low-visibility conditions and enhances focus. To increase sensitivity to road conditions, a subtle white light can be added to the instrument panel and front display area to provide clarity in key areas.

[0059] Music: Play soft background music with a low tempo or white noise from nature (such as rain or wind) to help relieve the driver's nervousness. Automatically lower the volume of the music so that the driver can hear the external environment more clearly and better perceive the surrounding vehicles and road conditions.

[0060] Fragrance: Choose a relaxing mint or pine scent to freshen the air and reduce driver stress. Avoid overly strong fragrances that can be distracting. The fragrance intensity can be adjusted to a low level for a gentle release.

[0061] It is understandable that by fusion of multimodal data, natural language description is generated, and atmosphere themes are dynamically generated with the help of theme generation model, the atmosphere themes can be automatically updated when the scene changes, so that the cockpit atmosphere can adapt to different driving situations in real time, such as sunny days, rainy days, high-speed driving, rural roads, etc., thereby enhancing the flexibility and adaptability of the system. Through intelligent scene perception, dangerous scenes can be identified and corresponding atmosphere settings, such as cold lights and low-tempo music, can be generated to assist drivers to concentrate and improve driving safety. It can automatically identify and understand the current overall driving scene of the vehicle, including the external environment and the in-car situation, and dynamically generate the most suitable atmosphere theme to meet personalized needs and improve user experience. And through intelligent scene perception, the changes in driving scenes can be captured in real time, and the atmosphere configuration can be dynamically adjusted to adapt to real-time changes in the driving process. For example, when switching from sunny days to rainy days, the system can automatically adjust to an atmosphere suitable for bad weather to enhance the driver's sense of security, improve the response speed and accuracy of the cockpit, and ensure that the driver and passengers can quickly obtain the most suitable cockpit atmosphere experience in any situation.

[0062] For example, Figure 3 This is a flowchart of a method for generating a cabin atmosphere theme according to the first embodiment of the present invention, as shown in FIG. Figure 3 As shown, one example is: Driving scenario: Family self-driving tour, current state: excited, happy, driving location: coastal highway; Scenario understanding: A family of four went on a self-driving tour in the suburbs in autumn, and they just set out at this moment, full of energy; Generate cockpit atmosphere theme: bright colors that match the outside picture, magnificent music. Another example is: Driving scenario: Commuting after get off work, current state: nervous, tired, focused driving, driving location: urban expressway; Scenario understanding: Commuting after get off work, it rained heavily on the road, visibility was low on the highway, and the driver was concentrating on driving; Generate cockpit atmosphere theme: cold lights make the driver more focused, and the volume in the front row is reduced.

[0063] An embodiment of the present invention provides a method for generating a cabin atmosphere theme, which obtains a cabin scene image related to the cabin, processes the cabin scene image to achieve perception and understanding of the cabin scene, and generates cabin scene features described in natural language; then the cabin scene features are processed based on a theme generation model, and a cabin atmosphere theme is generated in a generative manner. The method realizes the ability to automatically identify and understand the current overall driving scene of the vehicle, thereby dynamically generating a suitable cabin atmosphere theme without relying on a fixed preset atmosphere theme, thereby improving the adaptability, personalization, and flexibility of cabin atmosphere theme generation. By perceiving scene changes in real time, the cabin atmosphere can be quickly adjusted to ensure that the driver and passengers are always in a comfortable and safe environment, thereby improving the comfort and safety of the driving process.

[0064] As an optional embodiment of the embodiment of the present invention, on the basis of the above embodiment, the method can be optimized to, after generating the cabin atmosphere theme of the cabin, further include: matching the cabin atmosphere theme with the user's historical preference atmosphere information to obtain an updated cabin atmosphere theme; or receiving the user's adjustment operation on the cabin atmosphere theme to obtain an updated cabin atmosphere theme.

[0065] In this embodiment, after the cabin atmosphere theme is generated based on the theme generation model, the cabin atmosphere theme can be directly used as a recommended theme to control the cabin atmosphere. The recommended cabin atmosphere theme can also be combined with the user's preferences to provide a final configuration, which is recorded as an updated cabin atmosphere theme. Specifically, the cabin atmosphere theme output by the model is matched with the user's historical preferences to ensure that the matched cabin atmosphere theme meets the user's personalized needs. For example, if the user prefers a specific type of atmosphere element (for example, prefers jazz or likes wood fragrance), these preferences are matched first in the generated configuration.

[0066] Furthermore, after the cabin atmosphere theme is generated based on the theme generation model, the cabin atmosphere theme can also be fed back to the user, and a manual adjustment interface is provided to the user, allowing the user to adjust the cabin atmosphere theme according to personal needs based on the generated theme, such as modifying parameters such as lighting, music or fragrance in real time. The cabin atmosphere theme adjusted by the user is recorded as the updated cabin atmosphere theme.

[0067] For example, given the description "a sunny seaside highway, passengers are relaxed and happy": first, perform a cabin scene feature analysis: identify the "happy" emotion and the "sunny, seaside" scene; second, generate themes based on the cabin scene features: the theme generation model suggests "soft blue-green lighting + brisk background sound effects of waves + fresh citrus fragrance" to enhance the seaside feeling; then, match based on the user's historical preferences: if the user prefers a romantic atmosphere, soft music can be recommended, or the fragrance can be replaced with lavender; finally, receive dynamic adjustments from the user: if the user feels that the music is not cheerful enough, it can be adjusted to lighter music with a stronger sense of rhythm, and the feedback can be saved for optimizing the theme generation model next time.

[0068] The above technical solution adds the function of historical preference matching or user manual adjustment of the cabin atmosphere theme generated by the model, enhances the flexibility of intelligent generation and user control of the cabin atmosphere theme, and makes the cabin atmosphere theme more in line with the personalized needs of users.

[0069] Further, based on the above optional embodiment, after obtaining the updated cabin atmosphere theme, the method may further include: optimizing the theme generation model based on the updated cabin atmosphere theme.

[0070] In this embodiment, the theme generation model can also be continuously optimized according to the user's adjustment and feedback, so that the generated cabin atmosphere theme better meets the user's actual needs. Specifically, the updated cabin atmosphere theme is used as a training sample to train the theme generation model to obtain an optimized theme generation model.

[0071] The above technical solution adds the function of optimizing the theme generation model, improves the accuracy of the theme generation model, and makes the cabin atmosphere theme generated by the model more in line with the personalized needs of users.

[0072] As an optional embodiment of the embodiment of the present invention, on the basis of the above embodiment, it can be optimized that after generating the cabin atmosphere theme of the cabin, the method further includes: adjusting the atmosphere in the cabin based on the atmosphere parameters included in the cabin atmosphere theme.

[0073] In this embodiment, the specific atmosphere parameters included in the cabin atmosphere theme can be set according to actual conditions. For example, the atmosphere parameters can include atmosphere light parameters, music parameters, and fragrance parameters. Specifically, after the cabin atmosphere theme is generated, the atmosphere in the cabin can be dynamically adjusted and changed based on the atmosphere parameters included in the cabin atmosphere theme.

[0074] The above technical solution adds a function of controlling the cabin atmosphere after generating the cabin atmosphere theme.

[0075] Embodiment 2

[0076] Figure 4 A flow chart of another method for generating a cabin atmosphere theme provided in Embodiment 2 of the present invention. This embodiment is a further optimization of the above-mentioned embodiment. In this embodiment, the limitation and optimization of "generating cabin scene features that describe the cabin in natural language based on the cabin scene image" and the limitation and optimization of "generating a cabin atmosphere theme for the cabin based on the cabin scene features" are further provided.

[0077] like Figure 4 As shown, this embodiment 2 provides a method for generating a cabin atmosphere theme, which specifically includes the following steps:

[0078] S201: Acquire a cockpit scene image related to the cockpit.

[0079] S202: Input the in-cabin situation image and the out-cabin environment image into the target detection model to obtain in-cabin image features and out-cabin image features of the cockpit respectively.

[0080] In this embodiment, the target detection model can be specifically understood as a model for extracting features from an image, the input of the target detection model is an image, and the output is a feature extracted from the image. The target detection model can be obtained by training a basic model based on a training sample set of image-image features. For example, the basic model can adopt a convolutional neural network (CNN) architecture, such as a residual network (ResidualNetwork, ResNet). The CNN architecture is a deep learning neural network model that can be used for image recognition and analysis.

[0081] Specifically, the in-cabin scene image is input into the target detection model, and the output image features are recorded as in-cabin image features. The out-cabin environment image is input into the target detection model, and the output image features are recorded as out-cabin image features. Exemplarily, the in-cabin image features and the out-cabin image features can be represented in the form of vectors. It should be noted that before the in-cabin scene image and the out-cabin environment image are input into the target detection model, the in-cabin scene image and the out-cabin environment image can also be pre-processed based on computer vision (Computer Vision, CV) to facilitate the target detection model to extract image features from the image.

[0082] S203: Generate a cockpit scene feature that describes the cockpit in natural language according to the in-cabin image features and the out-cabin image features.

[0083] In this embodiment, after obtaining the in-cabin image features and the out-cabin image features, it is necessary to convert the in-cabin scene features and the out-cabin scene features into unified scene features described in natural language to provide more comprehensive information for generating the cabin atmosphere theme. The in-cabin image features and the out-cabin image are converted into a natural language description containing the overall scene information of the in-cabin and out-cabin, which is recorded as the cabin scene information. The cabin scene information combines the situational features in the cabin and the environmental features outside the cabin. By performing semantic recognition on the cabin scene, the cabin scene features that describe the cabin in natural language can be obtained.

[0084] The following two specific implementation methods are used to generate cockpit scene features.

[0085] As a specific implementation method, the steps of generating cabin scene features using natural language to describe the cabin can be optimized according to the cabin image features and the cabin image features, including:

[0086] a1) Fusing the in-cabin image features and the out-cabin image features to obtain a fused cockpit image feature of the cockpit.

[0087] In this embodiment, the in-cabin image features and the out-cabin image features are fused to obtain the fused cockpit image features of the cockpit. For example, the in-cabin image features and the out-cabin image features can be directly spliced ​​into a high-dimensional feature vector as the fused cockpit image features. Alternatively, the in-cabin image features and the out-cabin image features can be weighted summed based on the assigned weights, and the weighted summed features can be used as the fused cockpit image features.

[0088] b1) Inputting the cockpit image features into the image-text conversion model to generate cockpit scene information that describes the cockpit in natural language.

[0089] In this embodiment, the image-text conversion model can be specifically understood as a model that can convert image features into text. The input of the image-text conversion model is the image features, and the output is the natural language description corresponding to the image features. The image-text conversion model can be considered to be pre-trained. Specifically, the fused cockpit image features are input into the image-text conversion model, and the cockpit scene information of the cockpit is output in natural language.

[0090] c1) Perform semantic recognition on cockpit scene information and generate cockpit scene features that describe the cockpit in natural language.

[0091] In this embodiment, semantic recognition is performed on the cockpit scene information, and the recognized features are described in natural language and recorded as cockpit scene features.

[0092] For example, in Example 1, based on the understanding of the external environment image and the internal situation image, the cabin scene features are obtained as follows: weather: sunny, road condition: good, landscape: seaside road, number of passengers: 4, passenger distribution: front row: 2 adults, back row: 2 children, passenger emotional state: relaxed, happy. The corresponding driving scene is: family self-driving tour.

[0093] In Example 2, based on the external environment image and the internal situation image, the cabin scene features are obtained as follows: Weather: Rainy Night, Road Condition: Water on the Road, Landscape: Urban Expressway, Number of Passengers: 1, Passenger Distribution: Front Row: 1 Adult, Passenger Emotional State: Nervous, Concentrated. The corresponding driving scenario is: Commuting on a Rainy Night.

[0094] The above technical solution specifies the steps of generating cockpit scene features using natural language to describe the cockpit, and provides input data for the subsequent theme generation model. It can automatically adjust elements such as lighting, music and fragrance according to the emotional characteristics of different scenes, realize intelligent and personalized configuration of the cockpit atmosphere, meet the specific needs of users in different situations, and provide a more scene-specific experience.

[0095] As another specific implementation manner, the step of generating a cabin scene feature that describes the cabin in natural language according to the cabin image features and the cabin image features may be optimized, including:

[0096] a2) Inputting the in-cabin image features and the out-cabin image features into the image-text conversion model, respectively obtaining the in-cabin scene information and the out-cabin scene information describing the cockpit using natural language.

[0097] Different from the above implementation method in which the image features inside and outside the cabin are first fused, in this implementation method, the image features inside the cabin and the image features outside the cabin are first converted into scene information described in natural language. Specifically, the image features inside the cabin are input into the image-text conversion model, and the information describing the inside of the cabin in natural language is output, which is recorded as the scene information inside the cabin. The image features outside the cabin are input into the image-text conversion model, and the information describing the outside of the cabin in natural language is output, which is recorded as the scene information outside the cabin.

[0098] b2) Inputting the in-cabin scene information and the out-cabin scene information into a text generation model for fusion processing to generate cockpit scene information that describes the cockpit in natural language.

[0099] In this embodiment, the text generation model can be specifically understood as a model that processes the natural language description to obtain the processed natural language description. The text generation model can be obtained by pre-training. Specifically, the cabin scene information and the cabin scene information are input into the text generation model for fusion processing to generate the cabin scene information that describes the cabin in natural language.

[0100] c2) Perform semantic recognition on the cockpit scene information and generate cockpit scene features that describe the cockpit using natural language.

[0101] In this embodiment, it is also necessary to perform semantic recognition on the cockpit scene information, and the recognized features described in natural language are recorded as cockpit scene features.

[0102] The above technical solution specifies the steps of generating cockpit scene features using natural language to describe the cockpit, which provides input data for the subsequent topic generation model.

[0103] It is important to know that data fusion based on images inside and outside the car can transform multimodal data into a unified scene feature description, which can provide more comprehensive information for generating themes. And based on the sentiment analysis contained in the cabin situation, the emotions and keywords in the natural language description are matched with the preset situation tags to accurately adjust the cabin atmosphere, such as selecting the appropriate lighting color, music style and fragrance type.

[0104] For example, Figure 5 This is an example diagram of a process of generating cabin scene features in the execution of a method for generating a cabin atmosphere theme provided in the second embodiment of the present invention, such as Figure 5 As shown, in Example 1, the extracabin environment image and the intracabin situation image are processed to obtain the cockpit scene features that use natural language to describe the entire driving scene. The cockpit scene features specifically include: weather: clear, road conditions: good, landscape: seaside road, number of passengers: 4, passenger distribution: front row: 2 adults, back row: 2 children, passenger emotional state: relaxed, happy, and the corresponding driving scene is: family self-driving tour.

[0105] In Example 2, the external environment image and the internal situation image are processed to obtain the cockpit scene features that use natural language to describe the entire driving scene. The cockpit scene features specifically include: weather: rainy night, road conditions: water on the road, landscape: urban expressway, number of passengers: 1, passenger distribution: front row: 1 adult, passenger emotional state: nervous, focused, and the corresponding driving scene is: commuting on a rainy night.

[0106] S204: Input the cockpit scene features into a theme generation model to generate a cockpit atmosphere theme including at least one atmosphere control parameter.

[0107] Specifically, the cabin scene features are input into the theme generation model, and the theme generation model generates a cabin atmosphere theme based on each cabin scene feature. The cabin atmosphere theme includes at least one atmosphere control parameter for suggesting how to configure the atmosphere, such as a lighting configuration parameter, a music configuration parameter, or a fragrance configuration parameter. In this embodiment, the lighting, music, and fragrance of the cabin are automatically adjusted based on the generated configuration, and are appropriately adjusted according to the real-time status of the passengers, such as changes in facial expressions. Through dynamic atmosphere control, the cabin atmosphere is synchronized with the real-time situation.

[0108] The above technical solution specifies the steps for generating cockpit scene features and cockpit atmosphere themes. First, feature extraction is performed on multimodal image data. Then, the extracted image features are converted into unified cockpit scene features described in natural language. Finally, the cockpit scene features are processed based on the theme generation model to generate the cockpit atmosphere theme. By using the information from sensors and cameras inside and outside the vehicle, the vehicle's current overall driving scene, including the external environment and the situation inside the vehicle, is comprehensively perceived to generate a personalized cockpit atmosphere theme in real time, rather than relying on a fixed preset atmosphere. Through multimodal data fusion, natural language descriptions are generated, and atmosphere themes are dynamically generated with the help of the theme generation model, to achieve intelligent and personalized atmosphere adjustment.

[0109] In order to more clearly describe the cabin atmosphere theme generation method provided by the embodiment of the present invention, an actual application scenario of cabin atmosphere theme adjustment is taken as an example for explanation. Figure 6 This is a flowchart of a method for adjusting the cabin atmosphere theme in a certain application scenario provided by the second embodiment of the present invention, such as Figure 6 As shown, the execution steps of the cabin atmosphere theme adjustment method may specifically include:

[0110] S1. Acquire a cockpit scene image related to the cockpit.

[0111] S2. Input the in-cabin situation image and the out-cabin environment image into the target detection model to obtain the in-cabin image features and the out-cabin image features of the cockpit respectively.

[0112] S3. Fusing the in-cabin image features and the out-cabin image features to obtain fused cockpit image features of the cockpit.

[0113] S4. Input the cockpit image features into the image-text conversion model to generate cockpit scene information that describes the cockpit in natural language.

[0114] S5. Perform semantic recognition on the cockpit scene information to generate cockpit scene features that describe the cockpit using natural language.

[0115] S6. Inputting the cockpit scene features into a theme generation model to generate a cockpit atmosphere theme including at least one atmosphere control parameter.

[0116] S7. Adjust the atmosphere in the cabin based on the atmosphere parameters included in the cabin atmosphere theme.

[0117] Embodiment 3

[0118] Figure 7 This is a schematic diagram of the structure of a cabin atmosphere theme generation device provided in the third embodiment of the present invention. The device is applicable to the case of automatically generating a cabin atmosphere theme. The cabin atmosphere theme generation device can be implemented in the form of hardware and / or software and is generally integrated in an electronic device. Figure 7 As shown, the device includes: an image acquisition module 31, a feature description module 32 and a theme generation module 33; wherein,

[0119] An image acquisition module 31 is used to acquire cockpit scene images related to the cockpit;

[0120] A feature description module 32, for generating a cockpit scene feature for describing the cockpit in natural language according to the cockpit scene image;

[0121] The theme generation module 33 is used to generate a cabin atmosphere theme of the cabin according to the cabin scene characteristics and the theme generation model, and the theme generation model is a model for generating a cabin atmosphere theme.

[0122] The above technical solution obtains cockpit scene images related to the cockpit, realizes perception and understanding of the cockpit scene by processing the cockpit scene images, and generates cockpit scene features described in natural language; then the cockpit scene features are processed based on the theme generation model, and the cockpit atmosphere theme is generated in a generative way. It realizes the ability to automatically identify and understand the current overall driving scene of the vehicle, so as to dynamically generate a suitable cockpit atmosphere theme without relying on fixed preset atmosphere themes, thereby improving the adaptability, personalization and flexibility of cockpit atmosphere theme generation. By perceiving scene changes in real time, the cockpit atmosphere can be adjusted quickly to ensure that the driver and passengers are always in a comfortable and safe environment, thereby improving the comfort and safety of the driving process.

[0123] Optionally, the cockpit scene image includes an in-cabin situation image and an out-cabin environment image;

[0124] The feature description module 32 includes:

[0125] A feature extraction unit is used to input the in-cabin situation image and the out-cabin environment image into the target detection model to obtain the in-cabin image features and the out-cabin image features of the cockpit respectively;

[0126] The feature description unit is used to generate a cockpit scene feature that describes the cockpit in natural language according to the in-cabin image features and the out-cabin image features.

[0127] Optionally, the feature description unit is specifically used to:

[0128] Fusing the in-cabin image features and the out-cabin image features to obtain a fused cockpit image feature of the cockpit;

[0129] Input the cockpit image features into the image-text conversion model to generate cockpit scene information that describes the cockpit in natural language;

[0130] Semantic recognition is performed on the cockpit scene information to generate cockpit scene features that describe the cockpit in natural language.

[0131] Optionally, the feature description unit is specifically used to:

[0132] Input the in-cabin image features and the out-cabin image features into the image-text conversion model to obtain the in-cabin scene information and the out-cabin scene information describing the cockpit in natural language respectively;

[0133] Input the in-cabin scene information and the out-cabin scene information into the text generation model for fusion processing to generate cockpit scene information that describes the cockpit in natural language;

[0134] Semantic recognition is performed on the cockpit scene information to generate cockpit scene features that describe the cockpit in natural language.

[0135] Optionally, the theme generation module 33 is specifically used for:

[0136] The cockpit scene features are input into a theme generation model to generate a cockpit atmosphere theme including at least one atmosphere control parameter.

[0137] Optionally, the device further comprises a theme adjustment module, which is used to:

[0138] Matching the cabin atmosphere theme with the user's historical preference atmosphere information to obtain an updated cabin atmosphere theme; or

[0139] Receive the user's adjustment operation on the cockpit atmosphere theme and obtain an updated cockpit atmosphere theme.

[0140] Optionally, the device further includes a model optimization module, and after obtaining the updated cockpit atmosphere theme, further includes:

[0141] Based on the updated cockpit atmosphere theme, optimize the theme generation model.

[0142] Optionally, the device further comprises an atmosphere adjustment module, which is used to:

[0143] The atmosphere in the cabin is adjusted based on the atmosphere parameters included in the cabin atmosphere theme.

[0144] The cabin atmosphere theme generating device provided in the embodiment of the present invention can execute the cabin atmosphere theme generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0145] Embodiment 4

[0146] Figure 8 A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0147] like Figure 8 As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0148] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0149] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the cabin atmosphere theme generation method.

[0150] In some embodiments, the cabin atmosphere theme generation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the cabin atmosphere theme generation method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the cabin atmosphere theme generation method in any other appropriate manner (e.g., by means of firmware).

[0151] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0153] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a vehicle having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the vehicle. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0156] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0157] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for generating a cabin atmosphere theme as provided in any embodiment of the present invention.

[0158] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code for the operation of the present disclosure, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0159] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0160] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating a cabin atmosphere theme, characterized in that: include: Acquire cockpit scene images related to the cockpit; generating, according to the cockpit scene image, a cockpit scene feature that describes the cockpit in natural language; A cabin atmosphere theme of the cabin is generated according to the cabin scene features and a theme generation model, wherein the theme generation model is a model for generating a cabin atmosphere theme.

2. The method according to claim 1, characterized in that: The cockpit scene image includes an in-cabin situation image and an out-cabin environment image; The step of generating, according to the cockpit scene image, a cockpit scene feature that describes the cockpit in natural language comprises: Inputting the in-cabin situation image and the out-cabin environment image into a target detection model to obtain in-cabin image features and out-cabin image features of the cockpit respectively; A cockpit scene feature that describes the cockpit in natural language is generated according to the in-cabin image feature and the out-cabin image feature.

3. The method according to claim 2, characterized in that The step of generating a cabin scene feature that describes the cabin in natural language according to the cabin image feature and the cabin image feature includes: Fusing the in-cabin image features and the out-cabin image features to obtain fused cockpit image features of the cockpit; Inputting the cockpit image features into an image-text conversion model to generate cockpit scene information describing the cockpit in natural language; Semantic recognition is performed on the cockpit scene information to generate cockpit scene features that describe the cockpit in natural language.

4. The method according to claim 2, characterized in that: The step of generating a cabin scene feature that describes the cabin in natural language according to the cabin image feature and the cabin image feature includes: Inputting the in-cabin image features and the out-cabin image features into an image-text conversion model to obtain in-cabin scene information and out-cabin scene information describing the cockpit in natural language, respectively; Inputting the in-cabin scene information and the out-cabin scene information into a text generation model for fusion processing to generate cockpit scene information that describes the cockpit in natural language; Semantic recognition is performed on the cockpit scene information to generate cockpit scene features that describe the cockpit in natural language.

5. The method according to claim 1, characterized in that Generating a cockpit atmosphere theme of the cockpit according to the cockpit scene features includes: The cockpit scene features are input into the theme generation model to generate a cockpit atmosphere theme including at least one atmosphere control parameter.

6. The method according to claim 1, characterized in that: After generating the cockpit atmosphere theme of the cockpit, the method further includes: Matching the cabin atmosphere theme with the user's historical preference atmosphere information to obtain an updated cabin atmosphere theme; or Receive a user's adjustment operation on the cabin atmosphere theme, and obtain an updated cabin atmosphere theme.

7. The method according to claim 6, characterized in that After obtaining the updated cockpit atmosphere theme, the method further includes: Based on the updated cabin atmosphere theme, the theme generation model is optimized.

8. The method according to claim 1, characterized in that After generating the cockpit atmosphere theme of the cockpit, the method further includes: The atmosphere in the cabin is adjusted based on the atmosphere parameters included in the cabin atmosphere theme.

9. A cockpit atmosphere theme generating device, characterized in that: include: An image acquisition module, used to acquire cockpit scene images related to the cockpit; A feature description module, used to generate a cockpit scene feature describing the cockpit in natural language according to the cockpit scene image; A theme generation module is used to generate a cabin atmosphere theme of the cabin according to the cabin scene characteristics and a theme generation model, wherein the theme generation model is a model for generating a cabin atmosphere theme.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the cabin atmosphere theme generating method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the cockpit atmosphere theme generation method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method for generating a cabin atmosphere theme according to any one of claims 1 to 8 is implemented.

Images