Vehicle-mounted machine interface image processing method and device, vehicle-mounted electronic terminal and storage medium

Abstract

The invention relates to a vehicle-mounted machine interface image processing method and device, a vehicle-mounted electronic terminal and a storage medium, and the method comprises the steps: obtaining vehicle driving information based on a vehicle-mounted machine interface of a vehicle, and generating a first description word according to the vehicle driving information; obtaining a current image displayed on the vehicle-mounted terminal interface, and extracting edge features from the current image; and inputting the first descriptor and the edge feature of the current image into an image diffusion model for processing to generate a first target image. By adopting the method, the problem that the vehicle-mounted terminal interface cannot provide the visual experience matched with the driving situation can be solved, and the effect that the vehicle-mounted terminal interface can provide the visual experience matched with the driving situation is achieved.

Description

Technical Field

[0001] The present application relates to the field of vehicle-machine interface image display, and in particular to a vehicle-machine interface image processing method, device, vehicle-mounted electronic terminal and storage medium. Background Art

[0002] With the rapid development of automobile intelligence, in-vehicle interactive systems have replaced the mechanical control components of traditional cars. Users can directly navigate, search, play audio and video in the car interface, and the user experience of the in-vehicle interactive system has become increasingly important. As one of the important visual elements for user-car interaction, the car interface image has become increasingly personalized and adaptable. However, the current car interface images are usually fixed preset pictures that cannot provide a visual experience that matches the driving situation. The vehicle's operating status and driving mode may have an impact on the driver's psychological and emotional state. If the driver checks or operates the car interface at a high speed, there will be safety hazards.

[0003] There is currently no effective solution to the problem in related technologies that the car-machine interface cannot provide a visual experience that matches the driving scenario. Summary of the invention

[0004] Based on this, it is necessary to provide a vehicle-machine interface image processing method, device, vehicle-mounted electronic terminal and storage medium that can provide a visual experience matching the driving scenario in order to address the above technical problems.

[0005] In a first aspect, the present application provides a vehicle-machine interface image processing method, comprising:

[0006] Acquiring vehicle driving information based on a vehicle computer interface, and generating a first description word according to the vehicle driving information;

[0007] Acquire a current image displayed on the vehicle computer interface, and extract edge features from the current image;

[0008] The first description word and the edge feature of the current image are input into an image diffusion model for processing to generate a first target image.

[0009] In one embodiment, generating a first description word according to the vehicle driving information includes:

[0010] The vehicle driving information is input into a multimodal large model for processing, and a pre-configured example is used to guide the multimodal large model to process the input data to generate the first description word; wherein the example includes a relationship between the input data and the expected output.

[0011] In one embodiment, the vehicle driving information includes the vehicle driving speed, and before extracting the edge feature from the current image, the method further includes:

[0012] According to the vehicle speed, searching for a corresponding color value in a color mapping dictionary;

[0013] The hue and saturation of the current image are adjusted according to the determined color value.

[0014] In one embodiment, the method further comprises:

[0015] Acquire a real scene image captured by a camera of the vehicle, and generate a second description word according to the real scene image;

[0016] The second description word and the edge feature of the current image are input into an image diffusion model for processing to generate a second target image.

[0017] In one embodiment, generating a second description word according to the real scene image includes:

[0018] Inputting the real scene image into the multimodal large model for processing to generate semantic information;

[0019] Instruct the multimodal large model to integrate the semantic information with the current image to generate the second description word.

[0020] In one embodiment, the method further comprises:

[0021] In response to the content input by the user, generating a third description word;

[0022] The third description word and the edge feature of the current image are input into an image diffusion model for processing to generate a third target image.

[0023] In one embodiment, the method further comprises:

[0024] In response to a command input by a user, the vehicle interface display mode is switched, and the vehicle interface display mode includes:

[0025] The first display mode is used to generate a new vehicle interface image according to the vehicle driving information;

[0026] The second display mode is used to generate a new vehicle interface image based on the real scene image captured by the camera of the vehicle;

[0027] The third display mode is used to generate a new vehicle interface image based on the content input by the user.

[0028] In a second aspect, the present application also provides a vehicle-machine interface image processing device, comprising:

[0029] A generating module, configured to obtain vehicle driving information based on a vehicle-machine interface of the vehicle, and generate a first description word according to the vehicle driving information;

[0030] An extraction module, used to obtain a current image displayed on the vehicle computer interface and extract edge features from the current image;

[0031] The diffusion processing module is used to input the first description word and the edge feature of the current image into the image diffusion model for processing to generate a first target image.

[0032] In a third aspect, the present application also provides a vehicle-mounted electronic terminal, comprising a memory, a processor and a display screen, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented, and the display screen displays according to the execution result of the processor.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0034] The above-mentioned vehicle-machine interface image processing method, device, vehicle-mounted electronic terminal and storage medium obtain vehicle driving information according to the current vehicle-machine interface by integrating vehicle-related interface data, generate the first description word of the current vehicle information through a multimodal large model, integrate vehicle data such as weather, vehicle speed, and driving mode into the vehicle-machine interface image generation process, introduce a control network model, and use the current image based on the image diffusion model to create dynamic visual effects associated with driving operations, thereby enhancing the fun and safety of driving. Therefore, this embodiment solves the problem that the vehicle-machine interface cannot provide a visual experience that matches the driving situation, and achieves the effect that the vehicle-machine interface can provide a visual experience that matches the driving situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a hardware structure block diagram of a vehicle-mounted electronic terminal in one embodiment;

[0036] Figure 2 A diagram showing an application environment of a vehicle-machine interface image processing method in an embodiment;

[0037] Figure 3 A schematic diagram of a process of a vehicle computer interface image processing method in one embodiment;

[0038] Figure 4 A car wallpaper effect diagram corresponding to weather data in an embodiment;

[0039] Figure 5A data flow chart of an on-board electronic terminal running a vehicle-machine interface image processing method in one embodiment;

[0040] Figure 6 It is a structural block diagram of a vehicle-machine interface image processing device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0043] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, it can be executed on an in-vehicle electronic device. Figure 1 FIG. 1 is a hardware structure block diagram of a vehicle-mounted electronic terminal according to an embodiment of the present application. Figure 1 As shown, the vehicle-mounted electronic terminal may include one or more ( Figure 1Only one is shown in the figure) processor 101 and memory 102 for storing data, wherein processor 101 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above-mentioned vehicle-mounted electronic terminal may also include a transmission device 103 for communication functions and an input / output device 104, wherein the input / output device 104 may be a display screen for displaying according to the execution result of the processor 101. It can be understood by ordinary technicians in this field that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned vehicle-mounted electronic terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0044] The memory 102 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the vehicle-machine interface image processing method in this embodiment. The processor 101 executes various functional applications and data processing by running the computer program stored in the memory 102, that is, to implement the above method. The memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 102 may further include a memory remotely arranged relative to the processor 101, and these remote memories may be connected to the vehicle-mounted electronic terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0045] The transmission device 103 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the vehicle-mounted electronic terminal. In one example, the transmission device 103 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 103 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0046] The vehicle-machine interface image processing method provided in the embodiment of the present application can be applied to Figure 2 In the application environment shown. Among them, the vehicle-mounted electronic terminal 100 communicates with the server 200 through the network. The data storage system 300 can store the data that the server 200 needs to process. The data storage system 300 can be integrated on the server 200, or it can be placed on the cloud or other network servers. Among them, the server 200 can be implemented as an independent server or a server cluster composed of multiple servers.

[0047] The vehicle interface images of related technologies are poorly integrated with vehicle information. The vehicle interface images lack correlation with key information such as the vehicle's driving data and driving mode, and cannot provide a visual experience that matches the driving situation. However, the vehicle's operating status and driving mode may have an impact on the driver's psychological and emotional state. If these data are combined with the vehicle interface images, they can better reflect the driving situation and provide visual feedback that corresponds to the driving operation, thereby improving the driver's attention and driving safety to a certain extent.

[0048] Based on the analysis of the above situation, in one embodiment, Figure 3 A flow chart of a vehicle computer interface image processing method is provided. Figure 1 or Figure 2 The vehicle-mounted electronic terminal in the example is used to illustrate, including the following steps:

[0049] Step S101 : acquiring vehicle driving information based on a vehicle computer interface, and generating a first description word according to the vehicle driving information.

[0050] The vehicle computer interface includes but is not limited to a weather interface, a vehicle speed sensor interface, and a driving mode selector interface. Among them, the weather interface can provide weather data, the vehicle speed sensor can provide vehicle speed data, and the driving mode selector can provide driving mode data. Therefore, the vehicle driving information includes but is not limited to weather data, vehicle speed data, and driving mode data. Optionally, the driving mode includes a manual driving mode and an assisted driving mode. The on-board electronic terminal has a built-in processor, and these data can be transmitted to the processor in real time for subsequent processing.

[0051] After obtaining the vehicle driving information, the vehicle driving information is input into the multimodal large model (LLava) for processing to generate the first description word (prompt). The multimodal large model is an artificial intelligence system that can process multiple types of data such as text, images, and sounds.

[0052] Step S102, obtaining the current image displayed on the vehicle computer interface, and extracting edge features from the current image.

[0053] The current image includes the vehicle wallpaper (desktop background) or a partial image, which can be an image pre-configured in the vehicle system or an image selected by the user. After acquiring the current image, the current image is input into the control network model (ControlNet), and edge features are extracted from the current image to obtain an edge map of the current image. Among them, the control network model may include an encoder, a controller, and a generator. The encoder is used to extract features of the input image. The controller is used to receive the features of the encoder and adjust them according to the guidance image to generate a conditional signal useful to the generator. The generator generates the final image based on the conditional signal adjusted by the controller. This step uses the control network model to process the current image, which can ensure that the generated image has clear and accurate edges and shapes. It can perform specific edge precision control on the image generation process based on the current image.

[0054] Step S103: input the first description word and the edge feature of the current image into the image diffusion model for processing to generate a first target image.

[0055] The image diffusion model (Stable Diffusion) is an image generation model based on deep learning, which can use the diffusion process to generate new images. Specifically, in the noise addition stage (Forward Diffusion): Gaussian noise is gradually added to the original image, so that the information of the original image is gradually "diffused". In the denoising stage (Reverse Diffusion), try to restore the original image from completely random noise.

[0056] The image diffusion model will add elements related to vehicle driving information to the current image based on the requirements of the first descriptor, creating dynamic visual effects associated with driving operations. For example, if the speed of the car changes, the lines and colors on the wallpaper will also change; if the weather changes, the sky and clouds on the wallpaper will also change accordingly. If the speed is fast, the model will increase the blur effect of the objects on the wallpaper to reflect the sense of speed; if the weather is rainy, the model will add raindrops and blurred vision effects. The generated dynamic wallpaper will be displayed on the car screen, providing users with a richer and more interesting driving experience.

[0057] To obtain relevant data according to the weather interface, Figure 4 Car wallpaper renderings corresponding to weather data are provided, where a is the current image, and be are car wallpapers corresponding to the description words "spring", "summer", "autumn", and "winter", respectively.

[0058] In the above steps S101 to S103, by integrating the vehicle-related interface data, the vehicle driving information is obtained according to the current vehicle-machine interface, and the first description word of the current vehicle information is generated through the multimodal large model. The vehicle data such as weather, vehicle speed, and driving mode are integrated into the vehicle-machine interface image generation process, and the control network model is introduced. Based on the image diffusion model, the current image is used to create a dynamic visual effect associated with the driving operation, thereby enhancing the fun and safety of driving. Therefore, this embodiment solves the problem that the vehicle-machine interface cannot provide a visual experience that matches the driving situation, and achieves the effect that the vehicle-machine interface can provide a visual experience that matches the driving situation.

[0059] In one embodiment, in the above step S101, when generating the first description word based on the vehicle driving information, the vehicle driving information is input into the multimodal large model for processing, and a pre-configured example is used to guide the multimodal large model to process the input data to generate the first description word; wherein the example includes the relationship between the input data and the expected output.

[0060] In this embodiment, a multimodal large model is used to generate a first description of the current vehicle information based on the acquired vehicle driving information. When generating the first description, in addition to giving a description of the task to the multimodal large model, some pre-configured examples (for example, about 1-10) can also be provided. These examples show the input and expected output format, allowing the multimodal large model to learn how to complete the current task by observing these examples. This embodiment uses pre-configured examples to guide the multimodal large model, so that the multimodal large model can output descriptive words that meet expectations and improve the effectiveness of the descriptive words. The following is an example of using pre-configured examples to guide the multimodal large model, as shown below:

[0061] ##Role

[0062] You are a senior large model prompt engineer

[0063] ##Task

[0064] Generate a beautiful AI painting wallpaper prompt based on information extracted from [user current wallpaper] and [input parameters]

[0065] #Require

[0066] 1. Prompt cannot contain too many object elements

[0067] 2. Prompt cannot contain too much space information

[0068] ##Example

[0069] For example, if the weather is sunny, the car speed is fast, the driving mode is sports mode, and the current wallpaper is a road map, then the generated prompt will be "sunny road, fast-moving car, dynamic lines".

[0070] In one embodiment, the vehicle driving information includes the vehicle driving speed. In the above step S102, before extracting edge features from the current image, the method also includes: searching for a corresponding color value in a color mapping dictionary according to the vehicle driving speed; and adjusting the hue and saturation of the current image according to the determined color value.

[0071] In this embodiment, the color of the current image can be dynamically adjusted according to the driving state of the vehicle. By pre-configuring the color mapping dictionary, different colors can be presented in the current image later according to the change of vehicle speed. Specifically, a color mapping dictionary can be created to correspond vehicle speed intervals to specific color values. For example, when the vehicle speed is low, cold colors are used; as the vehicle speed increases, the color gradually transitions to warm colors. Exemplarily, the vehicle speed value is divided into different vehicle speed intervals through programming, and each vehicle speed interval corresponds to a different color value. The specific settings are as follows:

[0072] 0-30km / h: Cool colors (such as light blue or light green), low saturation.

[0073] 31-50km / h: Transitional color (such as light green), medium saturation.

[0074] 51-80km / h: Further transition to warm tones (such as yellow), medium to high saturation.

[0075] 81-120km / h: Warm colors (such as orange-red), high saturation.

[0076] In actual applications, the vehicle speed is monitored in real time, and the corresponding color value in the color mapping dictionary is found according to the current vehicle speed value. When the vehicle speed is between 0-30km / h, a cold color is selected, such as light blue or light green, and the saturation is kept low. As the vehicle speed increases, the color gradually transitions to a warm color. For example, when the vehicle speed reaches 80-120km / h, the color of the vehicle interface image changes to orange-red, and the color saturation is high. With this setting, the color of the vehicle interface image can be dynamically updated to ensure that as the vehicle speed changes, the color of the vehicle interface image can promptly reflect the current driving status of the vehicle.

[0077] The above embodiment provides one display mode of the vehicle-machine interface image, referred to as the first display mode, which is used to generate a new vehicle-machine interface image according to the vehicle driving information.

[0078] The vehicle interface images of related technologies lack adaptive capabilities. Vehicle interface images are usually fixed preset pictures that cannot be automatically adjusted and changed according to the vehicle status, environmental information or the user's personalized needs. For example, the lighting conditions inside the vehicle will be very different during the day and at night, but traditional vehicle interface images cannot automatically adapt to such changes and may appear too bright and dazzling at night; under different weather conditions, such as sunny, rainy or snowy days, the driver's mood and demand for visual atmosphere will also be different, but fixed vehicle interface images cannot meet these diverse needs.

[0079] Based on the analysis of the above situation, in one embodiment, a second display mode of the vehicle interface image is provided, which is used to generate a new vehicle interface image based on the real-scene image captured by the vehicle's camera. The vehicle-mounted electronic terminal can switch the vehicle interface display mode in response to the instruction input by the user. When the vehicle interface display mode is switched to the second display mode, the vehicle-mounted electronic terminal obtains the real-scene image captured by the vehicle's camera, generates a second description word based on the real-scene image; inputs the second description word and the edge features of the current image into the image diffusion model for processing to generate a second target image. Among them, when the vehicle-mounted electronic terminal generates the second description word based on the real-scene image, the real-scene image is input into the multimodal large model for processing to generate semantic information; the vehicle-mounted electronic terminal instructs the multimodal large model to integrate the semantic information with the current image to generate the second description word.

[0080] In this embodiment, an external camera is used to take real-time photos of the vehicle's current surroundings at a certain frequency, such as once every few minutes, to ensure that changes in the environment can be captured in a timely manner, rich image information can be obtained, and then the captured real-scene images are transmitted to the processor of the on-board electronic terminal.

[0081] A multimodal large model is used to analyze the acquired real-life images and extract semantic information. The multimodal large model will identify various elements in the real-life images, such as objects, colors, scenes, etc., and generate detailed descriptions related to these elements. For example, if there is a forest and a river in the real-life image, and the current image is a landscape picture, the second description word generated is "green forest, clear river, sunlight on the leaves". Optionally, when generating the second description word, in addition to giving a description of the task to the multimodal large model, you can also use the general language understanding and generation capabilities of the model pre-trained on large-scale data to try to handle completely new tasks. For example:

[0082] ##Task

[0083] Generate a beautiful AI painting car interface image prompt based on the integration of [user's current car interface image] and [semantic information]

[0084] #Require

[0085] 1. Fusion of the current vehicle interface image elements and the extracted voice information

[0086] 2. The generated prompt should contain text elements such as high definition and master level.

[0087] Next, determine the current image, which can be a vehicle desktop image pre-configured by the vehicle electronic terminal, or an image selected by the user. Input the current image into the control network model, extract edge features from the current image, and obtain an edge map of the current image. According to the extracted second description word, the image diffusion model is used to generate an image based on the current image. The image diffusion model will gradually diffuse and modify the current image so that it can be integrated into the semantic information described by the second description word, thereby generating a matching vehicle interface image. For example, if the second description word is "city skyline at sunset", the image diffusion model will add elements such as the color of the sunset and the outline of the city to the current image to generate a vehicle interface image with beautiful city scenery at sunset. The generated vehicle interface image will be displayed on the display screen of the vehicle electronic terminal, providing users with a more immersive and situational awareness driving experience.

[0088] The second display mode of the vehicle-machine interface image provided in this embodiment is based on the current surrounding environment information, and is obtained by taking photos through the external camera of the vehicle, and the semantics in the image are extracted through the multimodal large model to generate relevant second description words of the image semantic information. Then, according to the second description words, the control network model is introduced to generate a matching vehicle-machine interface image based on the current image based on the image diffusion model, providing a more immersive and situation-aware driving experience. The problem of the lack of adaptive ability of the vehicle-machine interface image in the related technology is solved, and the adaptability of the vehicle-machine interface image is realized.

[0089] The car interface images of the related technology have a low degree of customization. The car interface images that users can choose are limited, and it is difficult to meet the user's needs for unique and personalized car interface images. Users' aesthetics and preferences for car interface images are very diverse. Some people may like vibrant and dynamic designs, while others prefer a quiet and soothing atmosphere. The related technology cannot fully capture and meet these personalized needs, limiting the user's pursuit of personalized visual experience in the car environment.

[0090] Based on the analysis of the above situation, in one embodiment, a third display mode of the vehicle interface image is provided, which is used to generate a new vehicle interface image according to the content input by the user. The vehicle-mounted electronic terminal can switch the vehicle interface display mode in response to the instruction input by the user. When the vehicle interface display mode is switched to the third display mode, the vehicle-mounted electronic terminal generates a third description word in response to the content input by the user; the third description word and the edge features of the current image are input into the image diffusion model for processing to generate a third target image.

[0091] In this embodiment, the user can input detailed description content through the vehicle interface, that is, obtain the third description word. For example, "dreamlike starry sky style night driving vehicle interface image", "scenery vehicle interface image that matches the current weather", etc., and generate the third description word based on these description contents. After receiving the user input, the vehicle electronic terminal parses and processes the description word.

[0092] Next, determine the current image. The vehicle-mounted electronic terminal determines the current vehicle-machine interface image original image as the basis, which can be the vehicle-machine desktop image pre-configured by the vehicle-mounted electronic terminal, or the image selected by the user. Input the current image into the control network model, extract the edge features from the current image, and obtain the edge map of the current image. Based on the current image, use the image diffusion model to generate the image according to the third description word input by the user. The image diffusion model will modify and expand the original image according to the requirements of the description word to generate a highly personalized vehicle-machine interface image. For example, if the user enters "a landscape vehicle-machine interface image that matches the current weather", and the current weather is rainy, the image diffusion model will add raindrops, dark clouds, wet ground and other elements to the original image to generate a landscape vehicle-machine interface image that matches the rainy day. If the user enters "a night driving vehicle-machine interface image in the style of a dreamy starry sky", the image diffusion model will adjust the color, brightness, distribution of stars, etc., so that the generated vehicle-machine interface image conforms to the style of a dreamy starry sky. The generated customized vehicle-machine interface image will be displayed to the user on the display screen of the vehicle-mounted electronic terminal, and the user can select and save it according to his or her preferences.

[0093] In this embodiment, the user can input detailed description words (third description words) by customization, such as "dreamlike starry sky style night driving car interface image", "scenic car interface image that matches the current weather", etc., introduce the control network model, generate a new image based on the image diffusion model with the current image, and achieve highly personalized car interface image customization, ensuring that the generated car interface image has clear and accurate edges and shapes, and improving the quality and aesthetics of the image. The problem of low customization ability of car interface images in related technologies is solved, and a high degree of customization of car interface images is achieved.

[0094] In one embodiment, the vehicle-mounted electronic terminal includes the three vehicle-machine interface display modes described in the above embodiment, and the vehicle-mounted electronic terminal can switch the vehicle-machine interface display mode in response to the user input command. When using the vehicle-machine interface image system, the user can independently and conveniently select and switch between different modes according to their specific needs and preferences. These modes include real-time acquisition of environmental information mode, custom input mode, and interface acquisition of vehicle-machine information mode.

[0095] When the vehicle interface display mode is switched to the first display mode, the vehicle electronic terminal generates a vehicle interface image with dynamic visual effects associated with the driving operation through the interface connection with the vehicle, thereby enhancing the fun and safety during driving. When the vehicle interface display mode is switched to the second display mode, the vehicle electronic terminal can instantly capture various information of the surrounding environment through the camera outside the vehicle, and create a vehicle interface image that is highly consistent with the real-time environment, so that the user can immerse in a visual experience that corresponds to the current environment. When the vehicle interface display mode is switched to the third display mode, the vehicle electronic terminal allows the user to accurately input detailed third description words according to personal preferences, realize highly personalized vehicle interface image customization, and ensure that the generated vehicle interface image can fully meet the user's unique aesthetics and needs. This embodiment can obtain the vehicle interface image that best meets the user's expectations and needs by flexibly selecting and switching these different vehicle interface display modes, and improve the user's satisfaction and experience when using the vehicle electronic terminal.

[0096] In one embodiment, Figure 5 A data flow chart of a vehicle-mounted electronic terminal running a vehicle-machine interface image processing method is provided, such as Figure 5 As shown, the vehicle-mounted electronic terminal has the above three vehicle-machine interface display modes to meet the personalized needs of users in different scenarios. The vehicle-mounted electronic terminal uses the image diffusion model (Stable Diffusion) as the basic (Base) model, and extracts edge contours based on the current vehicle-machine wallpaper in combination with the control network model (ControlNet) technology, and obtains relevant information by automatically retrieving relevant data interfaces (weather interface, vehicle interface, etc.), or using a camera to capture real-life images outside the vehicle, and using a multimodal large model (LLava) to predict and generate the description words (prompt) of the artistic conception of the corresponding information, or custom input to generate the corresponding specific style and theme vehicle-machine wallpaper. Through this vehicle-mounted electronic terminal, users can flexibly select and switch between different modes according to the specific circumstances of actual needs, so as to achieve the purpose of obtaining the vehicle-machine wallpaper effect that best meets their expectations, and obtain the vehicle-machine wallpaper that best meets their expectations.

[0097] Based on the same inventive concept, the embodiment of the present application also provides a vehicle-machine interface image processing device for implementing the vehicle-machine interface image processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more vehicle-machine interface image processing device embodiments provided below can refer to the limitations of the vehicle-machine interface image processing method above, and will not be repeated here.

[0098] Figure 6 is a structural block diagram of the vehicle interface image processing device in this embodiment, as shown in Figure 6 As shown, the device includes: a generation module 11, an extraction module 12 and a diffusion processing module 13. The generation module 11 is used to obtain vehicle driving information based on the vehicle's vehicle-machine interface and generate a first description word according to the vehicle driving information; the extraction module 12 is used to obtain the current image displayed on the vehicle-machine interface and extract edge features from the current image; the diffusion processing module 13 is used to input the first description word and the edge features of the current image into the image diffusion model for processing to generate a first target image.

[0099] Each module in the above-mentioned vehicle interface image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0100] In one embodiment, the generation module 11 includes: a processing unit, which is used to input vehicle driving information into the multimodal large model for processing, and use a pre-configured example to guide the multimodal large model to process the input data to generate a first description word; wherein the example includes the relationship between the input data and the expected output.

[0101] In one embodiment, the device further includes: an adjustment module, which searches for a corresponding color value in a color mapping dictionary according to the vehicle's driving speed; and adjusts the hue and saturation of the current image according to the determined color value.

[0102] In one embodiment, the generation module 11 is also used to obtain a real scene image captured by a camera of the vehicle, and generate a second description word based on the real scene image; the diffusion processing module 13 is also used to input the second description word and the edge features of the current image into the image diffusion model for processing to generate a second target image.

[0103] In one embodiment, the generation module 11 includes: an integration unit, which is used to input the real scene image into the multimodal large model for processing to generate semantic information; and instruct the multimodal large model to integrate the semantic information with the current image to generate a second description word.

[0104] In one embodiment, the generation module 11 is further used to generate a third description word in response to the content input by the user; the diffusion processing module 13 is further used to input the third description word and the edge features of the current image into the image diffusion model for processing to generate a third target image.

[0105] In one embodiment, the device also includes: a response module, which switches the vehicle interface display mode in response to an instruction input by a user, and the vehicle interface display mode includes: a first display mode, which is used to generate a new vehicle interface image based on vehicle driving information; a second display mode, which is used to generate a new vehicle interface image based on a real-scene image captured by a camera of the vehicle; and a third display mode, which is used to generate a new vehicle interface image based on content input by the user.

[0106] In addition, in combination with the vehicle-machine interface image processing method provided in the above embodiments, a storage medium may also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by the processor, any of the vehicle-machine interface image processing methods in the above embodiments is implemented. In this embodiment, when the computer program is executed by the processor, the steps of the following method are implemented:

[0107] In one embodiment, when the computer program is executed by a processor, the steps of the following method are implemented:

[0108] Acquiring vehicle driving information based on a vehicle computer interface, and generating a first description word according to the vehicle driving information;

[0109] Obtain the current image displayed on the vehicle interface and extract edge features from the current image;

[0110] The first description word and the edge feature of the current image are input into the image diffusion model for processing to generate a first target image.

[0111] In one embodiment, when the computer program is executed by a processor, the steps of the following method are implemented:

[0112] The vehicle driving information is input into the multimodal large model for processing, and a pre-configured example is used to guide the multimodal large model to process the input data to generate a first description word; wherein the example includes a relationship between the input data and the expected output.

[0113] In one embodiment, when the computer program is executed by a processor, the steps of the following method are implemented:

[0114] According to the vehicle speed, the corresponding color value is found in the color mapping dictionary;

[0115] Adjust the hue and saturation of the current image based on the determined color values.

[0116] In one embodiment, when the computer program is executed by a processor, the steps of the following method are implemented:

[0117] Acquire a real scene image captured by a camera of the vehicle, and generate a second description word according to the real scene image;

[0118] The second description word and the edge feature of the current image are input into the image diffusion model for processing to generate a second target image.

[0119] In one embodiment, when the computer program is executed by a processor, the steps of the following method are implemented:

[0120] Input the real scene image into the multimodal large model for processing to generate semantic information;

[0121] Instructs the multimodal large model to integrate semantic information with the current image and generate a second description word.

[0122] In one embodiment, when the computer program is executed by a processor, the steps of the following method are implemented:

[0123] In response to the content input by the user, generating a third description word;

[0124] The third description word and the edge feature of the current image are input into the image diffusion model for processing to generate a third target image.

[0125] In one embodiment, when the computer program is executed by a processor, the steps of the following method are implemented:

[0126] In response to the user input command, the vehicle interface display mode is switched, and the vehicle interface display mode includes:

[0127] The first display mode is used to generate a new vehicle interface image according to the vehicle driving information;

[0128] The second display mode is used to generate a new vehicle interface image based on the real scene image captured by the vehicle's camera;

[0129] The third display mode is used to generate a new vehicle interface image based on the content input by the user.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0132] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0133] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A vehicle interface image processing method, characterized in that: include: Acquiring vehicle driving information based on a vehicle computer interface, and generating a first description word according to the vehicle driving information; Acquire a current image displayed on the vehicle computer interface, and extract edge features from the current image; The first description word and the edge feature of the current image are input into an image diffusion model for processing to generate a first target image.

2. The vehicle interface image processing method according to claim 1, characterized in that: Generating a first description word according to the vehicle driving information includes: The vehicle driving information is input into a multimodal large model for processing, and a pre-configured example is used to guide the multimodal large model to process the input data to generate the first description word; wherein the example includes a relationship between the input data and the expected output.

3. The vehicle-machine interface image processing method according to claim 1, characterized in that: The vehicle driving information includes the vehicle driving speed. Before extracting the edge feature from the current image, the method further includes: According to the vehicle speed, searching for a corresponding color value in a color mapping dictionary; The hue and saturation of the current image are adjusted according to the determined color value.

4. The vehicle-machine interface image processing method according to claim 1, characterized in that: The method further comprises: Acquire a real scene image captured by a camera of the vehicle, and generate a second description word according to the real scene image; The second description word and the edge feature of the current image are input into an image diffusion model for processing to generate a second target image.

5. The vehicle interface image processing method according to claim 4, characterized in that: Generating a second description word according to the real scene image includes: Inputting the real scene image into the multimodal large model for processing to generate semantic information; Instruct the multimodal large model to integrate the semantic information with the current image to generate the second description word.

6. The vehicle interface image processing method according to claim 1, characterized in that: The method further comprises: In response to the content input by the user, generating a third description word; The third description word and the edge feature of the current image are input into an image diffusion model for processing to generate a third target image.

7. The vehicle-machine interface image processing method according to claim 1, characterized in that: The method further comprises: In response to a command input by a user, the vehicle interface display mode is switched, and the vehicle interface display mode includes: The first display mode is used to generate a new vehicle interface image according to the vehicle driving information; The second display mode is used to generate a new vehicle interface image based on the real scene image captured by the camera of the vehicle; The third display mode is used to generate a new vehicle interface image based on the content input by the user.

8. A vehicle interface image processing device, characterized in that: include: A generating module, configured to obtain vehicle driving information based on a vehicle-machine interface of the vehicle, and generate a first description word according to the vehicle driving information; An extraction module, used to obtain a current image displayed on the vehicle computer interface and extract edge features from the current image; The diffusion processing module is used to input the first description word and the edge feature of the current image into the image diffusion model for processing to generate a first target image.

9. A vehicle-mounted electronic terminal, comprising a memory, a processor and a display screen, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented, and the display screen is displayed according to the execution result of the processor.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images