Environment reconstruction image processing method and device, electronic equipment and medium

Abstract

The invention discloses an environment reconstruction image processing method and device, electronic equipment and a medium, and relates to the field of intelligent driving. The method comprises the steps that semantic analysis is executed on a style instruction input by a user based on an intelligent cabin, a target material label corresponding to at least one material category is obtained, and the style instruction is used for indicating adjustment of a material style of an environment reconstruction image corresponding to the intelligent cabin; determining a plurality of pieces of target material data based on the target material label corresponding to each material category; and in the environment reconstruction image, based on the multiple pieces of target material data, correspondingly replacing the multiple pieces of original material data to obtain a target environment reconstruction image. According to the technical scheme, the user does not need to execute tedious manual multi-level click operation, but can input the style instruction into the intelligent cabin to enable the intelligent cabin to trigger the automatic switching process of the whole material style, so that the triggering step of changing the environment to reconstruct the material style in the real image is reduced, and the driving interaction experience of the intelligent cabin is improved.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent driving, and in particular to an environmental reconstruction image processing method, apparatus, electronic device, and medium. Background Technology

[0002] The vehicle's smart cockpit can display SR images (Surrounding Reality) on a screen. SR images are generated by the vehicle's sensors collecting data on the vehicle's surrounding environment, allowing users to observe the vehicle's surroundings through the SR images.

[0003] In existing technologies, when users want to change the style of SR images, they often need to manually perform multi-level selection operations on the display screen, which reduces the user's driving experience. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides an environment reconstruction image processing method, apparatus, electronic device, and medium to solve the problem that users need to manually perform multi-level point selection operations on a display screen to change the material style of an SR image.

[0005] The first aspect of this disclosure provides an environment reconstruction image processing method, comprising: Semantic parsing is performed on the style commands input by the user based on the smart cockpit to obtain target material tags corresponding to at least one material category; the style commands are used to instruct on adjusting the material style of the environmental reconstruction image corresponding to the smart cockpit; Based on the target material tags corresponding to each material category, multiple target material data are determined; In environmental reconstruction images, multiple original source data are replaced based on multiple target source data to obtain the target environment reconstruction image.

[0006] A second aspect of this disclosure provides an environment reconstruction image processing apparatus, comprising: The label determination module is used to obtain target material labels corresponding to at least one material category based on the style instructions input by the user to the smart cockpit; the style instructions are used to instruct the material style of the environmental reconstruction image corresponding to the smart cockpit to be adjusted. The material determination module is used to determine multiple target material data based on the target material tags corresponding to each material category; The image processing module is used to replace multiple original source data with multiple target source data in the environment reconstruction image to obtain the target environment reconstruction image.

[0007] A third aspect of this disclosure discloses a computer-readable storage medium storing a computer program for performing the environment reconstruction image processing method of any of the above embodiments.

[0008] A fourth aspect of this disclosure discloses an electronic device comprising: processor; Memory used to store processor-executable instructions; A processor is configured to read executable instructions from memory and execute the instructions to implement the environment reconstruction image processing method of any of the above embodiments.

[0009] The environmental reconstruction image processing method, apparatus, electronic device, and medium provided in the above embodiments of this disclosure perform semantic parsing on style commands input by the user based on the smart cockpit to obtain target material tags corresponding to at least one material category. The style command is used to instruct the adjustment of the material style of the environmental reconstruction image corresponding to the smart cockpit. Then, based on the target material tags corresponding to each material category, multiple target material data are determined. Finally, in the environmental reconstruction image, multiple original material data are replaced based on the multiple target material data to obtain the target environmental reconstruction image with the replaced material style. Therefore, the technical solution provided in the above embodiments of this disclosure eliminates the need for cumbersome manual multi-level selection operations. Instead, users can input a style command into the smart cockpit to trigger an automatic material style switching process, reducing the triggering steps for changing the material style in the environmental reconstruction image and improving driving safety and the driving interaction experience of the smart cockpit. Attached Figure Description

[0010] Figure 1 A vehicle system architecture provided as an exemplary embodiment of this disclosure; Figure 2 This is a schematic flowchart of an environmental reconstruction image processing method provided in an exemplary embodiment of the present disclosure; Figure 3 This is a schematic flowchart of step S201 of the environmental reconstruction image processing method provided in an exemplary embodiment of the present disclosure; Figure 4 This is a schematic flowchart of step S202 of the environment reconstruction image processing method provided in an exemplary embodiment of the present disclosure; Figure 5 This is a flowchart illustrating the process after step S2025 of the environmental reconstruction image processing method provided in an exemplary embodiment of this disclosure; Figure 6 This is a schematic flowchart of step S2021 of the environmental reconstruction image processing method provided in an exemplary embodiment of the present disclosure; Figure 7This is a schematic flowchart of the environmental reconstruction image processing method after step S201 provided in an exemplary embodiment of the present disclosure; Figure 8 This is a schematic flowchart of step S203 of the environmental reconstruction image processing method provided in an exemplary embodiment of the present disclosure; Figure 9 A schematic diagram of an environmental reconstruction image processing method apparatus provided as an exemplary embodiment of the present disclosure; Figure 10 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0011] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0012] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0013] Application Overview The intelligent cockpit of a vehicle is the core area that integrates a variety of intelligent interactive functions inside the vehicle. The intelligent cockpit can provide users with a convenient, comfortable and safe driving experience by integrating display, control and sensing technologies.

[0014] The environmental reconstruction and display function is an important interactive feature of the smart cockpit. It uses vehicle sensors, such as onboard cameras and radar, to collect data on the external environment and then displays a virtual or augmented reality image of the vehicle's surroundings in real time on the smart cockpit's interface, such as the instrument panel and central control screen. Users can intuitively understand road conditions around the vehicle within the smart cockpit, such as road markings, pedestrians, other vehicles, and obstacles, reducing blind spots and improving driving safety and convenience.

[0015] The reconstructed environment display image can include various material styles. In different scenarios, users can switch the material style of the reconstructed environment display image to obtain a visual experience that better suits their personal preferences or scenario needs. For example, when driving long distances, users may want the reconstructed environment display image to present a soft and soothing style to reduce visual fatigue; while at night or in complex road conditions, users may want to switch the reconstructed environment display image to a nighttime material style to reduce the display brightness of the reconstructed environment display image and avoid excessive brightness difference between the reconstructed environment display image and the nighttime environment, thereby improving driving safety.

[0016] However, the traditional way of adjusting the style of the content often requires users to perform cumbersome selection operations in the multi-level menu of the smart cockpit, find the settings interface for adjusting the style of the content, and then select the style of the content to be switched in the settings interface. This multi-level selection operation will distract the driver, affect driving safety, and reduce the user's driving experience.

[0017] Exemplary System Figure 1 An in-vehicle system architecture provided as an exemplary embodiment of this disclosure. For example... Figure 1 As shown, the in-vehicle system of this disclosure may include: an in-vehicle processing module 110, a processor 120, and a memory 130. The in-vehicle processing module 110 is used to collect user input control commands based on the smart cockpit, and to parse, convert, and transmit the control commands.

[0018] The processor 120, as the core computing unit of the vehicle system, is connected to the vehicle processing module 110 and the memory 130 via an internal bus or other communication methods. It is responsible for receiving parsed control commands from the vehicle processing module 110 and controlling the smart cockpit to perform corresponding interactive actions based on these commands. The memory 130 stores various types of data from the smart cockpit, allowing the processor 120 to retrieve the corresponding data at any time when responding to control commands.

[0019] For example, the vehicle-mounted processing module 110 may include an audio acquisition device 111, a text acquisition device 112, a communication device 113, and a neural network device 114, etc.

[0020] The audio acquisition device 110 is used to acquire voice control commands issued by the user in the smart cockpit, such as "switch the navigation interface to dark mode" or "adjust the ambient lighting in the car to blue." Since the processor 120 cannot directly respond to audio control commands, the vehicle processing module can also convert the voice control commands to convert analog audio signals into digital audio signals that can be processed by the processor 120.

[0021] The text acquisition device 112 is used to acquire text control commands input by the user based on the smart cockpit. For example, the smart cockpit includes a display screen, which can be a touch screen, allowing the user to directly input text control commands by clicking, swiping, or handwriting on the screen. For instance, if the user manually enters "Turn on AC mode" on the display screen, the text acquisition device can acquire the text "Turn on AC mode" as the text input command.

[0022] For example, the smart cockpit includes physical buttons, allowing users to input text control commands by clicking these buttons. For instance, a user selects a specific function option using the buttons to generate a text control command.

[0023] For example, after receiving a control command, the processor 120 can parse the format of the control command. If the control command is a voice control command, the processor 120 needs to first convert the voice control command into command text, and then perform semantic parsing on the command text to understand the intent of the voice control command, thereby executing the corresponding interactive action. If the control command is a text control command, the processor 120 does not need to perform voice-to-text conversion, but directly extracts the command text from the text control command and executes the subsequent interactive process.

[0024] The communication device 113 is responsible for data interaction with the processor 120, memory 130, and other in-vehicle devices (such as audio systems, air conditioning controllers, navigation modules, etc.) within the smart cockpit, enabling the transmission of control commands and related data. For example, the communication device 113 can also remotely interact with the smart cockpit's server, such as receiving system update commands and map data update packages from the server, or uploading user driving habit data and vehicle status information to the server, thereby enabling continuous optimization and upgrading of the smart cockpit's functions.

[0025] The communication device 113 can support multiple communication protocols, such as CAN bus protocol and Ethernet protocol, which can ensure the stability and real-time performance of data transmission with different types of vehicle equipment, enabling the intelligent cockpit system to work together efficiently.

[0026] The neural network device 114 performs deep semantic parsing on the acquired audio control commands or text control commands to obtain parsed control commands. For example, when a user says a vague voice control command such as "make this brighter," the neural network device 114 can combine the current cabin environment (such as current light intensity, time, etc.) and the user's historical operating habits to accurately understand whether the user intends to adjust the brightness of the interior lights or the display screen, and convert the parsed control command into a format that the processor 120 can recognize, so that the processor 120 can respond to the control command and increase the brightness of the interior lights or the display screen.

[0027] For example, the memory 130 may include a theme database 131 for storing material data of different material categories for environment reconstruction images under various theme styles, such as background image data, model material data, and various dynamic effect material data of environment reconstruction images. Among them, the background image data may include scene images of roads, buildings, vegetation, etc. under different weather conditions (such as sunny days, rainy days, and foggy days) and different time periods (such as daytime, dusk, and nighttime); the model material data may include geometric data and texture data of 3D models such as vehicles, pedestrians, traffic signs, and obstacles; the dynamic effect material data may include sequence frame data or particle effect parameters that simulate the dynamic characteristics of the real environment, such as raindrops, snowflakes, changes in light and shadow, and water flow.

[0028] The reconstructed environment images can include multiple different themes, which can be pre-set in the smart cockpit's memory 130. For example, the memory 130 can store different material data according to themes to form a complete set of material data corresponding to each theme, so that when the user needs to adjust the style of the reconstructed environment image material, the complete set of material data corresponding to the theme style in the memory 130 can be retrieved at any time.

[0029] Exemplary methods Figure 2 This is a schematic flowchart of an environmental reconstruction image processing method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to vehicles, in-vehicle electronic devices, or electronic devices. The following embodiments use in-vehicle electronic devices as the execution subject for illustrative purposes. Figure 2 As shown, it includes the following steps: S201: Perform semantic parsing on the style commands input by the user based on the smart cockpit to obtain target material tags corresponding to at least one material category.

[0030] In some embodiments, style instructions are used to instruct adjustments to the material style of the environmental reconstruction image corresponding to the smart cockpit. Style instructions can adjust the material style of one or more material data points in the environmental reconstruction image, or they can adjust the overall material style of the environmental reconstruction image. For example, a style instruction could be "Adjust the roadside grass to a 'cartoon' style," adjusting the material style of the vegetation material data in the environmental reconstruction image. For example, a style instruction could also be "Switch the interface style to 'cyberpunk style'," adjusting the overall material style of the environmental reconstruction image.

[0031] In some embodiments, the input method for style commands may include voice input and text input. For example, for voice input, a user can activate the smart cockpit's audio acquisition device by speaking a specified wake-up word, enabling the device to capture subsequent voice control commands issued by the user within the smart cockpit. These voice control commands constitute the user-input style commands. For instance, if a user says, "Hello, Xiao X, adjust the roadside grass to a 'cartoon' style," where "Hello, Xiao X" is the wake-up word, and the subsequent phrase "adjust the roadside grass to a 'cartoon' style" constitutes the voice-based style command.

[0032] For text input, users can input text control commands through the smart cockpit's touch screen, physical buttons, or connected external input devices (such as in-vehicle keyboards, mobile terminals, etc.). The text control commands are the style commands entered by the user.

[0033] In some embodiments, the display screen of the smart cockpit can display an interactive interface, and a designated area of ​​the interactive interface can display a text input box. Users can input text content through the text input box, such as "adjust the trees on both sides of the road to the 'ink painting' style". The smart cockpit will then use the controller to parse the input text content into a specific style instruction.

[0034] In some embodiments, the intelligent cockpit can perform semantic parsing of style instructions using a neural network device to extract the material categories contained in the style instructions and their corresponding target material tags. For example, the neural network device may include a pre-trained natural language processing model, such as the BERT model or an improved version thereof. Material categories are used to represent different types of material data in the reconstructed environment image; for example, buildings, sky, roads, vegetation, visual effects, color parameters, etc., are all different material categories. Each material category corresponds to a set of material data with similar visual features or functional attributes in the reconstructed environment image. For example, when a vehicle is driving in a city, there are multiple buildings on both sides of the road; these buildings have similar architectural features and belong to the same material category.

[0035] Target material tags are used to indicate the specific style that the material category needs to be adjusted to, such as "cartoon," "ink painting," "cyberpunk," "minimalist," and "retro." These material tags can be associated with a preset style feature library stored in the memory. Each material tag corresponds to a specific set of visual style parameters, such as color saturation, line thickness, texture features, and lighting effects.

[0036] For example, for the style instruction "adjust the roadside grass to 'cartoon' style", semantic parsing can extract the material category of the style instruction as "vegetation" and the target material tag as "cartoon", thus ensuring that the parsing results can accurately reflect the user's style adjustment intention.

[0037] S202: Based on the target material tags corresponding to each material category, determine multiple target material data.

[0038] After determining the target material tag, the target material data can be queried in the memory's material library based on the target material tag. Target material data is material data that matches both the material category and the target material tag. For example, if the material category is "vegetation" and the target material tag is "cartoon," the material library can be queried for cartoon-style vegetation material data that matches both the "vegetation" category and the "cartoon" material tag. Cartoon-style vegetation material data includes material data that conforms to the "cartoon" material style, such as the leaf shape, color scheme, and simplified texture details of cartoon elements. When the style command is "adjust the building to cyberpunk style," the material category is "building," and the target material tag is "cyberpunk." The material library can be queried for cyberpunk-style building material data that matches both the "building" category and the "cyberpunk style" tag. Building material data can include features unique to the cyberpunk style, such as neon lighting effects, towering geometric block structures, and strong color contrasts, thus providing accurate material support for subsequent environmental image style adjustments.

[0039] In some embodiments, the environmental reconstruction image may include material data corresponding to material categories with the target material tags, such as data on multiple buildings of different heights. Adjusting the material style of these buildings of different heights according to the same building data would destroy the realism and harmony of the environmental reconstruction image. Therefore, in order to ensure the accuracy and consistency of style adjustments for each material data in the environmental reconstruction image, differentiated style adjustment processing is required for target material data with different characteristics.

[0040] In some embodiments, to enhance the characteristics of the target material data, appropriate style elements can be added to these different target material data. Taking "cyberpunk style" as the target material tag and building material data as an example, neon signs conforming to the cyberpunk style can be added to the material data. The lighting effect of the neon signs can be adjusted to high color saturation to highlight the color characteristics of the cyberpunk style. For low-rise building material data, smaller neon signs can be used, while for tall building material data, relatively larger neon signs can be used.

[0041] By matching differentiated target material data with material data of different features in the environmental reconstruction image, it is possible to accurately fit the inherent attributes of each material data during the material style adjustment process, avoid the disproportion or detail distortion caused by uniform style processing, and thus improve the overall realism and style coordination of the environmental reconstruction image.

[0042] S203: In the environment reconstruction image, based on multiple target material data, multiple original material data are replaced accordingly to obtain the target environment reconstruction image.

[0043] In some embodiments, the location coordinates of the original material data whose style needs to be adjusted can be determined in the environment reconstruction image. For example, the pixel region boundaries of the original material data in the environment reconstruction image can be identified by an image segmentation algorithm to obtain the location coordinates of the original material data.

[0044] Subsequently, the retrieved target material data is adjusted according to the original material data to ensure that the size and angle of the target material data match the spatial proportion and spatial orientation of the original material data in the reconstructed environment image. For example, if the original material data is a building located on the left side of the road with a height of 1 / 3 of the height of the reconstructed environment image, the height of the target material data (such as a "cyberpunk style" building) should also be adjusted to 1 / 3 of the height of the reconstructed environment image, while maintaining its relative position to the road. Thus, the adjusted target material data replaces the original material data at its position coordinates to obtain the target environment reconstructed image.

[0045] In some embodiments, after replacing the original material data with the target material data, edge blending processing is also required on the material data whose style has not been adjusted, in order to reduce the incongruity between the target material data and other material data in the reconstructed environment image.

[0046] For example, edge blending processing may include techniques such as feathering edges and smoothing color transitions to eliminate stitching marks between the target material data and the original material data, so that the reconstructed image of the target environment presents a natural and coherent overall visual effect.

[0047] The method provided in this disclosure can perform semantic parsing on style commands input by the user, obtain the material category and corresponding target material tags from the style commands, and then determine multiple target material data based on the target material tags. Finally, in the reconstructed environment image, multiple original material data are replaced based on the multiple target material data to obtain a target environment reconstructed image with material style replacement. Therefore, the method provided in this disclosure can personalize the material style of the environment reconstructed image of the smart cockpit by directly inputting style commands, eliminating the need for manual multi-level selection operations on the smart cockpit's display screen, thus simplifying the process of adjusting the material style. At the same time, this semantic parsing-based material style adjustment method enhances the user's interactive experience and visual perception.

[0048] Figure 3 This is a schematic flowchart of step S201 of the environment reconstruction image processing method provided in an exemplary embodiment of the present disclosure.

[0049] like Figure 3 As shown above, in the above Figure 2 Based on the illustrated embodiment, step S201 may include the following steps: S2011: Perform text conversion on the style commands input by the user to the smart cockpit to obtain a text string.

[0050] In some embodiments, the format of the style command can be parsed to determine the input method of the style command. If the style command is input to the smart cockpit via voice input, that is, the style command is a voice control command, the style command needs to be converted into a text string.

[0051] In some embodiments, the memory may also store a speech processing model for the in-vehicle environment. For example, the speech processing model may include an acoustic model and a text model, such as an end-to-end automatic speech recognition (ASR) model based on a deep neural network. For example, the deep neural network may be a connectionist temporal classification (CTC) model or an attention model. This automatic speech recognition model can effectively suppress noise interference from wind noise, road noise, and in-vehicle conversations generated while driving, ensuring accurate text conversion of style instructions to obtain text strings even in high signal-to-noise ratio environments.

[0052] In some embodiments, the text string is a text sequence obtained by converting style instructions, and the content of the text string corresponds to the material data and corresponding material style that the user wants to adjust.

[0053] In some embodiments, if the style instruction is input to the smart cockpit via text input, there is no need for text conversion; the text string can be directly extracted from the style instruction.

[0054] S2012: Perform intent recognition on the text string to determine the intent information used to adjust the environment to reconstruct the image.

[0055] In some embodiments, the memory may also store a Large Language Model (LLM) Parser, which is a large language model that has undergone instruction fine-tuning, such as the GPT series models or the Llama series models. The LLM Parser can perform intent recognition on the text string output by the speech processing model to adjust the intent information of the reconstructed image.

[0056] Intent information is used to indicate the specific goal and direction of a user's expectation to adjust the material style of an environment reconstruction image through style commands, such as "adjust the material data of a specific area to the specified material style", "switch the interface style of the environment reconstruction image to a specified material style", or "add material data of a certain material style". Intent information can also indicate querying the material style of an environment reconstruction image, for example, "determine the current material style of the environment reconstruction image".

[0057] For example, when the text string is "Adjust the roadside grass to a 'cartoon' style", the intent information is "Adjust the style of the vegetation data in the reconstructed environment image to a cartoon style"; when the text string is "Switch the interface style to 'cyberpunk style'", the intent information is "Adjust the overall style of the reconstructed environment image to a cyberpunk style". The large language model instruction parser performs intent recognition on the text string, which can clarify whether the user is adjusting a local part of the reconstructed environment image or adjusting the overall style of the reconstructed environment image, further improving the accuracy and efficiency of semantic parsing.

[0058] S2013: Perform a structured transformation on the intent information to obtain the target material tags corresponding to the material categories.

[0059] In some embodiments, intent information can be input into a large language model (LLM). The LLM can extract information slots of multiple dimensions from the intent information and perform structured transformation on the information slots of multiple dimensions to obtain the target material tags corresponding to the material category.

[0060] For example, the structured transformation can be done using JSON (JavaScript Object Notation). For instance, for the intent "turn this city into a cyberpunk rainy night," the large language model will extract information slots from multiple dimensions, such as: style slot (base_style): "cyberpunk"; weather slot (weather): "heavy rain"; time slot (time_of_day): "night"; and object slot (target): "city". Then, the large language model can perform structured transformation on these multiple information slots to obtain "base_style": "cyberpunk", "weather": "heavy rain", "time_of_day": "night", and "target": "city", which are the target material tags.

[0061] The method provided in this disclosure converts style instructions from different input methods into text strings that can be processed by the intelligent cockpit through text conversion. This unifies the style instructions into a text format that facilitates intent recognition, avoiding processing logic confusion caused by differences in input methods (such as voice, text, touch selection, etc.). Based on this, intent recognition extracts the intent information expressed by the text strings, enabling the intelligent cockpit to accurately understand the user's style requirements for the reconstructed environment image. Then, through structured transformation, the intent information is converted into target material tags represented by information slots in multiple dimensions. This allows the intelligent cockpit to accurately match multiple target material data determined by multiple dimensions with the user's style instructions, improving the accuracy of the user's adjustments to the style of the reconstructed environment image.

[0062] Figure 4 This is a schematic flowchart of step S202 of the environment reconstruction image processing method provided in an exemplary embodiment of the present disclosure.

[0063] like Figure 4 As shown above, in the above Figure 2 Based on the illustrated embodiment, step S202 may include the following steps: S2021: Identify multiple material databases for different material categories.

[0064] In some embodiments, the theme database of the storage device can be divided into multiple material databases according to material categories. Each material database corresponds to a material category, such as buildings, visual effects, material textures, etc. Each material database may include multiple material data of the same material category.

[0065] For example, building material data in "ink painting style" and "cyberpunk style" belong to the same material category, i.e., both are building material data. The building material data in "ink painting style" and "cyberpunk style" can be stored in a material database for building material data. This will make it easier to query the target material data in the material database corresponding to the building material data when the style command needs to adjust the style of the building material data, thus improving the query efficiency of the target material data.

[0066] In some embodiments, based on material categories, multiple material databases may include geometric material databases, material / shader databases, and visual special effects (VFX) databases. Geometric material databases may include material data related to geometric models, such as vehicle material data, building material data, road material data, and vegetation material data. Material / shader databases may include material data related to materials and shaders, such as cartoon outline shaders, metallic rust materials, and holographic transparent materials. Visual special effects databases may include databases of visual effects such as weather and ambient lighting, for example, weather visual effects such as rain, snow, and sandstorms, and ambient lighting visual effects such as dusk, noon, and nighttime neon lights.

[0067] In some embodiments, the material database can also be divided into multiple sub-databases based on more granular material categories. For example, taking the geometry material database as an example, based on different material categories such as vehicle material data, building material data, road material data, and vegetation material data, corresponding sub-databases can also be divided, such as vehicle material database, building material database, road material database, and vegetation material database.

[0068] S2022: Obtain multiple material tags corresponding to multiple material data.

[0069] Material tags are tags generated by semantic processing of material data in advance. They are used to describe the style characteristics of the material data so that the material data can be accurately located and filtered in the future based on the target material tags.

[0070] In some embodiments, a single piece of material data can correspond to a single material tag. For example, for a piece of material data featuring a "cyberpunk style" building, the corresponding material tag would be "cyberpunk style". To enhance the query dimensionality of material data, a single piece of material data can also correspond to multiple material tags, which can be type attribute tags, style attribute tags, and function attribute tags, respectively. For instance, for a piece of material data featuring a "cyberpunk style" building, the material tags might include the type attribute tag "building", the style attribute tags "cyberpunk" and "futuristic", and the function attribute tag "glowing".

[0071] It should be noted that since the material database corresponds to material categories, material tags do not need to include category tags in order to shorten the tag length and improve the speed and accuracy of querying target material data.

[0072] In some embodiments, the processor can convert material tags into text embedding vectors and store them along with the corresponding material data in the corresponding material database. For example, the material database can be a vector database, such as the Milvus vector database or the Pinecone vector database, which supports efficient similarity queries.

[0073] S2023: Calculate the similarity between the target material tag corresponding to each material category and each material tag in the same material category.

[0074] For example, if the material category is geometric material and the target material tag is "cyberpunk style", the geometric material database can be located based on the material category. In this case, there is no need to query the material database corresponding to non-material categories, thereby improving the efficiency of determining the target material data.

[0075] In the geometric material database, the similarity between the target material tag and the material tags of each material data in the geometric material database can be calculated. For example, when there is material data A in the geometric material database, and its material tags are "cyberpunk style, futuristic city, dynamic lighting", the target material tag "cyberpunk style" can be compared with each material tag of material data A. Through a preset semantic similarity algorithm, the similarity between the target material tag and each material tag can be calculated.

[0076] S2024: In response to a similarity greater than or equal to a preset similarity threshold, determine multiple target material data.

[0077] In some embodiments, after calculating the similarity between the target material tag and each material tag, the similarity can be compared using a pre-set similarity threshold. For example, the preset similarity threshold can be flexibly set according to the needs of the actual application scenario, such as 80%. When the calculated similarity between the target material tag and the material tag of a certain material data reaches or exceeds 80%, the material data is initially selected as target material data.

[0078] In some embodiments, during the similarity calculation process, the material tags of all material data in the geometric material database can be traversed, and the similarity between each material tag and the target material tag can be calculated. All material data with a similarity greater than or equal to 80% can be filtered out, thus obtaining multiple target material data sets that meet the criteria. For example, if material data B exists in the geometric material database, with the material tags "cyberpunk architecture, metallic texture, neon colors," and the similarity between the target material tag "cyberpunk style" and "cyberpunk architecture" is 85%, exceeding the 80% threshold, then material data B will also be included in the set of multiple target material data sets. In this way, target material data matching the target material tags can be accurately filtered from a large amount of material data, providing a material foundation for subsequent environmental reconstruction image processing.

[0079] S2025: In response to all similarities being less than a preset similarity threshold, multiple target material data are generated based on the target material tags corresponding to each material category through the material generation model corresponding to the material data.

[0080] In some embodiments, if the similarity between the material tags of all material data in the material database and the target material tag is less than a preset similarity threshold, it means that there is no material data in the material database that matches the target material tag. At this time, the material data generation process can be triggered to input the style instruction into the large language model to generate the prompt word corresponding to the style instruction.

[0081] In some embodiments, the memory may also store a material generation model for generating target material data. Prompt words can be input into the material generation model to generate target material data based on the input prompt words. For example, the material generation model may be a "text-to-element" generation model, which can generate target material data based on input text-formatted prompt words.

[0082] Depending on the material category, the "text-to-element" generation model can employ various methods. For example, for geometric materials, the model can use a real-time Gaussian Splatting model, a Fast NeRF model, or a lightweight diffusion model to generate target material data based on prompts. For material / shader materials, the model can utilize specific diffusion models to generate texture maps for physically-based rendering (PBR) materials, such as Albedo maps, Normal maps, and Roughness maps, or directly generate shader code, i.e., the target material data. For visual effects materials, the model can dynamically adjust the particle system parameters of the smart cockpit, generate HDRI maps, or control volumetric lighting parameters to obtain the target material data.

[0083] In some embodiments, the target material data can be generated by invoking a "text-to-element" generation model through the vehicle's high-performance computing unit. For example, the high-performance computing unit can be an embedded neural network processor (NPU).

[0084] The method provided in this disclosure constructs a structured and searchable material data system by establishing multiple material databases for different material categories and associating each material data with a material tag generated through semantic processing. Based on this, an efficient and accurate material data matching mechanism is achieved by calculating the similarity between the target material tag and each material tag under the same material category and setting a preset similarity threshold as a judgment criterion. When the similarity is greater than or equal to the similarity threshold, the target material data can be directly queried and determined from the material database, thus making full use of the existing resources of the material database. When all similarities are less than the similarity threshold, it indicates that there is no matching target material data in the material database. In this case, multiple target material data corresponding to the target data tag are generated through the material generation model corresponding to the material data, thereby reducing the limitations of the material database in determining the target material data and improving the efficiency and scope of determining the target material data.

[0085] Figure 5 This is a schematic flowchart of the environmental reconstruction image processing method after step S2025 provided in an exemplary embodiment of the present disclosure.

[0086] like Figure 5 As shown above, in the above Figure 4 Based on the illustrated embodiment, the following steps may be included after step S2025: S20251: In response to a user's cache instruction based on at least one target material data input, determine the target material category corresponding to the target material data.

[0087] In some embodiments, a prompt message can be generated based on the target material data and displayed on the smart cockpit's screen. The prompt message asks the user whether to save the target material data. For example, the prompt message could be "Whether to cache the generated 'cyberpunk style' architectural model," guiding the user to input a cache instruction or a denial instruction based on the prompt message. For example, a cache instruction indicates that the user wants to cache the target material data, while a denial instruction indicates that the user does not want to cache the target material data.

[0088] When a user enters a cache command, the target material category to which the target material data belongs can be identified, such as geometric material category, material / shader material category, and visual effects material category, so that the target material data can be stored in the corresponding material database according to the target material category.

[0089] In some embodiments, if multiple target material data exist, the user can input a caching command based on one or a portion of the target material data. Based on this, after receiving the caching command, the vehicle can respond to the caching command and determine the target material category of the target material data specified by the caching command from among the multiple target material data.

[0090] S20252: Cache the target material data to the material database corresponding to the target material category.

[0091] In some embodiments, based on the target material category, the corresponding material database can be queried in the storage. For example, if the target material category is geometry, the geometry material database is queried; if it is a material / shader material category, the material / shader material database is queried; if it is a visual effects material category, the visual effects material database is queried. After determining the corresponding material database, the target material data can be cached in the corresponding material database.

[0092] In some embodiments, after caching the target material data to the corresponding material database, the target material data can be semantically processed to generate material tags for the target material data.

[0093] In some embodiments, before caching the target material data to the corresponding material database, a temporary material library can be created in the storage. The temporary material library is used to temporarily cache the target material data that has not been stored in the material database. For example, the temporary material library can be a free area in the storage, and the size of the temporary material library can be dynamically adjusted according to the available space in the current storage to avoid occupying too many system resources.

[0094] To save memory space, the storage system can periodically clear target material data from the temporary material library. For example, a periodic cleanup mechanism can be set up to automatically delete target material data from the temporary material library at certain time intervals (such as weekly or monthly) to free up temporary storage space and ensure efficient use of the storage system.

[0095] The method provided in this disclosure, through real-time interaction with the user, determines the target material data corresponding to the user-inputted cache command and the target material category of the target material data. Based on the target material category, it determines the corresponding material database in the memory and stores the target material data in a material database of the same material category. Therefore, the method provided in this disclosure can cache target material data generated by the generative model and satisfactory to the user in the material database, increasing the material diversity of the database. When the user submits the same or similar style command again, the cached material data can be quickly queried and retrieved directly from the expanded material database without repeatedly starting the generative model, thereby reducing the consumption of computing resources and improving the response speed of material style switching.

[0096] Figure 6 This is a schematic flowchart of step S2021 of the environmental reconstruction image processing method provided in an exemplary embodiment of the present disclosure.

[0097] like Figure 6 As shown above, in the above Figure 4 Based on the illustrated embodiment, step S2021 may include the following steps: S20211: Determine the multiple theme materials data and corresponding theme styles pre-installed in the smart cockpit.

[0098] In some embodiments, the smart cockpit's memory may store multiple theme material data and corresponding multiple theme styles. The multiple theme material data are complete sets of material data corresponding to multiple theme styles. The theme styles can be pre-generated, such as minimalist style, fresh style, night style, etc. Each theme style corresponds to a complete set of material data, which includes multiple material data of different material categories.

[0099] For example, the preset generation method can utilize procedural content generation (PCG) technology or AI-assisted generation tools, such as a diffusion-based 3D asset generator, to create material data in batches that conform to the theme style. For instance, it can generate material data for 10 different types of hovercraft and 20 types of futuristic buildings for a "cyberpunk" theme. This material data can adhere to unified specifications (such as uniform size ratios, interface specifications, and performance budgets like model face count and texture size) to ensure that the material data can be seamlessly combined to form environmental reconstruction images.

[0100] For example, for a minimalist style, the complete set of asset data may include minimalist geometric asset models, such as minimalist building asset data, vehicle asset data, and vegetation asset data. It may also include minimalist shader / material asset data, such as minimalist road surface materials and corresponding shader parameters. Furthermore, it may include minimalist visual effects, such as minimalist lighting effects and weather effects.

[0101] S20212: Perform material deconstruction on the theme material data according to different material categories to obtain multiple material data corresponding to the theme material data.

[0102] In some embodiments, the smart cockpit can deconstruct the complete set of material data corresponding to the theme style in the theme database in an offline state, so as to deconstruct the complete set of material data into multiple different material categories and independent material data, so as to facilitate the subsequent division of the material database.

[0103] To facilitate subsequent retrieval of target material data, each material data can store material tags corresponding to the theme style. For example, for each material data of "ink painting style", material tags used to represent "ink painting style" can be stored.

[0104] S20213: Classify the multiple material data according to different material categories to obtain the multiple material databases.

[0105] After deconstructing multiple sets of complete material data with different themes and styles, multiple material data of the same material category can be divided into a material database according to the material category.

[0106] For example, the intelligent cockpit can deconstruct the complete set of material data for "Ink Painting Style" and "Cyberpunk Style" in the theme database, obtaining multiple material data sets for "Ink Painting Style" and "Cyberpunk Style". The geometric material data for "Ink Painting Style" and "Cyberpunk Style" belong to the same material category; therefore, they can be grouped together into a geometric material database. Similarly, all material data corresponding to all theme styles can be classified in the above manner, resulting in multiple material databases. In this way, multiple material data sets of different theme styles can be combined according to style commands to form new environmental reconstruction images. For example, when the style command is "adjust the building to 'Cyberpunk Style'", if the theme style of the current environmental reconstruction image is minimalist, then while retaining other material data for minimalist style, the buildings can be replaced separately with building materials for "Cyberpunk Style", achieving diversified adjustment of material styles.

[0107] The method provided in this disclosure fully utilizes existing theme material data by deconstructing multiple thematic material data pre-set in a smart cockpit to construct multiple material databases categorized by material type. By deconstructing the entire set of material data, material data from different thematic styles can be cross-combined and used, thereby generating mixed-style environmental reconstruction images far exceeding the original number of pre-set themes, enriching the diversity and flexibility of style switching.

[0108] In some embodiments, when the smart cockpit's storage does not have a media database, or when no target media data matching the target media tag is found in the media database, a communication connection with a cloud server can be established via a communication device. The cloud server can store a cloud-based media database, and the media data stored in the cloud-based media database may differ from the media data stored in the vehicle-side media database, thereby expanding the scope of the search for target media data. For example, if no target media data matching the target media tag is found in the media database, the vehicle can upload the target media tag to the cloud server via the communication device, and the cloud server can perform a secondary search in the cloud-based media database based on the target media tag.

[0109] For example, if the smart cockpit's storage contains a media database, it can perform a secondary query for the target media data in the cloud media database if no matching media data is found in the database. If the smart cockpit's storage does not contain a media database, it can directly query the target media data in the cloud media database via the cloud server.

[0110] Figure 7 This is a schematic flowchart of the environmental reconstruction image processing method after step S201 provided in an exemplary embodiment of the present disclosure.

[0111] like Figure 7 As shown above, in the above Figure 2 Based on the illustrated embodiment, the following steps may be included after step S201: S2014: In response to the target material tag not including material tags corresponding to the theme style, determine the target theme style currently displayed in the smart cockpit.

[0112] When a style instruction does not involve adjusting the theme style of the reconstructed environment image, the target material tags obtained by semantic parsing the style instruction do not include material tags corresponding to the style theme. For example, a style instruction can adjust the material style of one or more material data in the reconstructed environment image; in this case, the target material tag corresponds to the material style of the material data. To ensure that the target material data queried subsequently conforms to the overall style of the reconstructed environment image, the target theme style currently displayed in the smart cockpit can be obtained.

[0113] The target theme style refers to the current theme style of the intelligent cockpit environment reconstruction image. For example, the target theme style can be identified and obtained by an application used to adjust the theme style of the environment reconstruction image after the intelligent cockpit receives a style instruction, so as to query the target material data in combination with the target theme style.

[0114] S2015: Based on the target theme style and the target material tags corresponding to each material category, query multiple target material data in multiple material databases.

[0115] To map a target theme style to target material data, a new target material tag can be generated by combining the target theme style and the target material tag. For example, if the target theme style of the smart cockpit is cartoon style, and the target material tag is "add snow visual effects," the target material tag does not reflect the specific material style of "snow visual effects." Therefore, the target theme style can be used as the material style of the target material tag, and the new target material tag will be "add cartoon-style snow visual effects."

[0116] Based on the new target material tags, the visual effects material database can be located by "snow visual effects", and specific target material data such as snowflake images and snow lighting effects in "cartoon style" can be queried in the visual effects material database by "cartoon style".

[0117] The method provided in this disclosure, when the target material tag does not contain a theme style, queries the target material data in conjunction with the current theme style to ensure the stylistic coordination and consistency of the generated environment reconstruction image. When the user's style instruction does not explicitly specify a material theme, the method automatically queries the material database for target material data based on the target theme style currently displayed in the smart cockpit. Therefore, the target environment reconstruction image generated in this way can satisfy the user's adjustment of the material style of local material data while maintaining consistency with the target theme style of the smart cockpit, thus improving the overall stylistic appearance and user experience of the target environment reconstruction image.

[0118] Figure 8 This is a schematic flowchart of step S203 of the environment reconstruction image processing method provided in an exemplary embodiment of the present disclosure.

[0119] like Figure 8 As shown above, in the above Figure 2 Based on the illustrated embodiment, step S203 may further include the following steps: S2031: Determine multiple display positions corresponding to multiple original material data in the environmental reconstruction image.

[0120] In some embodiments, the original material data can be determined based on the material category obtained by performing semantic parsing on the style instruction. For example, the style instruction "adjust the roadside grass to a 'cartoon' style" requires adjusting the material style of "grass". "Grass" corresponds to a type of vegetation material data in the geometric material data, namely grass material data. Therefore, the grass material data in the environment reconstruction image can be marked as the original material data.

[0121] After determining the original source data, multiple display positions of the original source data can be determined in the environmental reconstruction image. For example, the display positions can be represented in the form of coordinates. For instance, in the environmental reconstruction image, a three-dimensional coordinate system can be used to determine the coordinates of the display positions. With the center of the intelligent cockpit display interface as the origin, an X-axis, Y-axis, and Z-axis coordinate system can be established to determine the coordinate values ​​of each source data on different coordinate axes, thereby determining the coordinates of the display positions of the original source data. This provides a precise location basis for subsequent replacement or adjustment of target source data based on the display positions.

[0122] S2032: In multiple display locations of the environment reconstruction image, multiple original material data are replaced by multiple target material data to obtain the target environment reconstruction image.

[0123] In some embodiments, the environment reconstruction image can be rendered by the rendering pipeline, and the multiple rendered data can be combined and displayed by a combiner. For example, the rendering pipeline is a processing flow consisting of multiple rendering stages, such as vertex shading, geometry shading, and pixel shading. The combiner is used to merge and overlay the multiple data processed by the rendering pipeline according to preset rules such as hierarchical relationships, transparency, and blending modes, ultimately forming a complete environment reconstruction image.

[0124] The rendering pipeline can render the objects to be rendered from the original material data to the target material data at multiple display locations in the environment reconstruction image. This allows the target material data to replace the original material data in the environment reconstruction image, and the combiner combines the target material data with the original material data in the environment reconstruction image that has not changed its material style to obtain the target environment reconstruction image.

[0125] The method provided in this disclosure obtains the display position of the original material data in the reconstructed environment image and determines the corresponding coordinates. After retrieving the target material data, it renders and displays the target material data at the display position through a rendering pipeline and a combiner, thus obtaining the target environment reconstructed image. Therefore, after adjusting the material style, the target material data remains in the position of the original material data, and the original material layout of the reconstructed environment image does not change due to the style adjustment. Furthermore, by combining the target material data with the original material data whose style has not been adjusted, it is ensured that material data of different styles in the target environment reconstructed image can be naturally integrated, maintaining the overall scene's harmony and realism.

[0126] Exemplary device Figure 9 This is a schematic diagram of an apparatus for an environmental reconstruction image processing method provided as an exemplary embodiment of the present disclosure. The apparatus 900 of this embodiment can be used to implement the corresponding method embodiments of the present disclosure. See also... Figure 9 The device 900 may include: a label determination module 910, a material determination module 920, and an image processing module 930.

[0127] The label determination module 910 performs semantic parsing on the style commands input by the user based on the smart cockpit, obtaining target material labels corresponding to at least one material category. The style commands are used to instruct the adjustment of the material style of the environment reconstruction image corresponding to the smart cockpit. The material determination module 920 determines multiple target material data based on the target material labels corresponding to each material category. The image processing module 930 replaces multiple original material data in the environment reconstruction image based on the multiple target material data to obtain the target environment reconstruction image.

[0128] In some embodiments, the label determination module 910 is further configured to perform text conversion on the style instructions input by the user to the smart cockpit to obtain a text string, perform intent recognition on the text string to determine the intent information for adjusting the environment to reconstruct the image, and then perform structured conversion on the intent information to obtain the target material label corresponding to the material category.

[0129] In some embodiments, the material determination module 920 is further configured to determine multiple material databases for different material categories, wherein the material databases include multiple material data. Then, multiple material tags corresponding to the multiple material data are obtained, wherein the material tags are generated by performing semantic processing on the material data. Next, the similarity between the target material tag corresponding to each material category and each material tag of the same material category is calculated. In response to a similarity greater than or equal to a preset similarity threshold, multiple target material data are determined. In response to all similarities being less than the preset similarity threshold, multiple target material data are generated based on the target material tags corresponding to each material category using the material generation model corresponding to the material data.

[0130] In some embodiments, the device 900 may further include a storage module, which is configured to determine the target material category corresponding to the target material data in response to a caching instruction input by a user based on at least one target material data, and then cache the target material data to the material database corresponding to the target material category.

[0131] In some embodiments, the material determination module 920 is further configured to determine multiple theme material data and corresponding multiple theme styles pre-installed in the smart cockpit; perform material deconstruction on the theme material data according to different material categories to obtain multiple material data corresponding to the theme material data, wherein the material data stores material tags corresponding to the theme style; classify the multiple material data according to different material categories to obtain multiple material databases.

[0132] In some embodiments, the material determination module 920 is further configured to determine the target theme style currently displayed in the smart cockpit in response to the target material tag not including a material tag corresponding to the theme style; and to query multiple target material data in multiple material databases based on the target theme style and the target material tag corresponding to each material category.

[0133] In some embodiments, the image processing module 930 is further configured to determine multiple display positions corresponding to multiple original material data in the environment reconstruction image; and to replace multiple original material data with multiple target material data at multiple display positions in the environment reconstruction image to obtain a target environment reconstruction image.

[0134] The beneficial technical effects corresponding to the exemplary embodiment of this device 900 can be found in the corresponding beneficial technical effects in the exemplary method section above, and will not be repeated here.

[0135] Exemplary electronic devices Figure 10 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure.

[0136] like Figure 10 As shown, the electronic device 1000 includes at least one processor 1001 and a memory 1002.

[0137] The processor 1001 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1000 to perform desired functions.

[0138] The memory 1002 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1001 may execute one or more computer program instructions to implement the environment reconstruction image processing methods and / or other desired functions of the various embodiments of this disclosure described above.

[0139] In one example, the electronic device 1000 may also include an input device 1003 and an output device 1004, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0140] The input device 1003 may also include, for example, a keyboard, a mouse, etc.

[0141] The output device 1004 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0142] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device 1000 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1000 may include any other suitable components depending on the specific application.

[0143] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the environmental reconstruction image processing methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0144] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0145] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the environmental reconstruction image processing methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0146] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0147] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0148] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. An environment reconstruction image processing method, comprising: Semantic parsing is performed on the style commands input by the user based on the smart cockpit to obtain target material tags corresponding to at least one material category; The style command is used to instruct on adjusting the material style of the environmental reconstruction image corresponding to the smart cockpit; Based on the target material tags corresponding to each material category, multiple target material data are determined; In the reconstructed environment image, based on the multiple target material data, multiple original material data are replaced accordingly to obtain the target environment reconstructed image.

2. The environmental reconstruction image processing method according to claim 1, wherein, The process of performing semantic parsing on the style commands input by the user based on the smart cockpit to obtain target material tags corresponding to at least one material category includes: The style commands input by the user into the smart cockpit are converted into text strings. Intent recognition is performed on the text string to determine the intent information used to adjust the reconstructed environmental image; The intent information is subjected to a structured transformation to obtain the target material tags corresponding to the material category.

3. The environmental reconstruction image processing method according to claim 1, wherein, The process of determining multiple target material data based on the target material tags corresponding to each material category includes: Multiple material databases for different material categories are identified, wherein the material database includes multiple material data; Obtain multiple material tags corresponding to the multiple material data, wherein the material tags are generated by performing semantic processing on the material data; Calculate the similarity between the target material tag corresponding to each material category and each material tag in the same material category; In response to the similarity being greater than or equal to a preset similarity threshold, the plurality of target material data are determined; In response to all of the aforementioned similarities being less than a preset similarity threshold, the multiple target material data are generated based on the target material tags corresponding to each material category through the material generation model corresponding to the material data.

4. The environmental reconstruction image processing method according to claim 3, wherein, In response to the similarity being less than a preset similarity threshold, after generating the multiple target material data based on the target material tags corresponding to each material category using the material generation model corresponding to the material data, the process further includes: In response to a user's cache instruction based on at least one of the target material data, the target material category corresponding to the target material data is determined; The target material data is cached in the material database corresponding to the target material category.

5. The environmental reconstruction image processing method according to claim 3, wherein, The multiple material databases used to determine different material categories include: Determine multiple thematic material data and corresponding thematic styles pre-installed in the smart cockpit; The theme material data is deconstructed according to different material categories to obtain multiple material data corresponding to the theme material data. The material data contains material tags corresponding to the theme style. The multiple material data are classified according to different material categories to obtain the multiple material databases.

6. The environmental reconstruction image processing method according to claim 5, wherein, After performing semantic parsing on the style commands input by the user based on the smart cockpit to obtain target material tags corresponding to at least one material category, the process further includes: In response to the fact that the target material tag does not include a material tag corresponding to the theme style, the target theme style currently displayed in the smart cockpit is determined; Based on the target theme style and the target material tags corresponding to each material category, multiple target material data are queried from the multiple material databases.

7. The environmental reconstruction image processing method according to claim 1, wherein, In the reconstructed environment image, based on the multiple target material data, multiple original material data are replaced accordingly to obtain the target environment reconstructed image, including: Determine multiple display positions corresponding to multiple original source data in the reconstructed environment image; In the multiple display locations of the reconstructed environment image, the multiple original material data are replaced by the multiple target material data to obtain the target environment reconstructed image.

8. An environmental reconstruction image processing apparatus, comprising: The tag determination module is used to obtain target material tags corresponding to at least one material category based on the style instructions input by the user for the smart cockpit; The style command is used to instruct on adjusting the material style of the environmental reconstruction image corresponding to the smart cockpit; The material determination module is used to determine multiple target material data based on the target material tags corresponding to each material category; An image processing module is used to replace multiple original material data with corresponding original material data based on the multiple target material data in the environment reconstruction image to obtain a target environment reconstruction image.

9. A computer-readable storage medium storing a computer program for performing the environment reconstruction image processing method as described in any one of claims 1-7.

10. An electronic device, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the environment reconstruction image processing method as described in any one of claims 1-7.

Images