Control method and device and carrying tool

Abstract

The embodiment of the invention provides a control method and device and a carrying tool. The method can be applied to the field of intelligent cabins. The method comprises the steps that when it is detected that a user in a cabin indicates input of image generation, sensing data are obtained, and the sensing data comprise a first image collected by a camera of a carrying tool or a voice signal collected by a microphone of the carrying tool; and according to the sensing data, controlling a display area in the carrying tool to display a second image. The embodiment of the invention can be applied to an intelligent automobile or an electric automobile, and is helpful for improving the individuation degree of the content displayed in the cabin, thereby being helpful for improving the use experience of a user.

Description

Control method, device and vehicle Technical Field

[0001] Embodiments of the present application relate to the field of smart cockpits, and more specifically, to a control method, device, and vehicle. Background Art

[0002] With the increasing number of display screens in vehicle cabins, and the content displayed on these screens (e.g., wallpapers) becoming increasingly homogenized, improving the personalization of the content displayed on these screens has become an urgent issue.

[0003] Summary of the Invention

[0004] The present application provides a control method, device, and vehicle that help improve the personalization of content displayed in the cabin, thereby helping to improve the user experience.

[0005] The vehicles in this application may include road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or recreational equipment. For example, the vehicle may be a vehicle, which is a vehicle in a broad sense, and may be a vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a mower, a harvester, etc.), amusement equipment, a toy vehicle, etc. The embodiments of this application do not specifically limit the type of vehicle. For another example, the vehicle may be a vehicle such as an airplane or a ship.

[0006] In a first aspect, the present application provides a control method, which is applied to a cockpit of a vehicle, wherein the cockpit includes a first display area, and the method includes: when an input indicating generation of an image by a first user in the cockpit is detected, obtaining sensor data, the sensor data including a first image captured by a camera of the vehicle or a first voice signal captured by a microphone of the vehicle; and according to the sensor data, controlling the first display area to display a second image.

[0007] Based on the above technical solution, the vehicle can perform secondary creation based on data collected by the camera or microphone, thereby controlling the first display area to display the second image. In this way, the user can view the image created by using local materials through the display area in the cabin, avoiding the visual fatigue caused by the display area showing a single and homogeneous image, helping to enhance the user's visual experience and personalize the content displayed in the cabin, thereby helping to enhance the user's driving experience.

[0008] When the first image includes images captured by the camera inside the cabin, the problem of poor selfie or group photo effects caused by the single camera arrangement position inside the cabin or the single landscape background inside the cabin can be solved, which helps to improve the shooting effect inside the cabin and enhance the user's shooting fun.

[0009] In some possible implementations, the first display area is a display area on a display screen in the vehicle cabin. For example, if the vehicle is a vehicle, the first display area may be a display area on a central control screen in the vehicle cabin.

[0010] In some possible implementations, the area where the first user is located is associated with the first display area.

[0011] Taking the vehicle as an example, if the first user is located in the main driving area, then the first display area can be the display area on the central control screen, or the first display area can be the display area on the central control screen close to the main driving area.

[0012] In some possible implementations, detecting an input from a first user in the cabin instructing to generate an image includes: detecting a voice instruction issued by the first user, where the voice instruction is used to instruct to generate an image.

[0013] In some possible implementations, before detecting the input of the first user in the cabin indicating the generation of an image, the method also includes: controlling the first display area to display a first interface, the first interface including a control, and the control is a control for generating an image; wherein, detecting the input of the first user in the cabin indicating the generation of an image includes: detecting the operation of the user clicking the control.

[0014] In some possible implementations, the method further includes: setting the second image as wallpaper on the first display area.

[0015] In some possible implementations, controlling the first display area to display the second image based on the sensing data includes: generating the second image based on the sensing data; and controlling the first display area to display the second image.

[0016] In some possible implementations, the first image may be one or more frames of images.

[0017] In some possible implementations, the second image may be a photo (eg, wallpaper), a moving picture (eg, a graphics interchange format (GIF) moving picture), or a video.

[0018] In combination with the first aspect, in certain implementations of the first aspect, the sensor data includes the first voice signal, and controlling the first display area to display the second image based on the sensor data includes: controlling the first display area to display the second image based on characteristic information of the first voice signal; wherein the characteristic information of the first voice signal includes at least one of the pitch, timbre and tone of the first voice signal.

[0019] Based on the above technical solution, after acquiring the voice signal captured by the microphone, the second image can be generated based on the characteristic information of the voice signal. This allows the user to view an image created based on the characteristic information of their voice signal on the in-cabin display area, avoiding the visual fatigue caused by the display area displaying a single, homogeneous image. This helps enhance the user's visual experience and personalize the content displayed in the cabin, thereby improving the user's driving experience.

[0020] Furthermore, by incorporating the characteristic information of the voice signal, the resulting image can be made more consistent with the user's expectations. For example, if the voice signal is detected as that of an elderly person or a child, the resulting image can be generated using an image style that matches the user's age. This makes the image displayed in the in-cabin display more appropriate to the user's age profile, helping to enhance the user experience.

[0021] Alternatively, when the tone of the voice signal is detected to be happy or sad, the final image can be generated using an image style that matches the user's mood, so that the image displayed through the display area in the cockpit is more in line with the user's current mood, helping to enhance the user's driving experience.

[0022] In some possible implementations, controlling the first display area to display the second image based on the characteristic information of the first voice signal includes: generating the second image based on the characteristic information of the first voice signal; and controlling the first display area to display the second image.

[0023] In combination with the first aspect, in certain implementations of the first aspect, the sensor data includes the first image, and the method further includes: performing style transfer on the first image to obtain the second image.

[0024] Based on the above technical solution, the second image can be obtained by performing style transfer on the image captured by the camera, which helps to improve the user's visual experience and avoids the visual fatigue caused to the user by displaying a single and homogeneous wallpaper on the car display screen, thereby helping to improve the user's driving experience.

[0025] At the same time, when the camera is an in-cabin camera, it can solve the problem of poor selfie or group photo effects caused by a single camera layout position in the cabin or a single landscape background in the cabin, which helps to improve the shooting effect in the cabin and enhance the user's shooting fun.

[0026] In combination with the first aspect, in certain implementations of the first aspect, the sensor data includes the first image, and the method further includes: determining text description content based on the first image, the text description content including at least one item of the scene, object, person or environment corresponding to the first image; and determining the second image based on the text description content.

[0027] Based on the above technical solution, the corresponding text description content can be determined based on the first image, and then the second image can be generated based on the text description content, thereby realizing secondary creation of the image captured by the camera, avoiding the visual fatigue caused to the user by displaying a single and homogeneous image in the display area, and helping to enhance the user's visual experience and the degree of personalization of the content displayed in the cabin, thereby helping to enhance the user's driving experience.

[0028] In some possible implementations, determining text description content based on the first image includes: inputting the first image into an image to text model to obtain the text description content.

[0029] In some possible implementations, determining the second image based on the text description content includes: inputting the text description content into a text-to-image model to obtain the second image.

[0030] In some possible implementations, upon detecting an input from a first user in the cabin indicating generation of an image, the vehicle may periodically acquire images captured by a camera outside the cabin.

[0031] For example, in a first cycle, a first image captured by an external camera is acquired, and a first text description corresponding to the first image is obtained based on the first image. Then, based on the first text description, the first display area is controlled to display the second image.

[0032] For another example, during the second cycle, a fifth image captured by the exterior camera is acquired, and a second text description corresponding to the fifth image is obtained based on the fifth image. Based on the second text description, the first display area is controlled to display the sixth image. The first text description and the second text description are different.

[0033] This allows the image displayed in the first display area to be updated immediately after the camera outside the cabin captures an image with different text descriptions, further enhancing the user's visual experience and avoiding the visual fatigue caused by the vehicle's display area displaying a single, homogeneous wallpaper. Furthermore, the content displayed in the first display area always matches the scenery outside the cabin, helping to enhance the user's driving experience.

[0034] In combination with the first aspect, in certain implementations of the first aspect, the sensor data includes the first image, and the method further includes: inputting the first image into an image to image model to obtain the second image.

[0035] Based on the above technical solution, the second image can be generated by inputting the image captured by the camera into the image model, thus achieving secondary creation of the camera-captured image. This avoids the visual fatigue caused by the display area showing a single, homogeneous image, helps to enhance the user's visual experience and personalize the content displayed in the cabin, thereby helping to improve the user's driving experience.

[0036] In combination with the first aspect, in certain implementations of the first aspect, the cabin also includes a second display area, and the method also includes: when a second voice signal of a second user in the cabin is detected, controlling the second display area to display a third image based on the second voice signal and the second image, and the second display area is associated with the area where the second user is located.

[0037] Based on the above technical solution, when a second voice signal is detected from a second user in the cabin, a third image can be generated based on the second image and the second voice signal. Simultaneously, the vehicle can switch from displaying the second image in the first display area to displaying the third image in the second display area, allowing the second user to view the updated image in their own display area, thereby improving the user experience.

[0038] For example, taking the vehicle as an example, if the second user is located in the passenger seat area, the second display area can be the display area on the passenger seat entertainment screen, or the second display area can be the display area near the passenger seat area in the central control screen.

[0039] In some possible implementations, detecting the second voice signal of the second user in the cockpit includes: after controlling the first display area to display the second image, detecting the second voice signal of the second user in the cockpit.

[0040] In some possible implementations, the second voice signal is used to modify the second image, or to recreate the second image.

[0041] In combination with the first aspect, in certain implementations of the first aspect, before obtaining the sensor data, the method also includes: controlling the first display area to display a fourth image; wherein, controlling the first display area to display the second image based on the sensor data includes: controlling the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and the sensor data.

[0042] Based on the above technical solution, a second image can be generated using sensor data and the original image displayed on the display area. Simultaneously, the vehicle (e.g., a vehicle) can switch from displaying the fourth image on the first display area to displaying the second image, thereby automatically updating the wallpaper displayed on the first display area. This avoids visual fatigue caused by displaying a single, homogeneous image on the display area, helps enhance the user's visual experience and the personalization of the content displayed in the cabin, thereby helping to improve the user's driving experience.

[0043] In combination with the first aspect, in some implementations of the first aspect, the method further includes: establishing a connection with a terminal device; and sending the second image to the terminal device.

[0044] Based on the above technical solution, after the vehicle and the terminal device establish a connection, the vehicle can also send the generated second image to the terminal device, so that the user can view the image generated by the vehicle on the terminal device, and save the image locally on the terminal device in time, share it with other users through the terminal device, or continue to create the image through the local photo editing software of the terminal device, which helps to improve the user experience.

[0045] In some possible implementations, sending the second image to the terminal device includes: sending the second image to the terminal device via a short-range communication technology; or sending the second image to the terminal device via a server.

[0046] In some possible implementations, before sending the second image to the terminal device, the method further includes: determining that the account logged in on the vehicle is associated with the account logged in on the terminal device.

[0047] Exemplarily, the account logged in on the vehicle is associated with the account logged in on the terminal device, including: the account logged in on the vehicle is the same as the account logged in on the terminal device, or the account logged in on the vehicle and the account logged in on the terminal device are in the same family group, or the account logged in on the terminal device is an account authorized by the vehicle.

[0048] In combination with the first aspect, in some implementations of the first aspect, the method further includes: sending the second image to a cloud server.

[0049] Exemplarily, the cloud server may be a server corresponding to a wallpaper application.

[0050] Based on the above technical solution, the vehicle can facilitate other users to download the second image through the wallpaper application by sending the information of the second image to the cloud server.

[0051] In second aspect, the present application provides a control device, which includes a detection unit, an acquisition unit and a control unit, wherein the acquisition unit is used to acquire sensor data when the detection unit detects input of a first user instruction to generate an image in the cabin of a vehicle, and the sensor data includes a first image captured by the camera of the vehicle or a first voice signal captured by the microphone of the vehicle; the control unit is used to control the first display area to display a second image according to the sensor data, and the cabin includes the first display area.

[0052] In combination with the second aspect, in certain implementations of the second aspect, the sensing data includes the first voice signal, and the control unit is used to: control the first display area to display the second image based on characteristic information of the first voice signal; wherein the characteristic information of the first voice signal includes at least one of the pitch, timbre and tone of the first voice signal.

[0053] In combination with the second aspect, in certain implementations of the second aspect, the sensor data includes the first image, and the device further includes: a style transfer unit, configured to perform style transfer on the first image to obtain the second image.

[0054] In combination with the second aspect, in certain implementations of the second aspect, the sensor data includes the first image, and the device also includes: a determination unit, configured to determine text description content based on the first image, the text description content including at least one item of the scene, object, person, or environment corresponding to the first image; and determine the second image based on the text description content.

[0055] In combination with the second aspect, in certain implementations of the second aspect, the sensing data includes the first image, and the device further includes: an image generation unit, configured to input the first image into an image model to obtain the second image.

[0056] In combination with the second aspect, in certain implementations of the second aspect, the cabin also includes a second display area, and the control unit is further used to: when the detection unit detects a second voice signal of a second user in the cabin, control the second display area to display a third image based on the second voice signal and the second image, and the second display area is associated with the area where the second user is located.

[0057] In combination with the second aspect, in certain implementations of the second aspect, the control unit is used to: control the first display area to display the fourth image before the acquisition unit acquires the sensor data; and control the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and the sensor data.

[0058] In combination with the second aspect, in some implementations of the second aspect, the apparatus further includes: a connection establishing unit, configured to establish a connection with a terminal device; and a first sending unit, configured to send the second image to the terminal device.

[0059] In combination with the second aspect, in some implementations of the second aspect, the device further includes: a second sending unit, configured to send the second image to a cloud server.

[0060] In a third aspect, the present application provides a control device comprising a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the control device executes any possible method in the first aspect.

[0061] In a fourth aspect, the present application provides a control system, which includes a display device and a computing platform, and the computing platform includes any possible device in the second aspect or the third aspect mentioned above.

[0062] In some possible implementations, the control system further includes a sensor, which includes a camera and / or a microphone.

[0063] In a fifth aspect, the present application provides a vehicle comprising any possible device in the second aspect, or the device described in the third aspect, or the system described in the fourth aspect.

[0064] In some possible implementations, the vehicle is a vehicle.

[0065] In a sixth aspect, the present application provides a computer program product, comprising: a computer program code, which, when executed on a computer, enables the computer to execute any possible method in the first aspect.

[0066] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or separately from the processor, and this application does not make any specific restrictions on this.

[0067] In a seventh aspect, the present application provides a computer-readable medium storing a program code, which enables the computer to execute any possible method in the first aspect when the computer program code is run on the computer.

[0068] In an eighth aspect, the present application provides a chip system comprising a processor for calling a computer program or computer instruction stored in a memory so that the processor executes any possible method in the above-mentioned first aspect.

[0069] In combination with the eighth aspect, in a possible implementation, the processor is coupled to the memory through an interface.

[0070] In combination with the eighth aspect, in one possible implementation, the chip system also includes a memory, in which a computer program or computer instructions are stored. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] FIG1 is a schematic block diagram of a vehicle provided in an embodiment of the present application.

[0072] FIG2 is a schematic diagram of the distribution of display screens in a vehicle cabin according to an embodiment of the present application.

[0073] FIG3 is a set of graphic user interfaces GUI provided by an embodiment of the present application.

[0074] FIG4 is another set of GUIs provided in an embodiment of the present application.

[0075] FIG5 is another set of GUIs provided in an embodiment of the present application.

[0076] FIG6 is another set of GUIs provided in an embodiment of the present application.

[0077] FIG7 is another set of GUIs provided in an embodiment of the present application.

[0078] FIG8 is another set of GUIs provided in an embodiment of the present application.

[0079] FIG9 is another GUI provided by an embodiment of the present application.

[0080] FIG10 is a schematic flow chart of a control method provided in an embodiment of the present application.

[0081] FIG11 is a schematic block diagram of a control system provided in an embodiment of the present application.

[0082] FIG12 is a schematic block diagram of a control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a way to describe the association relationship of associated objects, indicating that there can be three kinds of relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more. For example, "at least one of A and B" is similar to "A and / or B", describing the association relationship of associated objects, indicating that there can be three kinds of relationships, for example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0084] In the embodiments of the present application, prefixes such as "first" and "second" are used only to distinguish different description objects and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present application does not constitute a restriction on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary restriction. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0085] FIG1 is a schematic block diagram of a vehicle 100 provided in an embodiment of the present application. The vehicle 100 may include a perception system 120, a display device 130, and a computing platform 150, wherein the perception system 120 may include one or more sensors for sensing information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a positioning system, which may be a global positioning system (GPS), a BeiDou system, or other positioning systems. The perception system 120 may also include one or more of an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and a camera device.

[0086] Some or all functions of vehicle 100 may be controlled by computing platform 150. Computing platform 150 may include one or more processors, such as processors 151 to 15n (n is a positive integer). A processor is a circuit capable of processing signals. In one implementation, the processor may be a circuit capable of reading and executing instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor may implement certain functions through the logical relationships of hardware circuits. The logical relationships of the hardware circuits may be fixed or reconfigurable. For example, the processor may be a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration file to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, the processor may also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n may call the instructions in the memory and execute the instructions to implement corresponding functions.

[0087] The display devices 130 in the cockpit are mainly divided into two categories: the first is the vehicle-mounted display screen; the second is a projection display screen, such as a head-up display (HUD). The vehicle-mounted display screen is a physical display screen and a key component of the in-vehicle infotainment system. The cockpit can be equipped with multiple displays, such as the digital instrument panel, the central control screen, the display in front of the front passenger (also known as the front passenger), the display in front of the left rear passenger, and the display in front of the right rear passenger. Even the vehicle windows can serve as display screens. A head-up display, also known as a head-up display system, is primarily used to display driving information such as speed and navigation on a display device in front of the driver (such as the windshield). This reduces the driver's gaze shift time, avoids pupil changes caused by the driver's gaze shift, and improves driving safety and comfort. HUDs include, for example, combiner-HUD (C-HUD), windshield-HUD (W-HUD), and augmented reality HUD (AR-HUD). It should be understood that other types of HUD systems may appear as technology evolves, and this application is not limited to this.

[0088] Figure 2 shows a schematic diagram of an exemplary display screen layout within a vehicle cabin, as provided in an embodiment of the present application. As shown in Figure 2 , the vehicle cabin may include display screen 201 (or, alternatively, a central control screen), display screen 202 (or, alternatively, a passenger entertainment screen), display screen 203 (or, alternatively, a second-row left entertainment screen), display screen 204 (or, alternatively, a second-row right entertainment screen), and an instrument panel.

[0089] It should be understood that the graphical user interface (GUI) in the following embodiments is described using the five-seater vehicle shown in FIG2 as an example, and the embodiments of the present application are not limited thereto. For example, for a seven-seater sport utility vehicle (SUV), the cockpit may include a central control screen, a co-pilot entertainment screen, an entertainment screen in the second row left area, an entertainment screen in the second row right area, an entertainment screen in the third row left area, and an entertainment screen in the third row right area. For another example, for a passenger car, the cockpit may include a front row entertainment screen and a rear row entertainment screen; or, the cockpit may include a display screen in the driving area and an entertainment screen in the passenger area. In one implementation, the entertainment screen in the passenger area may also be set on the top of the cockpit.

[0090] FIG3 is a set of GUIs provided in an embodiment of the present application.

[0091] As shown in FIG3( a ), the vehicle display screen 201 includes a content display area 301 and a function bar 302. The content display area 301 displays the user account avatar 3011, a Bluetooth icon 3012, a Wi-Fi icon 3013, a cellular network signal icon 3014, a wallpaper 3015, and a wallpaper change control 3016. The function bar 302 includes an icon 3021 for switching to the desktop, an icon 3022 for the vehicle's internal circulation system, an icon 3023 for the driver's seat heating function, an icon 3024 for the driver's area air conditioning temperature, an icon 3025 for the passenger seat air conditioning temperature, an icon 3026 for the passenger seat heating function, and a volume setting icon 3027.

[0092] As shown in (b) in Figure 3, when it is detected that the user clicks the control 3016, a prompt box 3017 can be displayed through the content display area 301, wherein the prompt box 3017 includes a control for starting the external camera 3018, a control for starting the internal camera, a control for starting the microphone, a control for secondary creation of the current wallpaper, and a cancel control.

[0093] As shown in (c) of FIG. 3 , when detecting that a user clicks on a control 3018 , the vehicle may activate an external camera and obtain an image 3019 captured by the external camera.

[0094] In one embodiment, the acquiring of the image 3019 captured by the camera outside the cockpit includes: acquiring one frame of image or multiple frames of image captured by the camera outside the cockpit.

[0095] As shown in (d) of FIG. 3 , after acquiring an image 3019 captured by a camera outside the cockpit, the vehicle may generate an image 3020 based on the image 3019 and display the image 3020 through the content display area 301 .

[0096] Exemplarily, the vehicle may generate image 3020 based on image 3019 , including: the vehicle may input image 3019 into an image to image model to obtain image 3020 .

[0097] For example, the vehicle can generate image 3020 based on image 3019 by inputting image 3019 into an image-to-text model to obtain textual description content corresponding to image 3019, such as the textual description content including at least one of a scene (e.g., winter, outdoors, mountain climbing, etc.), an object (e.g., white clouds, snow-capped mountains, etc.), a person (e.g., two climbers), or an environment (e.g., severe cold, clear weather, blue sky and white clouds, etc.). The textual description content corresponding to image 3019 is input into the text-to-image model to obtain image 3020.

[0098] In one embodiment, the image 3020 may be a photo, a moving picture, or a video.

[0099] In the above embodiment, the method is described by detecting that the user clicks control 3016 and selects control 3018 in prompt box 3017, thereby acquiring image 3019 captured by the camera outside the cabin, and determining image 3020 based on image 3019. The embodiments of the present application are not limited thereto.

[0100] For example, voice commands can be used to trigger the capture of images by cameras outside the cabin, which are then used to determine the final image displayed on the vehicle's display. For example, if a user issues the voice command "Xiao A Xiao A, create a wallpaper based on the scenery outside the car," the image captured by the camera outside the cabin can be obtained and a new wallpaper can be generated based on it.

[0101] For another example, the content display area may not include control 3016. The vehicle may periodically acquire images captured by the exterior camera while driving. For example, in the first cycle, image 1 captured by the exterior camera is input into the image-to-text model to obtain text description 1. This text description 1 is then input into the text-to-image model to obtain image 2, and the content display area 301 is controlled to display image 2. In the second cycle, image 3 captured by the exterior camera is input into the image-to-text model to obtain text description 2. If text description 2 differs from text description 1, this text description 2 may be input into the text-to-image model to obtain image 4, and the content display area 301 is controlled to switch from displaying image 2 to displaying image 4.

[0102] In this embodiment of the present application, the image captured by the camera outside the cabin can be used as a basis for secondary creation to obtain another image. In this way, the user can view the image created by using local materials on the vehicle display screen, which helps to enhance the user's visual experience and avoids the visual fatigue caused by the vehicle display screen showing a single and homogeneous wallpaper, thereby helping to improve the user's driving experience.

[0103] FIG4 is another set of GUIs provided in an embodiment of the present application.

[0104] As shown in (a) of FIG4 , the prompt box 3017 further includes a start-up in-car camera control 401. When a user clicks on the control 401, the in-cabin camera may be started and images captured by the in-cabin camera may be acquired.

[0105] Exemplarily, the in-cabin camera may include a camera of a driver monitor system (DMS) or a camera of a cabin monitor system (CMS).

[0106] As shown in FIG4( b ), the vehicle may obtain an image 402 captured by a camera in the cabin.

[0107] In one embodiment, the image 402 may include one or more frames of images.

[0108] As shown in (c) of FIG. 4 , after acquiring an image 402 captured by the camera in the cabin, the vehicle may generate an image 403 based on the image 402 and display the image 403 through the content display area 301 .

[0109] For example, the vehicle can generate image 403 based on image 402 by performing style transfer on image 402 to obtain image 403 in another style. For example, the other style includes traditional Chinese painting, comics, bold lines, watercolor, or oil painting. The style of image 403 can be bold lines.

[0110] In this embodiment of the present application, images captured by in-cabin cameras can be used as a basis for secondary creation to generate another image. This allows users to view images created using locally sourced material on the vehicle's display screen, enhancing their visual experience and avoiding the visual fatigue often associated with displaying a single, homogeneous wallpaper, thereby improving the user's driving experience. Furthermore, this approach addresses the issue of poor selfie or group photo quality resulting from a single in-cabin camera placement or a single in-cabin landscape background, improving in-cabin photography and enhancing user enjoyment.

[0111] The above describes the process of automatically generating a new image by performing style transfer on an image captured in the cockpit, in conjunction with the GUI shown in Figure 4. In an embodiment of the present application, after obtaining the image captured in the cockpit, the user can be prompted to select their preferred style, thereby allowing the user to participate in the creation of a new wallpaper.

[0112] FIG5 is another set of GUIs provided in an embodiment of the present application.

[0113] As shown in (a) in Figure 5, after the vehicle obtains the image 402 captured by the camera in the cabin, it can display a prompt box 501 through the content display area, wherein the prompt box includes the prompt information "The image in the cabin has been collected, please select one from the following styles", controls corresponding to the Chinese painting style, controls corresponding to the comic style, controls 502 corresponding to the watercolor style, controls corresponding to the oil painting style, and a cancel control.

[0114] The above is merely an illustration using traditional Chinese painting style, cartoon style, watercolor style, and oil painting style as examples, and the present application embodiment is not limited thereto. For example, the prompt box 501 may also include more or fewer styles.

[0115] As shown in (b) of FIG5 , when detecting a user clicking a control 502 , the vehicle may transfer the style of the image 402 captured by the in-cabin camera to a watercolor style, thereby generating an image 503 and displaying the image 503 through the content display area 301 .

[0116] In this embodiment of the present application, after acquiring an image captured by the camera, the user can be prompted to select their preferred style. Upon detecting that the user has selected a style, the style transfer can be performed on the image captured by the camera to generate a new image. By allowing the user to participate in the generation of the new image, the final new image displayed on the display screen can better meet the user's expectations, helping to enhance the user's visual experience and driving experience.

[0117] FIG6 is another set of GUIs provided in an embodiment of the present application.

[0118] As shown in FIG6(a), the prompt box 3017 also includes a start-up in-car camera control 601. When a user clicks on the control 601, the microphone in the cabin can be controlled to collect the voice command (or voice signal) issued by the user.

[0119] As shown in (b) of Figure 6, the vehicle receives the voice command "Xiao A Xiao A, help me replace the wallpaper with white clouds surrounding snow-capped mountains" through the microphone.

[0120] As shown in (c) of FIG. 6 , in response to acquiring the voice command, the vehicle may generate an image 602 according to the voice command and display the image 602 through the content display area 301 .

[0121] Exemplarily, the vehicle may generate the image 602 according to the voice command, including: the vehicle may generate the image 602 according to the voice command through a text to image model.

[0122] In this embodiment of the present application, after receiving a voice command captured by the microphone, a new image can be generated based on the voice command. In this way, the user can view the image created by the voice command issued by the user on the vehicle display screen, which helps to enhance the user's visual experience and avoids the visual fatigue caused by the vehicle display screen displaying a single and homogeneous wallpaper, thereby helping to improve the user's driving experience.

[0123] In one embodiment, the display screen in the cockpit may be controlled to display the image 602 according to the area where the user issuing the voice command is located.

[0124] For example, the vehicle can determine the area where the user who issued the voice command is located through the voice command collected by the microphone array in the cabin. For example, if the area where the user who issued the voice command is located is the passenger seat area, the image 602 can be displayed on the display screen 202.

[0125] In one embodiment, the vehicle may generate the image 602 according to the voice command, including: generating the image 602 according to feature information of the voice command, where the feature information includes at least one of pitch, timbre, and tone.

[0126] For example, the tone can be used to indicate whether the voice instruction is a rising tone or a falling tone, etc. Table 1 shows a correspondence between a tone and an image style.

[0127] Table 1

[0128] For example, when it is detected that the tone of the voice instruction is a rising tone, a fashion-style image may be generated according to the voice instruction.

[0129] For another example, when it is detected that the tone of the voice instruction is a falling tone, a black and white image can be generated according to the voice instruction.

[0130] For example, the timbre can be used to indicate whether the voice instruction is a male voice or a female voice, etc. Table 2 shows a correspondence between timbre and image style.

[0131] Table 2

[0132] For example, when it is detected that the timbre of the voice instruction is a male voice, an image in the style of traditional Chinese painting can be generated according to the voice instruction.

[0133] For another example, when it is detected that the timbre of the voice instruction is a female voice, a watercolor-style image can be generated according to the voice instruction.

[0134] In an embodiment of the present application, when the timbre of the voice command is detected to be male or female, the final image can be generated using an image style that matches the user's gender, so that the image displayed through the in-vehicle display screen is more in line with the user's gender characteristics, which helps to improve the user's driving experience.

[0135] For example, the timbre can also be used to indicate that the voice instruction is the voice of an old person or a child. Table 3 shows another correspondence between timbre and image style.

[0136] Table 3

[0137] For example, when it is detected that the timbre of the voice instruction is that of an elderly person, an image in the style of oil painting may be generated according to the voice instruction.

[0138] For another example, when it is detected that the timbre of the voice instruction is a child's voice, a comic-style image can be generated according to the voice instruction.

[0139] In an embodiment of the present application, when it is detected that the timbre of the voice command is the voice of an elderly person or a child, the final image can be generated using an image style that matches the user's age, so that the image displayed through the in-vehicle display screen is more in line with the user's age characteristics, which helps to improve the user's driving experience.

[0140] For example, the tone can be used to reflect the mood of the user when issuing the voice command (eg, happy or sad, etc.) Table 4 shows another correspondence between tone and image style.

[0141] Table 4

[0142] For example, when it is detected that the tone of the voice instruction is a happy tone, a fresh-style image can be generated according to the voice instruction.

[0143] For another example, when it is detected that the tone of the voice instruction is sad, a black and white image can be generated according to the voice instruction.

[0144] In an embodiment of the present application, the tone of the user's voice command can be used to reflect the user's mood. When it is detected that the tone of the voice command is happy or sad, the final image can be generated using an image style that matches the user's mood, so that the image displayed through the in-vehicle display screen is more in line with the user's current mood, which helps to improve the user's driving experience.

[0145] FIG7 is another set of GUIs provided in an embodiment of the present application.

[0146] As shown in FIG. 7( a ) , the vehicle may display an image 701 through the content display area 301 .

[0147] The above image 701 can be an image captured by the above-mentioned camera outside the cabin, an image captured by the camera inside the cabin, or an image generated by a voice command captured by a microphone, or the image 701 can be an image saved locally in the vehicle (for example, a wallpaper downloaded by the user through a wallpaper application).

[0148] As shown in (b) of FIG7 , the vehicle receives the voice command “Xiao A Xiao A, add an eagle to the picture” through the microphone.

[0149] As shown in (c) of FIG. 7 , in response to acquiring the voice command, the vehicle may generate the image 702 based on the image 701 and the voice command and display the image 702 through the content display area 301 .

[0150] In one embodiment, the image 702 may be generated based on the area where the user issuing the voice command is located. For example, if the area where the user issuing the voice command is located is the passenger seat area, an eagle may be added to the right area of ​​the image 701. For example, if the area where the user issuing the voice command is located is the driver seat area, an eagle may be added to the left area of ​​the image 701.

[0151] In the embodiment of the present application, users can use voice commands to edit or recreate images stored locally in the vehicle, or images determined by data collected by the camera or microphone. This facilitates user creation of images through voice commands, turning every user in the cabin into an image editing artist. This also ensures that the image presented to the user better meets their expectations, helping to enhance the user's driving experience.

[0152] The technical solution of the embodiment of the present application can also be applied to the scenario where the vehicle display screen is a long screen (or, it can also be called a long continuous screen). Figure 8 shows another set of GUIs provided by the embodiment of the present application.

[0153] As shown in Figure 8(a), the central control screen and the passenger screen in the vehicle's cockpit can be the same screen. This screen can be divided into two display areas: area 801 and area 802. Area 801 is the display area closest to the driver, while area 802 is the display area closest to the passenger. Area 801 displays the profile picture of user A in the driver's seat, along with the Bluetooth icon, Wi-Fi icon, cellular network signal icon, and wallpaper 803. Area 802 displays the profile picture of user B in the passenger seat, along with wallpaper 804.

[0154] When it is detected that user A in the main driving area issues a voice command 1 "Xiao A Xiao A, help me replace the wallpaper with white clouds surrounding snow-capped mountains", a GUI as shown in (b) of Figure 8 can be displayed.

[0155] As shown in FIG8(b), in response to receiving the voice command 1 issued by user A, an image 805 may be generated according to the voice command 1, and the control area 801 switches from displaying the wallpaper 802 to displaying the image 805. At this time, the area 802 may continue to display the wallpaper 804.

[0156] When it is detected that user B in the passenger seat area issues a voice command 2 “Xiao A Xiao A, add an eagle to the picture”, a GUI as shown in (c) of FIG. 8 may be displayed.

[0157] As shown in FIG8(c), in response to receiving the voice command 2 issued by user B, an image 806 may be generated based on the image 805 and the voice command 2, and the control area 802 may switch from displaying the wallpaper 804 to displaying the image 806. At this time, the area 801 may switch from displaying the image 805 to displaying the wallpaper 803.

[0158] In an embodiment of the present application, when a user B in the cockpit wishes to edit or re-create an image by issuing a voice command, the vehicle can display the edited or re-created image on a display area (for example, area 802) where the user B issuing the voice command is located, thereby facilitating user B to conveniently view the edited or re-created image, and helping to improve the user experience.

[0159] After a new image is generated by a voice command captured by a camera or microphone, the vehicle can also send the image to the terminal device. Figure 9 is another GUI provided by an embodiment of the present application.

[0160] As shown in FIG9 , the vehicle can display an image 901 via the content display area 301 . Image 901 can be an image generated based on data collected by the camera or microphone in the aforementioned embodiments. The vehicle can send image 901 to the mobile phone. In response to receiving image 901 from the vehicle, the mobile phone can display a prompt box, which includes the prompt message "Photo received from the vehicle," a thumbnail 902 of image 901, a Set as mobile phone wallpaper control 903, a Save to album control 904, and an Ignore control 905.

[0161] In one embodiment, the vehicle can send image 901 to the mobile phone, including: the vehicle can send a thumbnail of image 901 to the mobile phone. In this way, by sending the thumbnail information of image 901 to the mobile phone, the time delay when the vehicle sends the original image content of image 901 to the mobile phone is reduced, and the speed at which the user can view the image generated by the vehicle on the mobile phone is increased, which helps to improve the user experience. At the same time, it also reduces the transmission resource overhead caused when the user does not want to save image 901 on the mobile phone.

[0162] In one embodiment, when detecting that the user clicks on the control 903 , the mobile phone may replace the wallpaper with the image 901 .

[0163] In one embodiment, when the mobile phone detects the user clicking on control 903, the mobile phone can replace the wallpaper with image 901, including: when the user clicks on control 903, the mobile phone can request the original image content of image 901 from the vehicle; in response to receiving the request sent by the mobile phone, the vehicle can send the original image content of image 901 to the mobile phone; in response to receiving the original image content of image 901 sent by the vehicle, the mobile phone can set the original image content of image 901 as the wallpaper of the mobile phone.

[0164] In one embodiment, when detecting that the user clicks the control 904 , the mobile phone may save the image 901 in the mobile phone's photo album.

[0165] In one embodiment, when the mobile phone detects the user clicking on the control 904, the mobile phone can save the image 901 in the mobile phone's album, including: when the user clicks on the control 904, the mobile phone can request the vehicle for the original image content of the image 901; in response to receiving the request sent by the mobile phone, the vehicle can send the original image content of the image 901 to the mobile phone; in response to receiving the original image content of the image 901 sent by the vehicle, the mobile phone can save the original image content of the image 901 in the mobile phone's album.

[0166] In one embodiment, the vehicle and the mobile phone can establish a connection via short-range communication technology, including but not limited to Bluetooth connection or Wi-Fi connection, etc. In this way, when the vehicle generates the image 901, the image 901 can be sent to the mobile phone via the short-range communication technology.

[0167] In one embodiment, before the vehicle transmits image 901 to the mobile phone via short-range communication technology, it can also determine whether the account logged into the vehicle is associated with the account logged into the mobile phone. For example, if the account logged into the vehicle and the account logged into the mobile phone are the same, image 901 can be sent to the mobile phone; or, if the account logged into the vehicle and the account logged into the mobile phone are in the same family group, image 901 can be sent to the mobile phone; or, if the account logged into the mobile phone is an account authorized by the vehicle, image 901 can be sent to the mobile phone. This can avoid causing interference to strangers when image 901 is sent to their mobile phones.

[0168] In one embodiment, the vehicle and the mobile phone can establish a connection through a cloud server. The vehicle can upload the image 901 to the cloud server, and the cloud server can be a server for users in a family group to share information.

[0169] For example, the account logged into the vehicle is account 1, and account 1 and account 2 are in the same family group. When the vehicle generates image 901, it can upload image 901 to the cloud server. The cloud server can send image 901 to other devices under account 1 except vehicle 1 and send image 901 to devices under account 2.

[0170] In one embodiment, when the vehicle generates the image 901, the image 901 can also be uploaded to the server corresponding to the wallpaper mall. In this way, other users can view the image 901 through the wallpaper application, making it convenient for other users to download the image 901 through the wallpaper application.

[0171] FIG10 shows a schematic flow chart of a control method 1000 provided in an embodiment of the present application. The method 1000 can be executed by a vehicle (e.g., a vehicle), or the method 1000 can be executed by the above-mentioned computing platform, or the method 1000 can be executed by a system consisting of a computing platform and a sensor, or the method 1000 can be executed by a system-on-a-chip (SoC) in the above-mentioned computing platform, or the method 1000 can be executed by a processor in the computing platform. The method 1000 includes:

[0172] S1010, when detecting an input of a first user instructing to generate an image in the cabin, acquiring sensor data, the sensor data including a first image captured by a camera of the vehicle or a first voice signal captured by a microphone of the vehicle.

[0173] Exemplarily, as shown in (b) in FIG3 , the first user-indicated input for generating an image may be an operation of clicking control 3018 .

[0174] Exemplarily, as shown in FIG4( a ), the first user's input for indicating generation of an image may be an operation of clicking a control 401 .

[0175] Exemplarily, as shown in FIG6( a ), the first user's input for indicating generation of an image may be an operation of clicking a control 601 .

[0176] For example, as shown in (a) of FIG8 , the first user-indicated input for generating an image may be a user issuing a voice instruction 1 “Xiao A Xiao A, help me replace the wallpaper with one of white clouds surrounding snow-capped mountains”.

[0177] Optionally, the camera of the vehicle includes a camera outside the vehicle cabin or a camera inside the cabin.

[0178] S1020: Control the first display area to display a second image according to the sensing data.

[0179] Optionally, the sensor data includes the first voice signal, and controlling the first display area to display the second image based on the sensor data includes: controlling the first display area to display the second image based on characteristic information of the first voice signal; wherein the characteristic information of the first voice signal includes at least one of the pitch, timbre and tone of the first voice signal.

[0180] Exemplarily, the vehicle can save the correspondence between the characteristic information of the voice signal and the image style, and control the first display area to display the second image based on the characteristic information of the first voice signal, including: controlling the first display area to display the second image based on the characteristic information of the first voice signal and the correspondence.

[0181] Illustratively, the corresponding relationship may be as shown in Table 1, Table 2, Table 3 or Table 4 above.

[0182] Optionally, the sensor data includes the first image, and the method further includes: performing style migration on the first image to obtain the second image.

[0183] For example, as shown in (c) of FIG4 , the first image may be image 402, and the second image may be image 403. Image 403 may be obtained by performing style transfer on image 402.

[0184] Optionally, the sensor data includes the first image, and the method further includes: determining text description content based on the first image, the text description content including at least one item of the scene, object, person or environment corresponding to the first image; and determining the second image based on the text description content.

[0185] For example, as shown in (c) and (d) of FIG3 , the first image may be image 3019, and the second image may be image 3020. The vehicle may input image 3019 into an image-to-text model to obtain a text description corresponding to image 3019. For example, the scene corresponding to the first image is "winter" and "outdoors," the objects included in the first image are "snow-capped mountains" and "white clouds," and the environment in the first image is "clear weather." The vehicle may input the text description corresponding to image 3019 into the text-to-image model to obtain image 3020.

[0186] Optionally, the sensor data includes the first image, and the method further includes: inputting the first image into an image to image model to obtain the second image.

[0187] 3( c ) and ( d ), the first image may be image 3019 , and the second image may be image 3020 . The vehicle may input image 3019 into an image to image model to obtain image 3020 .

[0188] Optionally, the cabin also includes a second display area, and the method also includes: when a second voice signal of a second user in the cabin is detected, controlling the second display area to display a third image based on the second voice signal and the second image, and the second display area is associated with the area where the second user is located.

[0189] For example, as shown in FIG8(c), the first display area may be area 801, and the second display area may be area 802. Upon detecting voice command 2 sent by user B in the passenger seat area, the vehicle may generate image 806 based on voice command 2 and image 805 and control area 802 to display image 806.

[0190] Optionally, before obtaining the sensor data, the method also includes: controlling the first display area to display a fourth image; wherein, controlling the first display area to display the second image based on the sensor data includes: controlling the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and the sensor data.

[0191] For example, as shown in (a)-(c) of FIG7 , the fourth image may be image 701, the second image may be image 702, and the sensor data may be a user's voice command "Xiao A Xiao A, add an eagle to the picture." The vehicle may generate image 702 based on image 701 and the voice command and control content display area 301 to switch from displaying image 701 to displaying image 702.

[0192] Optionally, the method 1000 further includes: establishing a connection with a terminal device; and sending the second image to the terminal device.

[0193] For example, as shown in FIG9 , after the vehicle and the mobile phone establish a connection, the vehicle may send the generated image 902 to the mobile phone.

[0194] Optionally, sending the second image to the terminal device includes: sending a thumbnail of the second image to the terminal device.

[0195] Optionally, before sending the second image to the terminal device, the method 1000 further includes: determining that the account logged in on the vehicle is associated with the account logged in on the terminal device.

[0196] In this way, the second image is sent after it is determined that the account on the vehicle is associated with the account logged in on the terminal device, which can avoid the interference caused by the second image to unfamiliar passengers in the cabin.

[0197] Optionally, the method 1000 further includes: sending the second image to a cloud server.

[0198] Exemplarily, the cloud server may be a server for users in a family group to share information, or the cloud server may be a server corresponding to a wallpaper application.

[0199] Figure 11 shows a schematic block diagram of a control system 1100 provided in an embodiment of the present application. The control system 1100 includes an image acquisition device 1101, an audio acquisition device 1102, an image style transfer model 1103, an image-to-text model 1104, a text-to-image model 1105, an image-to-image model 1106, an automatic speech recognition (ASR) module 1107, and a speech-to-image model 1108.

[0200] The image acquisition device 1101 is used to acquire images captured by the in-cabin camera and / or the out-cabin camera. The image acquisition device 1101 can send the acquired images to the image style transfer model 1103, the image to text model 1104, or the image to image model 1106.

[0201] The audio collection device 1102 is used to obtain the voice signal collected by the microphone and send the obtained voice signal to the speech recognition model 1107.

[0202] The image style transfer model 1103 is used to perform style transfer on the image acquired by the image acquisition device 1101, thereby obtaining an image after style transfer.

[0203] Image-to-text model 1104 is configured to obtain a corresponding text description of an image captured by image acquisition device 1101. Exemplarily, the text description includes at least one of the scene, object, person, or environment corresponding to the image. Image-to-text model 1104 is further configured to send the determined text description to text-to-image model 1105.

[0204] The text to image model 1105 is used to generate a new image based on the text description content sent by the image to text model 1104.

[0205] The image to image model 1106 is used to generate a new image based on the image acquired by the image acquisition device 1101.

[0206] The ASR module 1107 is used to analyze the voice signal acquired by the audio acquisition device 1102 and identify the text content corresponding to the voice signal. The ASR module 1107 is also used to send the identified text content to the speech-to-image model 1108.

[0207] The speech-to-image model 1108 is used to generate an image based on the text content sent by the ASR module 1107.

[0208] Optionally, the speech-to-image model 1108 and the text-to-image model 1105 may be the same model, or may be different models.

[0209] Optionally, the system 1100 may further include a multi-stream model for acquiring image information, including images generated by the image style transfer model 1103, images generated by the text-to-image model 1105, images generated by the image-to-image model 1106, images generated by the speech-to-image model 1108, or images originally displayed on the vehicle display screen. The multi-stream model 1109 is further configured to receive text content sent by the ASR module 1107. The multi-stream model can generate a new image based on the text content and the acquired image information.

[0210] Optionally, the system 1100 can send images generated by the text-to-image model 1105, the image-to-image model 1106, the speech-to-image model 1108, or the multi-stream model to the vehicle desktop application. The vehicle desktop application controls the vehicle display screen to display the images.

[0211] Optionally, the system 1100 can send images generated by the image style transfer model 1103, the text-to-image model 1105, the image-to-image model 1106, the speech-to-image model 1108, or the multi-stream model 1109 to an in-vehicle application, a terminal device (e.g., a mobile phone), or a cloud server. For example, the cloud server can be a server used for information sharing among users in a family group, or the cloud server can be a server for a wallpaper application.

[0212] Figure 12 shows a schematic block diagram of a control device 1200 provided in an embodiment of the present application. As shown in Figure 12, the device 1200 includes a detection unit 1210, an acquisition unit 1220, and a control unit 1230. The acquisition unit 1220 is configured to acquire sensor data, including a first image captured by a camera of the vehicle or a first voice signal captured by a microphone of the vehicle, when the detection unit 1210 detects input indicating that a first user in the vehicle's cabin is generating an image. The control unit 1230 is configured to control a first display area to display a second image based on the sensor data. The cabin includes the first display area.

[0213] Optionally, the sensing data includes the first voice signal, and the control unit 1230 is used to: control the first display area to display the second image according to characteristic information of the first voice signal; wherein the characteristic information of the first voice signal includes at least one of the pitch, timbre and tone of the first voice signal.

[0214] Optionally, the sensor data includes the first image, and the apparatus 1200 further includes: a style transfer unit, configured to perform style transfer on the first image to obtain the second image.

[0215] Exemplarily, the functions implemented by the style transfer unit can be implemented by the image style transfer model 1103 in the above-mentioned system 1100.

[0216] Optionally, the sensor data includes the first image, and the device 1200 further includes: a determination unit, configured to determine text description content based on the first image, the text description content including at least one item of the scene, object, person or environment corresponding to the first image; and determine the second image based on the text description content.

[0217] Optionally, the functions implemented by the determination unit may be implemented by the image to text model 1104 and the text to image model 1105 in the above-mentioned system 1100 .

[0218] Optionally, the sensing data includes the first image, and the apparatus 1200 further includes: an image generating unit, configured to input the first image into an image model to obtain the second image.

[0219] Exemplarily, the functions implemented by the image generation unit can be implemented by the image to image model 1106 in the above-mentioned system 1100.

[0220] Optionally, the cabin also includes a second display area, and the control unit 1230 is further used to: when the detection unit 1210 detects a second voice signal of a second user in the cabin, control the second display area to display a third image based on the second voice signal and the second image, and the second display area is associated with the area where the second user is located.

[0221] Optionally, the control unit 1230 is used to: control the first display area to display the fourth image before the acquisition unit acquires the sensor data; and control the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and the sensor data.

[0222] Optionally, the apparatus 1200 further includes: a connection establishing unit, configured to establish a connection with a terminal device; and a first sending unit, configured to send the second image to the terminal device.

[0223] Optionally, the device 1200 further includes: a second sending unit, configured to send the second image to a cloud server.

[0224] For example, the detection unit 1210 may be the computing platform in FIG1 , or a processing circuit, processor, or controller within the computing platform. Taking the detection unit 1210 as the processor 151 within the computing platform, the processor 151 may obtain data collected by a touch sensor on an in-vehicle display screen and, based on the data, determine that an input indicating the generation of an image by a first user within the cabin has been detected.

[0225] For another example, acquisition unit 1220 may be the computing platform shown in Figure 1 or a processing circuit, processor, or controller within the computing platform. For example, if acquisition unit 1220 is processor 152 within the computing platform, processor 152 may acquire sensor data when processor 151 determines that an input indicating image generation is detected by a first user within the cabin. For example, processor 152 may control a camera outside the cabin to activate and acquire a first image captured by the camera. Alternatively, processor 152 may control a microphone within the cabin to activate and acquire a voice signal captured by the microphone.

[0226] For another example, the control unit 1230 may be the computing platform in FIG1 or a processing circuit, processor, or controller in the computing platform. For example, if the control unit 1230 is the processor 153 in the computing platform, the processor 153 may generate a second image based on the sensor data. For example, the processor 153 may store the aforementioned image style transfer model 1103; or the processor 153 may store the aforementioned image to text model 1104 and text to image model 1105; or the processor 153 may store the aforementioned image to image model 1106; or the processor 153 may store the aforementioned ASR module 1107 and speech to image model 1108.

[0227] The functions implemented by the above detection unit 1210, the functions implemented by the acquisition unit 1220, and the functions implemented by the control unit 1230 can be implemented by different processors, or some functions can be implemented by the same processor, or they can also be implemented by the same processor. The embodiments of the present application do not limit this.

[0228] It should be understood that the division of the various units in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. Furthermore, the units in the device may be implemented in the form of a processor calling software; for example, the device includes a processor connected to a memory storing instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or the functions of the various units in the device, where the processor is, for example, a general-purpose processor such as a CPU or a microprocessor, and the memory is a memory within the device or a memory external to the device. Alternatively, the units in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented through the design of the hardware circuits. The hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units may be implemented through the design of the logical relationships between the components within the circuits. In another implementation, the hardware circuit may be implemented using a PLD, such as an FPGA, which may include a large number of logic gate circuits, and the connections between the logic gate circuits may be configured using a configuration file to implement the functions of some or all of the above units. All units of the above apparatus may be implemented entirely in the form of software called by a processor, or entirely in the form of hardware circuits, or partially in the form of software called by a processor and the rest in the form of hardware circuits.

[0229] Each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0230] In addition, the various units in the above apparatus may be fully or partially integrated together, or may be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the apparatus. The at least one processor may be of different types, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0231] An embodiment of the present application also provides a control device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to execute the control method executed by the above embodiment.

[0232] Alternatively, if the device is located in a vehicle, the processing unit may be the processors 151 - 15n shown in FIG. 1 .

[0233] An embodiment of the present application further provides a control system, which may include a computing platform and a display device, and the computing platform may include the above-mentioned control device 1200.

[0234] Exemplarily, the display device may include an in-vehicle display screen, such as one or more of the display screen 201 , the display screen 202 , the display screen 203 or the display screen 204 in FIG. 2 .

[0235] Optionally, the vehicle may also include sensors, such as a camera or a microphone.

[0236] An embodiment of the present application also provides a vehicle, which may include the above-mentioned control device 1200 or control system.

[0237] An embodiment of the present application further provides a computer program product, which includes: computer program code, which, when executed on a computer, enables the computer to execute the control method in the above embodiment.

[0238] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable medium stores a program code. When the computer program code is run on a computer, the computer executes the control method in the above embodiment.

[0239] An embodiment of the present application further provides a chip, which includes a circuit, and the circuit is used to execute the control method in the above embodiment.

[0240] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or a power-on erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0241] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0242] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0243] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0244] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0245] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0246] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0247] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0248] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be covered and fall within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A control method, characterized in that: The method is applied to a cockpit of a vehicle, wherein the cockpit includes a first display area, and the method includes: When an input indicating generation of an image by a first user in the cockpit is detected, sensor data is acquired, the sensor data including a first image acquired by a camera of the vehicle or a first voice signal acquired by a microphone of the vehicle; According to the sensing data, the first display area is controlled to display a second image.

2. The method according to claim 1, characterized in that The sensor data includes the first voice signal, and controlling the first display area to display the second image according to the sensor data includes: controlling the first display area to display the second image according to the feature information of the first voice signal; The characteristic information of the first speech signal includes at least one of the pitch, timbre and tone of the first speech signal.

3. The method according to claim 1, characterized in that The sensor data includes the first image, and the method further includes: Perform style transfer on the first image to obtain the second image.

4. The method according to claim 1, characterized in that The sensor data includes the first image, and the method further includes: Determining text description content according to the first image, the text description content including at least one of a scene, an object, a person, or an environment corresponding to the first image; The second image is determined according to the text description content.

5. The method according to claim 1, characterized in that The sensor data includes the first image, and the method further includes: The first image is input into an image model to obtain the second image.

6. The method according to any one of claims 1 to 5, characterized in that The cockpit also includes a second display area, and the method further includes: When a second voice signal of a second user in the cabin is detected, the second display area is controlled to display a third image according to the second voice signal and the second image, and the second display area is associated with an area where the second user is located.

7. The method according to any one of claims 1 to 6, characterized in that Before acquiring the sensor data, the method further includes: Controlling the first display area to display a fourth image; Wherein, controlling the first display area to display the second image according to the sensing data includes: According to the fourth image and the sensing data, the first display area is controlled to switch from displaying the fourth image to displaying the second image.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Establish a connection with the terminal device; The second image is sent to the terminal device.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: The second image is sent to the cloud server.

10. A control device, characterized in that: The device comprises a detection unit, an acquisition unit and a control unit, wherein: The acquisition unit is used to acquire sensor data when the detection unit detects an input of a first user instruction to generate an image in the cabin of the vehicle, wherein the sensor data includes a first image acquired by a camera of the vehicle or a first voice signal acquired by a microphone of the vehicle; A control unit is used to control a first display area to display a second image according to the sensing data, and the cockpit includes the first display area.

11. The device according to claim 10, characterized in that The sensor data includes the first voice signal, and the control unit is used to: controlling the first display area to display the second image according to the feature information of the first voice signal; The characteristic information of the first speech signal includes at least one of the pitch, timbre and tone of the first speech signal.

12. The device according to claim 10, characterized in that The sensor data includes the first image, and the apparatus further includes: A style transfer unit is used to perform style transfer on the first image to obtain the second image.

13. The device according to claim 10, characterized in that The sensor data includes the first image, and the apparatus further includes: a determining unit, configured to determine text description content according to the first image, wherein the text description content includes at least one of a scene, an object, a person, or an environment corresponding to the first image; The second image is determined according to the text description content.

14. The device according to claim 10, characterized in that The sensor data includes the first image, and the apparatus further includes: The image generating unit is used to input the first image into the image model to obtain the second image.

15. The device according to any one of claims 10 to 14, characterized in that The cockpit also includes a second display area, and the control unit is further used for: When the detection unit detects a second voice signal of a second user in the cabin, the second display area is controlled to display a third image according to the second voice signal and the second image, and the second display area is associated with an area where the second user is located.

16. The device according to any one of claims 10 to 15, characterized in that The control unit is used for: Before the acquisition unit acquires the sensor data, controlling the first display area to display a fourth image; According to the fourth image and the sensing data, the first display area is controlled to switch from displaying the fourth image to displaying the second image.

17. The device according to any one of claims 10 to 16, characterized in that The device also includes: A connection establishing unit, used for establishing a connection with a terminal device; The first sending unit is used to send the second image to the terminal device.

18. The device according to any one of claims 10 to 17, characterized in that The device also includes: The second sending unit is used to send the second image to the cloud server.

19. A control device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program stored in the memory so that the device performs the control method according to any one of claims 1 to 9.

20. A control system, characterized in that: It comprises a display device and a computing platform, wherein the computing platform comprises a control device as claimed in any one of claims 10 to 19.

21. A vehicle, characterized in that: The method comprises a control device according to any one of claims 10 to 19, or a control system according to claim 20.

22. The vehicle according to claim 21, characterized in that The vehicle is a vehicle.

23. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the control method according to any one of claims 1 to 9 is implemented.

24. A chip, characterized in that: The chip includes a processor and a data interface, and the processor reads instructions stored in a memory through the data interface to execute the control method according to any one of claims 1 to 9.

Images