Control methods, control devices, means of transport

Abstract

Embodiments of this application provide a control method and apparatus, as well as a means of transport. The method may be applied in the field of intelligent cockpits. The method includes acquiring sensor data when input from a user in the cockpit is detected to instruct the generation of an image, the sensor data including a first image acquired by a camera of the means of transport or an audio signal acquired by a microphone of the means of transport, and controlling a display area in the means of transport to display a second image based on the sensor data. Embodiments of this application can be applied to intelligent vehicles or electric vehicles to help improve the degree of personalization of content displayed in the cockpit and to improve the user experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent cockpits, and more specifically, to control methods, devices, and means of transportation.

Background Art

[0002] Currently, there are increasingly more displays in the vehicle cockpit, and the content (e.g., wallpaper) displayed on the displays is becoming increasingly uniform. How to improve the degree of personalization of the content displayed on the displays has become an urgent problem to be solved.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present application provides a control method and device, as well as a means of transportation, which are useful for improving the degree of personalization of the content displayed in the cockpit and useful for improving the user experience.

[0004] The means of transportation in the present application may include road means of transportation, water means of transportation, air means of transportation, industrial devices, agricultural devices, entertainment devices, etc. For example, the means of transportation may be a vehicle. The vehicle is a vehicle in a broad sense and may be a means of transportation (tool) (e.g., commercial vehicle, passenger car, motorcycle, aircraft, train, etc.), an industrial vehicle (e.g., forklift, trailer, tractor, etc.), a work vehicle (e.g., shovel, bulldozer, crane, etc.), an agricultural device (e.g., lawn mower, harvester, etc.), an amusement device, a toy vehicle, etc. The type of vehicle is not particularly limited in the embodiments of the present application. As another example, the means of transportation may be a means of transportation such as an aircraft or a ship.

Means for Solving the Problems

[0005] According to a first aspect, the present application provides a control method which is applied to the cockpit of a means of transport, the cockpit includes a first display area, the method includes acquiring sensor data when input from a first user in the cockpit is detected to instruct the generating of an image, the sensor data including a first image acquired by a camera of the means of transport or a first audio signal acquired by a microphone of the means of transport, and controlling the first display area to display a second image based on the sensor data.

[0006] Based on the above technical solution, the vehicle can control the first display area to display a second image by recreating data acquired by a camera or microphone. In this way, the user can see an image created based on locally acquired content in the display area within the cockpit, thereby avoiding the visual fatigue caused to the user by displaying a single, homogeneous image in the display area, improving the user's visual perception and the degree of personalization of the content displayed in the cockpit, and improving the user's driving experience.

[0007] If the first image includes an image acquired by a camera inside the cockpit, it can solve the problem of poor selfie or group photo effects due to the placement of a single camera inside the cockpit or a single background scene inside the cockpit, and as a result, it can improve the shooting effect inside the cockpit and the user's enjoyment of taking photos.

[0008] In some possible embodiments, the first display area is a display area on a display in the cockpit of a means of transport. For example, if the means of transport is a vehicle, the first display area may be a display area on a central display in the vehicle cockpit.

[0009] In some possible implementations, the area where the first user is located is associated with the first display area.

[0010] For example, the mode of transportation is a vehicle. If the first user is located in the driver's area, the first display area may be a display area on the central display, or the first display area may be on the central display and close to the driver's area.

[0011] In some possible embodiments, detecting input from a first user in the cockpit to instruct the system to generate an image includes detecting a voice command given by the first user, the voice command instructing the system to generate an image.

[0012] In some possible embodiments, the method further includes controlling a first display area to display a first interface, which includes controls, which are controls for generating an image, before an input from a first user in the cockpit is detected to instruct the generation of an image, and the detection of an input from a first user in the cockpit to instruct the generation of an image includes detecting an operation by the user of tapping the controls.

[0013] In some possible implementations, the method further includes setting a second image as the wallpaper in the first display area.

[0014] In some possible embodiments, controlling the first display area to display the second image based on the sensor data includes generating the second image based on the sensor data and controlling the first display area to display the second image.

[0015] In some possible embodiments, the first image may be an image of one or more frames.

[0016] In some possible implementations, the second image may be a photograph (e.g., wallpaper), an animation (e.g., a graphics interchange format (GIF) animation), a video, or the like.

[0017] Referring to the first embodiment, in some implementations of the first embodiment, the sensor data includes a first audio signal, and controlling the first display area to display a second image based on the sensor data includes controlling the first display area to display a second image based on feature information of the first audio signal, wherein the feature information of the first audio signal includes at least one of the pitch, timbre, and tone of the first audio signal.

[0018] Based on the above proposed technology, after acquiring an audio signal via a microphone, a second image can be acquired based on the characteristic information of the audio signal. In this way, the user can see an image created based on the characteristic information of the audio signal provided by the user in the display area within the cockpit. As a result, visual fatigue caused to the user by displaying a single, homogeneous image in the display area can be avoided, improving the user's visual perception and the degree of personalization of the content displayed in the cockpit, thereby improving the user's driving experience.

[0019] Furthermore, based on the characteristic information of the audio signal, the final generated image can better match the user's expectations. For example, if the tone of the audio signal is detected to be that of an elderly person or a child, the final image can be generated based on an image style that matches the user's age. This allows the image displayed in the cockpit display area to better suit the user's age characteristics and improve the user experience.

[0020] Alternatively, if the tone of the audio signal is detected to be happy or sad, the final image may be generated based on an image style that matches the user's mood. As a result, the image displayed in the cockpit display area may better suit the user's current mood and improve the user's driving experience.

[0021] In some possible embodiments, controlling the first display area to display the second image based on the characteristic information of the first audio signal includes generating the second image based on the characteristic information of the first audio signal and controlling the first display area to display the second image.

[0022] Referring to the first aspect, in some implementations of the first aspect, the sensor data includes a first image, and the method further includes performing a style transformation on the first image in order to obtain a second image.

[0023] Based on the above technical solution, the second image may be obtained by performing a style transformation on the image acquired by the camera, which can improve the user's visual perception, avoid the visual fatigue caused to the user by displaying a single, homogeneous wallpaper on the in-car display, and improve the user's driving experience.

[0024] Furthermore, if the camera is an in-cockpit camera, it can solve the problem of poor selfie or group photo results due to a single camera placement position or a single background scene within the cockpit, thereby improving the shooting effect within the cockpit and the user's enjoyment of photography.

[0025] Referring to the first aspect, in some implementations of the first aspect, the sensor data includes a first image, and the method is to determine text description content based on the first image, where the text description content includes at least one of a scene, an object, a person, or an environment corresponding to the first image, and further includes determining a second image based on the text description content.

[0026] Based on the above technical solution, the text description content corresponding to the first image may be first determined based on the first image, and then the second image is generated based on the text description content. In this way, by reconstructing the image acquired by the camera, it is possible to avoid visual fatigue caused to the user by displaying a single and homogeneous image in the display area, improve the degree of personalization of the user's visual perception and the content displayed in the cockpit, and improve the user's driving experience.

[0027] In some possible embodiments, determining text description content based on the first image includes inputting the first image into an image-to-text model to obtain the text description content.

[0028] In some possible implementations, determining the second image based on the text description content includes inputting the text description content into a text-to-image model to obtain the second image.

[0029] In some possible embodiments, when an input is detected that instructs to generate an image from a first user in the cockpit, the vehicle may periodically obtain an image acquired by a camera outside the cockpit.

[0030] For example, during a first period, a first image is acquired by a camera outside the cockpit, and a first text description content corresponding to the first image is acquired based on the first image. In this way, the first display area is controlled to display a second image based on the first text description content.

[0031] In another example, during the second period, a fifth image is acquired by a camera outside the cockpit, and a second text description content corresponding to the fifth image is acquired based on the fifth image. In this way, the first display area is controlled to display a sixth image based on the second text description content, which is different from the second text description content.

[0032] In this way, after the external camera of the cockpit acquires images of different text description content, the image displayed in the first display area can be updated in a timely manner, thereby further improving the user's visual experience and avoiding the visual fatigue caused to the user by displaying a single, uniform wallpaper in the display area of ​​the vehicle. In addition, the content displayed in the first display area always matches the scenery outside the cockpit to help improve the user's driving experience.

[0033] Referring to the first aspect, in some implementations of the first aspect, the sensor data includes a first image, and the method further includes inputting the first image into an image-to-image model in order to obtain a second image.

[0034] Based on the technical solution described above, a second image can be obtained by inputting the image acquired by the camera into an image-to-image model to recreate the image acquired by the camera. In this way, it is possible to avoid the visual fatigue caused to the user by displaying a single, homogeneous image in the display area, improve the user's visual perception and the degree of personalization of the content displayed in the cockpit, and improve the user's driving experience.

[0035] Referring to the first embodiment, in some implementations of the first embodiment, the cockpit further includes a second display area, the method further includes controlling the second display area to display a third image based on the second audio signal and a second image when a second audio signal is detected from a second user in the cockpit, the second display area being associated with an area where the second user is located.

[0036] Based on the above proposed technology, if it is detected that a second user in the cockpit has emitted a second audio signal, a third image can be generated based on the second image and the second audio signal. The transport may also switch from displaying the second image in the first display area to displaying the third image in the second display area, thereby allowing the second user to see the updated image in the display area where the second user is located, improving the user experience.

[0037] For example, the mode of transportation is a vehicle. If the second user is located in the passenger seat area, the second display area could be the display area on the passenger seat entertainment screen, or the second display area could be on the central display and be close to the passenger seat area.

[0038] In some possible embodiments, detecting a second audio signal from a second user in the cockpit includes detecting a second audio signal from a second user in the cockpit after the first display area has been controlled to display the second image.

[0039] In some possible embodiments, a second audio signal is used to modify or recreate a second image.

[0040] Referring to the first aspect, in some implementations of the first aspect, the method further includes controlling a first display area to display a fourth image before acquiring sensor data, controlling the first display area to display a second image based on the sensor data, and controlling the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and sensor data.

[0041] Based on the aforementioned technical solution, the second image can be initially generated based on sensor data and the original image displayed in the display area. Furthermore, the means of transport (e.g., a vehicle) can automatically update the wallpaper displayed in the first display area by switching from displaying the fourth image to displaying the second image in the first display area. In this way, visual fatigue caused to the user by displaying a single, homogeneous image in the display area can be avoided, improving the user's visual perception and the degree of personalization of the content displayed in the cockpit, thereby improving the user's driving experience.

[0042] Referring to the first embodiment, in some implementations of the first embodiment, the method further includes establishing a connection to a terminal device and transmitting a second image to the terminal device.

[0043] Based on the above technical solution, after a connection is established between the transport and the terminal device, the transport can further transmit the generated second image to the terminal device. As a result, the user can view the image generated by the transport on the terminal device, store the image locally on the terminal device in a timely manner, share the image with other users via the terminal device, or continue to create images based on local map restoration software on the terminal device, thereby improving the user experience.

[0044] In some possible implementations, transmitting a second image to a terminal device includes transmitting the second image to the terminal device using short-range communication technology, or transmitting the second image to the terminal device via a server.

[0045] In some possible embodiments, before transmitting a second image to the terminal device, the method further includes determining that the account logged into the transportation is associated with the account logged into the terminal device.

[0046] For example, an account logged into a transportation service can be associated with an account logged into a terminal device if the account logged into the transportation service is the same as the account logged into the terminal device, if the account logged into the transportation service and the account logged into the terminal device belong to the same family group, or if the account logged into the terminal device is an account authorized by the transportation service.

[0047] Referring to the first aspect, in some implementations of the first aspect, the method further includes sending a second image to a cloud server.

[0048] For example, the cloud server could be a server that supports the wallpaper application.

[0049] Based on the aforementioned technical solution, the transportation device sends information about the second image to a cloud server, which allows another user to download the second image via a wallpaper application.

[0050] According to a second aspect, the present application provides a control device comprising a detection unit, an acquisition unit, and a control unit. The acquisition unit is configured to acquire sensor data when the detection unit detects input from a first user in the cockpit of a means of transport to instruct it to generate an image, the sensor data comprising a first image acquired by a camera of the means of transport or a first audio signal acquired by a microphone of the means of transport, and the control unit is configured to control a first display area to display a second image based on the sensor data, the cockpit comprising the first display area.

[0051] Referring to the second embodiment, in some implementations of the second embodiment, the sensor data includes a first audio signal, and the control unit is configured to control a first display area to display a second image based on feature information of the first audio signal, the feature information of the first audio signal includes at least one of the pitch, timbre, and tone of the first audio signal.

[0052] Referring to the second aspect, in some implementations of the second aspect, the sensor data includes a first image, and the device further includes a style transformation unit configured to perform a style transformation on the first image to obtain a second image.

[0053] Referring to the second aspect, in some implementations of the second aspect, the sensor data includes a first image, the device further includes a determination unit configured to determine text description content based on the first image, the text description content includes at least one of a scene, object, person, or environment corresponding to the first image, and to determine a second image based on the text description content.

[0054] Referring to the second aspect, in some implementations of the second aspect, the sensor data includes a first image, and the device further includes an image generation unit configured to input the first image into an image-image model to obtain a second image.

[0055] Referring to the second aspect, in some implementations of the second aspect, the cockpit further includes a second display area, and the control unit is further configured to control the second display area to display a third image based on the second audio signal and the second image when a detection unit detects a second audio signal from a second user in the cockpit, the second display area being associated with the area where the second user is located.

[0056] Referring to the second aspect, in some implementations of the second aspect, the control unit is configured to control the first display area to display a fourth image before the acquisition unit acquires sensor data, and to control the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and sensor data.

[0057] Referring to the second embodiment, in some implementations of the second embodiment, the device further includes a connection establishment unit configured to establish a connection to a terminal device, and a first transmission unit configured to transmit a second image to the terminal device.

[0058] Referring to the second aspect, in some implementations of the second aspect, the device further includes a second transmission unit configured to transmit a second image to a cloud server.

[0059] According to a third aspect, the present application provides a control device, the control device comprising a processing unit and a storage unit, the storage unit configured to store instructions, the processing unit executing instructions stored in the storage unit, thereby the control device performing a method according to any one of the possible implementations of the first aspect.

[0060] According to a fourth aspect, the present application provides a control system comprising a display device and a computing platform, the computing platform comprising a device in any one of the possible implementations of the second or third aspect.

[0061] In some possible implementations, the control system further includes sensors, which include cameras and / or microphones.

[0062] According to the fifth aspect, the present application provides a means of transport which includes an apparatus according to any one of the possible embodiments of the second aspect, or an apparatus according to the third aspect, or a system according to the fourth aspect.

[0063] In some possible embodiments, the means of transportation is a vehicle.

[0064] According to the sixth aspect, the present application provides a computer program product, the computer program product including computer program code, when the computer program code is executed on a computer, the computer becomes capable of performing a method according to any one of the possible implementations of the first aspect.

[0065] It should be noted that all or part of the computer program code may be stored in a first storage medium. The first storage medium may be packaged together with the processor or packaged separately from the processor. This is not particularly limited in this application.

[0066] According to the seventh aspect, the present application provides a computer-readable medium that stores program code. When the computer program code is executed on a computer, the computer is able to perform a method according to any one of the possible implementations of the first aspect.

[0067] According to the eighth aspect, the present application provides a chip system, the chip system including a processor, the processor being configured to call computer programs or computer instructions stored in memory, thereby enabling the processor to perform a method according to any one of the possible implementations of the first aspect.

[0068] In relation to the eighth aspect, in possible implementations, the processor is coupled to memory via an interface.

[0069] In relation to the eighth aspect, in possible implementations, the chip system further includes memory, which stores computer programs or computer instructions. [Brief explanation of the drawing]

[0070] [Figure 1] This is a block diagram of a vehicle according to an embodiment of the present invention. [Figure 2] This is a schematic diagram of the display distribution in the driver's seat of a vehicle according to an embodiment of the present invention. [Figure 3(a)] A group of graphical user interfaces (GUIs) according to one embodiment of the present invention is shown. [Figure 3(b)] A group of graphical user interfaces (GUIs) according to one embodiment of the present invention is shown. [Figure 3(c)] A group of graphical user interfaces (GUIs) according to one embodiment of the present invention is shown. [Figure 3(d)] A group of graphical user interfaces (GUIs) according to one embodiment of the present invention is shown. [Figure 4(a)] This invention presents a group of other GUIs according to one embodiment. [Figure 4(b)] This invention presents a group of other GUIs according to one embodiment. [Figure 4(c)] This invention presents a group of other GUIs according to one embodiment. [Figure 5(a)] This invention presents another group of GUIs according to one embodiment. [Figure 5(b)]This invention presents another group of GUIs according to one embodiment. [Figure 6(a)] This invention presents a group of other GUIs according to one embodiment. [Figure 6(b)] This invention presents a group of other GUIs according to one embodiment. [Figure 6(c)] This invention presents a group of other GUIs according to one embodiment. [Figure 7(a)] This invention presents another group of GUIs according to one embodiment. [Figure 7(b)] This invention presents another group of GUIs according to one embodiment. [Figure 7(c)] This invention presents another group of GUIs according to one embodiment. [Figure 8(a)-1] This invention presents another group of GUIs according to one embodiment. [Figure 8(a)-2] This invention presents another group of GUIs according to one embodiment. [Figure 8(b)-1] This invention presents another group of GUIs according to one embodiment. [Figure 8(b)-2] This invention presents another group of GUIs according to one embodiment. [Figure 8(c)-1] This invention presents another group of GUIs according to one embodiment. [Figure 8(c)-2] This invention presents another group of GUIs according to one embodiment. [Figure 9A] Another GUI according to one embodiment of the present invention is shown. [Figure 9B] Another GUI according to one embodiment of the present invention is shown. [Figure 10] This is a schematic flowchart of a control method according to one embodiment of this application. [Figure 11] This is a block diagram of a control system according to one embodiment of the present invention. [Figure 12] This is a block diagram of a control device according to one embodiment of the present application. [Modes for carrying out the invention]

[0071] The following describes the technical solutions in the embodiments of this application with reference to the accompanying drawings. In the description of the embodiments of this application, " / " means "or" unless otherwise specified. For example, A / B may represent A or B. In this specification, "and / or" describes only the relationship of association for the purpose of describing the related subjects and indicates that there may be three relationships. For example, A and / or B may represent the following three cases: A only exists, A and B both exist, or B only exists. "At least one" means one or more. For example, similar to "A and / or B", "at least one of A and B" describes the relationship of association between related objects and indicates that there may be three relationships. For example, at least one of A and B may represent the following three cases: A only exists, A and B both exist, or B only exists.

[0072] The prefixes "first," "second," etc., used in the embodiments of this application are intended solely to distinguish different subjects and do not impose any restrictions on the position, order, priority, quantity, content, etc., of the described subjects. The use of prefixes such as ordinal numbers used to distinguish the subjects described in the embodiments of this application does not constitute a limitation on the described subjects. For descriptions of what is described, please refer to the descriptions in the claims or the context of the embodiments, and the use of prefixes should not constitute a redundant limitation. Also, in the description of the embodiments, unless otherwise specified, "plural" means two or more.

[0073] Figure 1 is a block diagram of a vehicle 100 according to an embodiment of the present application. The vehicle 100 may include a sensing system 120, a display device 130, and a computing platform 150. The sensing system 120 may include one or more sensors that sense information about the surrounding environment of the vehicle 100. For example, the sensing system 120 may include a positioning system. The positioning system may be a global positioning system (GPS), a Beidou system, or another positioning system. The sensing system 120 may further include one or more of an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0074] Some or all of the functions of vehicle 100 may be controlled by a computing platform 150. The computing platform 150 may include one or more processors such as processors 151 to 15n (where n is a positive integer). A processor is a circuit having signal processing capabilities. In one implementation, a processor may be a circuit having the ability to read and execute instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which may be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, a processor may implement specific functions by using logic relationships of hardware circuits. Logic relationships of hardware circuits may be fixed or reconfigurable. For example, a processor may be a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process by which the processor loads a configuration document and implements the hardware circuit configuration can be understood as the process by which the processor loads instructions and implements some or all of the functions of the aforementioned units. In addition, the processor may alternatively be a hardware circuit designed for artificial intelligence and can be understood as an ASIC, for example, a neural network processing unit (NPU), a tensor processing unit (TPU), or a deep learning processing unit (DPU). In addition, the computing platform 150 may further include memory, which is configured to store instructions.Some or all of the processors 151 through 15n may call instructions in memory and execute those instructions to implement the corresponding functions.

[0075] The display devices 130 in the cockpit are mainly classified into two types: the first type is an in-vehicle display, and the second type is a projection display, such as a head-up display (HUD) device. An in-vehicle display is a physical display and is an important part of the in-vehicle infotainment system. Multiple displays can be arranged in the cockpit, such as a digital instrument display, a central display, a display in front of the front passenger (also called the front passenger), a display in front of the left rear passenger, and a display in front of the right rear passenger. Even a window can be used as a display. A head-up display is also called a head-up display system. A head-up display is mainly configured to display driving information such as speed and navigation on a display device in front of the driver (e.g., the windshield) to reduce the time the driver's eyes move, avoid pupillary changes caused by the driver's eye movement, and improve driving safety and comfort. For example, HUDs include combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality-HUD (AR-HUD) systems. It should be understood that HUDs may also have other types of systems with technological advancements. This is not limited to this application.

[0076] Figure 2 is a diagram illustrating an example of display distribution in a vehicle cockpit according to an embodiment of the present application. As shown in Figure 2, the vehicle cockpit may include a display 201 (sometimes referred to as the central display), a display 202 (sometimes referred to as the passenger-side entertainment screen), a display 203 (sometimes referred to as the entertainment screen in the left area of ​​the second row), a display 204 (sometimes referred to as the entertainment screen in the right area of ​​the second row), and an instrument panel.

[0077] It should be understood that the graphical user interface (GUI) in the following embodiments will be explained using the five-seat vehicle shown in Figure 2 as an example. Embodiments of this application are not limited thereto. For example, the cockpit of a seven-seat sport utility vehicle (SUV) may include a central display, a front passenger entertainment screen, an entertainment screen for the left area of ​​the second row, an entertainment screen for the right area of ​​the second row, an entertainment screen for the left area of ​​the third row, and an entertainment screen for the right area of ​​the third row. In one embodiment, the passenger compartment entertainment screen may be located at the top of the cockpit.

[0078] Figures 3(a) to 3(d) show a group of GUIs according to one embodiment of the present application.

[0079] As shown in Figure 3(a), the vehicle display 201 includes a content display area 301 and a function bar 302. The current content display area 301 displays profile image information 3011 of the logged-in user account on the vehicle, a Bluetooth function icon 3012, a Wi-Fi function icon 3013, a cellular network signal icon 3014, a wallpaper 3015, and a wallpaper change control 3016. The function bar 302 includes a desktop display switching icon 3021, an in-car circulation icon 3022, a driver's seat heating function icon 3023, a driver's seat air conditioning temperature display icon 3024, a passenger seat air conditioning temperature display icon 3025, a passenger seat heating function icon 3026, and a volume setting icon 3027.

[0080] As shown in Figure 3(b), when a user taps control 3016, a prompt box 3017 may appear in the content display area 301. The prompt box 3017 includes an external camera start control 3018, an internal camera start control, a microphone start control, a current wallpaper re-creation control, and a cancel control.

[0081] As shown in Figure 3(c), when the vehicle detects that the user has tapped the control 3018, it can activate the camera outside the cockpit and obtain the image 3019 acquired by the camera outside the cockpit.

[0082] In one embodiment, acquiring an image 3019 obtained by a camera outside the cockpit includes acquiring a single frame image or multiple frames images obtained by a camera outside the cockpit.

[0083] As shown in Figure 3(d), the vehicle can acquire an image 3019 obtained by a camera outside the cockpit, generate an image 3020 based on the image 3019, and display the image 3020 in the content display area 301.

[0084] For example, the ability of a vehicle to generate image 3020 based on image 3019 includes the vehicle inputting image 3019 into an image-to-image model to obtain image 3020.

[0085] For example, a vehicle being able to generate image 3020 based on image 3019 involves inputting image 3019 into an image-text model to obtain text description content corresponding to image 3019. For example, the text description content includes at least one of the following: scenario (e.g., winter, outdoors, or mountain climbing), object (e.g., white clouds or snow-covered mountain), person (e.g., two climbers), or environment (e.g., extremely cold, sunny, or blue sky and white clouds). The text description content corresponding to image 3019 is input into the text-image model to obtain image 3020.

[0086] In one embodiment, image 3020 may be a photograph, animation, video, or the like.

[0087] The embodiments described above are illustrated by using an example in which it is detected that the user taps control 3016, selects control 3018 in prompt box 3017, obtains image 3019 acquired by a camera outside the cockpit, and determines image 3020 based on image 3019. Embodiments of the present application are not limited thereto.

[0088] For example, by using voice commands to trigger further control of cameras outside the cockpit to acquire images, the final image displayed on the in-car display is determined based on those images. For instance, if it is detected that the user has given the voice command "Xiao A, Xiao A, create a wallpaper based on the scene outside the car," images acquired by cameras outside the cockpit may be retrieved, and a new wallpaper will be generated based on those images.

[0089] In another example, the content display area does not have to include control 3016. During the movement process, the vehicle may periodically acquire images acquired by cameras outside the cockpit. For example, in a first period, image 1 acquired by a camera outside the cockpit is input into an image-text model to acquire text description content 1, text description content 1 is input into a text-image model to acquire image 2, and the content display area 301 is controlled to display image 2. In a second period, image 3 acquired by a camera outside the cockpit is input into an image-text model to acquire text description content 2, and if text description content 2 is different from text description content 1, text description content 2 is input into a text-image model to acquire image 4, and the content display area 301 may be controlled to switch from displaying image 2 to displaying image 4.

[0090] In the embodiments of the present invention, an image acquired by a camera outside the cockpit can be recreated to acquire a different image. In this way, the user can view an image created based on locally acquired content on the in-vehicle display, thereby improving the user's visual experience, avoiding the visual fatigue caused to the user by displaying a single, homogeneous wallpaper on the in-vehicle display, and improving the user's driving experience.

[0091] Figures 4(a) to 4(c) show another group of GUIs according to one embodiment of the present application.

[0092] As shown in Figure 4(a), the prompt box 3017 further includes an in-vehicle camera start control 401. When the system detects that the user has tapped the control 401, the in-cockpit camera is activated and an image acquired by the in-cockpit camera can be captured.

[0093] For example, cameras in the cockpit may include cameras of the driver monitor system (DMS) or cameras of the cockpit monitor system (CMS).

[0094] As shown in Figure 4(b), the vehicle can acquire images 402 obtained by a camera in the cockpit.

[0095] In one embodiment, image 402 may include one or more frames of an image.

[0096] As shown in Figure 4(c), the vehicle can acquire an image 402 obtained by a camera in the cockpit, generate an image 403 based on the image 402, and display the image 403 in the content display area 301.

[0097] For example, a vehicle being able to generate image 403 based on image 402 includes the vehicle performing a style transformation on image 402 to obtain image 403 in a different style. For example, the different style may include Chinese painting, cartoon, bold lines, watercolor, or oil painting. The style of image 403 may also be a bold line style.

[0098] In the embodiments of this invention, an image acquired by a camera in the cockpit can be recreated to acquire a different image. In this way, the user can view an image created based on locally acquired content on the in-vehicle display, thereby improving the user's visual experience, avoiding the visual fatigue caused by displaying a single, uniform wallpaper on the in-vehicle display, and improving the user's driving experience. Furthermore, the problem of poor selfie or group photo effects due to the placement of a single camera in the cockpit or a single landscape background in the cockpit can be solved, thereby improving the shooting effect in the cockpit and the user's enjoyment of taking photos.

[0099] The process of automatically acquiring a new image after style conversion has been performed on an image acquired in the cockpit has been described above with reference to the GUI shown in Figures 4(a) to 4(c). In this embodiment of the present invention, after acquiring the image collected in the cockpit, the user may be further prompted to select a preferred style, allowing the user to participate in the process of creating a new wallpaper.

[0100] Figures 5(a) and 5(b) show another group of GUIs according to one embodiment of the present application.

[0101] As shown in Figure 5(a), after the vehicle acquires an image 402 obtained by a camera in the cockpit, it can display a prompt box 501 in the content display area. The prompt box includes prompt information such as "Image of the cockpit has been acquired. Please select one of the following styles," a control corresponding to the Chinese painting style, a control corresponding to the cartoon style, a control 502 corresponding to the watercolor style, a control corresponding to the oil painting style, and a cancel control.

[0102] The above is merely illustrated by using, as examples, Chinese painting style, cartoon style, watercolor style, and oil painting style. This embodiment of the present application is not limited thereto. For example, the prompt box 501 may further include more or fewer styles.

[0103] As shown in Figure 5(b), when a user taps the control 502, the vehicle converts the style of the image 402 acquired by the camera in the cockpit to a watercolor style to generate image 503, and can display image 503 in the content display area 301.

[0104] In the embodiments of the present invention, after acquiring an image by the camera, the user may be further prompted to select a preferred image style. After it is detected that the user has selected a style, a style transformation may be performed on the image acquired by the camera to obtain a new image. In this way, the user participates in the process of generating a new image, and as a result, the new image finally displayed on the display may better meet the user's expectations and improve the user's visual and driving experience.

[0105] Figures 6(a) to 6(c) show a group of other GUIs according to one embodiment of the present application.

[0106] As shown in Figure 6(a), the prompt box 3017 further includes an in-vehicle camera start control 601. When a user taps the control 601, a microphone in the cockpit may be controlled to acquire a voice command (or voice signal) given by the user.

[0107] As shown in Figure 6(b), the vehicle receives the voice command "Xiao A, Xiao A, help me change the wallpaper to one of white clouds surrounding snow-capped mountains" via the microphone.

[0108] As shown in Figure 6(c), in response to an acquired voice command, the vehicle can generate an image 602 based on the voice command and display the image 602 in the content display area 301.

[0109] For example, the ability of a vehicle to generate an image 602 based on a voice command includes the vehicle generating an image 602 based on a voice command by using a text-image model.

[0110] In the embodiments of the present invention, after acquiring a voice command via a microphone, a new image can be acquired based on the voice command. In this way, the user can view the image created based on the voice command transmitted by the user on the in-vehicle display, thereby improving the user's visual experience, avoiding the visual fatigue caused to the user by displaying a single, homogeneous wallpaper on the in-vehicle display, and improving the user's driving experience.

[0111] In one embodiment, the display in the cockpit may be controlled to show image 602 based on the area where the user giving voice commands is located.

[0112] For example, the vehicle may determine the area where the user giving the voice command is located, based on the voice command acquired by the microphone array in the cockpit. For example, if the area where the user giving the voice command is located is the passenger seat, image 602 may be displayed on the display 202.

[0113] In one embodiment, the ability of a vehicle to generate an image 602 based on a voice command includes generating an image 602 based on feature information of the voice command, the feature information including at least one of pitch, timbre, and tone.

[0114] For example, pitch may indicate whether the voice command is on an ascending or descending pitch. Table 1 shows the correspondence between pitch and image style.

[0115] [Table 1]

[0116] For example, if it is detected that the pitch of a voice command is an upward pitch, an image of a fashion style may be generated based on the voice command.

[0117] As another example, if it is detected that the pitch of the voice command is a descending pitch, a black and white style image may be generated based on the voice command.

[0118] For example, the timbre may indicate whether the voice command is in a male or female voice. Table 2 shows the correspondence between timbre and image style.

[0119] [Table 2]

[0120] For example, if the tone of a voice command is detected to be male, it is possible to generate an image in the Chinese style based on the voice command.

[0121] As another example, if the tone of a voice command is detected to be female, a watercolor-style image can be generated based on the voice command.

[0122] In the embodiments of the present invention, if the tone of a voice command is detected to be male or female, the final image can be generated based on an image style that matches the user's gender. This allows the image displayed on the in-vehicle display to better suit the user's gender characteristics and improve the user's driving experience.

[0123] For example, the tone may indicate whether the voice command is in the voice of an elderly person or a child. Table 3 shows another correspondence between tone and image style.

[0124] [Table 3]

[0125] For example, if the tone of a voice command is detected to be that of an elderly person, an oil painting-style image may be generated based on the voice command.

[0126] As another example, if the tone of a voice command is detected to be that of a child, a cartoon-style image can be generated based on the voice command.

[0127] In the embodiments of the present invention, if the tone of a voice command is detected to be that of an elderly person or a child, the final image can be generated based on an image style that matches the user's age. This allows the image displayed on the in-vehicle display to better suit the user's age characteristics and improve the user's driving experience.

[0128] For example, tone may reflect the user's mood when giving speech commands (e.g., happy, sad, etc.). Table 4 shows another correspondence between grayscale and image style.

[0129] [Table 4]

[0130] For example, if the tone of voice in a voice command is detected to be happy, a fresh-style image can be generated based on the voice command.

[0131] As another example, if the tone of a voice command is detected to be sad, a black and white style image may be generated based on the voice command.

[0132] In the embodiments of the present invention, the tone of voice of the user's voice commands can reflect the user's mood. If the tone of the voice command is detected to be happy or sad, the final image may be generated based on an image style that matches the user's mood, and as a result, the image displayed on the in-vehicle display can better suit the user's current mood and improve the user's driving experience.

[0133] Figures 7(a) to 7(c) show another group of GUIs according to one embodiment of the present application.

[0134] As shown in Figure 7(a), the vehicle may display image 701 in the content display area 301.

[0135] Image 701 may be an image acquired by a camera outside the cockpit, an image acquired by a camera inside the cockpit, or an image generated based on voice commands acquired by a microphone, or Image 701 may be an image stored locally in the vehicle (for example, a wallpaper downloaded by the user via a wallpaper application).

[0136] As shown in Figure 7(b), the vehicle receives the voice command "Xiao A, Xiao A, add eagle to image" via the microphone.

[0137] As shown in Figure 7(c), in response to the acquired voice command, the vehicle can generate image 702 based on image 701 and the voice command, and display image 702 in the content display area 301.

[0138] In one embodiment, image 702 may be generated based on the area where the user who made the voice command is located. For example, if the area where the user who made the voice command is located is the passenger seat, an eagle may be added to the right-hand area of ​​image 701. For example, if the area where the user who made the voice command is located is the driver's area, an eagle may be added to the left-hand area of ​​image 701.

[0139] In the embodiments of the present invention, the user can edit or recreate images based on voice commands, either images locally stored in the vehicle or images determined based on data acquired by a camera or microphone. In this way, the user can create images based on voice commands, and as a result, all users in the cockpit become image editing artists, and the images ultimately presented to the user can better meet the user's expectations and improve the user's driving experience.

[0140] The technical solutions in the embodiments of this application may be further applied to scenarios where the in-vehicle display is a long screen (sometimes referred to as a long connected screen). Figures 8(a)-1 to 8(c)-2 show another group of GUIs according to one embodiment of this application.

[0141] As shown in Figures 8(a)-1 and 8(a)-2, the central display and the front passenger screen in the vehicle cockpit may be the same screen. The screen may be divided into two display areas, namely area 801 and area 802. Area 801 is the display area closer to the driver, and area 802 is the display area closer to the passenger seat. Area 801 displays the profile image information of user A in the driver's area, a Bluetooth function icon, a Wi-Fi function icon, a cellular network signal icon, and wallpaper 803. Area 802 displays the profile image information of user B in the passenger seat and wallpaper 804.

[0142] When it is detected that user A in the driver's area has given voice command 1, "Xiao A, Xiao A, help me change the wallpaper to one with snow-covered mountains surrounded by white clouds," the GUI shown in Figures 8(b)-1 and 8(b)-2 may be displayed.

[0143] As shown in Figures 8(b)-1 and 8(b)-2, in response to an acquired voice command 1 given by user A, image 805 may be generated based on voice command 1, and area 801 is controlled to switch from displaying wallpaper 802 to displaying image 805. In this case, wallpaper 804 may continue to be displayed in area 802.

[0144] When it is detected that user B in the passenger seat has given voice command 2, "Xiao A, Xiao A, add an eagle to the image," the GUI shown in Figures 8(c)-1 and 8(c)-2 may be displayed.

[0145] As shown in Figures 8(c)-1 and 8(c)-2, in response to the acquired voice command 2 given by user B, image 806 may be generated based on image 805 and voice command 2, and region 802 is controlled to switch from displaying wallpaper 804 to displaying image 806. In this case, region 801 may switch from displaying image 805 to displaying wallpaper 803.

[0146] In this embodiment of the present invention, when user B wishes to edit or recreate an image by giving a voice command in the cockpit, the vehicle may display the edited or recreated image in a display area (e.g., area 802) of the area where user B is located, based on the area where user B is located, thereby allowing user B to conveniently view the edited or recreated image and improving the user experience.

[0147] After a new image is generated based on a voice command acquired by a camera or microphone, the vehicle can also transmit the image to a terminal device. Figures 9A and 9B show another GUI according to one embodiment of the present application.

[0148] As shown in Figures 9A and 9B, the vehicle may display an image 901 in the content display area 301. In the above embodiment, the image 901 may be an image generated based on data acquired by a camera or microphone. The vehicle may transmit the image 901 to a mobile phone. In response to the received image 901 transmitted by the vehicle, the mobile phone may display a prompt box. The prompt box includes prompt information "Please receive the photo transmitted by the vehicle", a thumbnail 902 of the image 901, a control 903 for setting the image as the mobile phone's wallpaper, a control 904 for storing the image in an album, and a control 905 for ignoring the image.

[0149] In one embodiment, if the vehicle can transmit image 901 to a mobile phone, the vehicle can transmit a thumbnail of image 901 to the mobile phone. In this way, information about the thumbnail of image 901 is transmitted to the mobile phone, which reduces the delay when the vehicle transmits the original image content of image 901 to the mobile phone, increases the speed at which the user can view the image generated by the vehicle via the mobile phone, and improves the user experience. Furthermore, the transmission resource overhead that occurs when the user does not wish to store image 901 on the mobile phone is also reduced.

[0150] In one embodiment, when the mobile phone detects a user tapping control 903, it can replace the wallpaper with image 901.

[0151] In one embodiment, when the user taps the control 903, the mobile phone can replace the wallpaper with image 901. When the user taps the control 903, the mobile phone may request the original image content of image 901 from the vehicle, and in response to the reception request sent by the mobile phone, the vehicle may send the original image content of image 901 to the mobile phone, and in response to the received original image content of image 901 sent by the vehicle, the mobile phone may set the original image content of image 901 as the mobile phone's wallpaper.

[0152] In one embodiment, when the mobile phone detects a user tapping the control 904, it may store the image 901 in the mobile phone's photo album.

[0153] In one embodiment, when the user taps the control 904, the mobile phone may store the image 901 in the mobile phone's album. When the user taps the control 904, the mobile phone may request the original image content of image 901 from the vehicle, and in response to the reception request sent by the mobile phone, the vehicle may send the original image content of image 901 to the mobile phone, and in response to the received original image content of image 901 sent by the vehicle, the mobile phone may store the original image content of image 901 in the mobile phone's album.

[0154] In one embodiment, a vehicle and a mobile phone may establish a connection by using short-range communication technology. Short-range communication technology includes, but is not limited to, Bluetooth® connection, Wi-Fi connection, etc. In this way, when generating image 901, the vehicle may transmit image 901 to the mobile phone by using short-range communication technology.

[0155] In one embodiment, before transmitting image 901 to a mobile phone using short-range communication technology, the vehicle may further determine that the account logged into the vehicle is associated with the account logged into the mobile phone. For example, if the account logged into the vehicle is the same as the account logged into the mobile phone, image 901 may be transmitted to the mobile phone; or if the account logged into the vehicle and the account logged into the mobile phone are in the same family group, image 901 may be transmitted to the mobile phone; or if the account logged into the mobile phone is an account authorized by the vehicle, image 901 may be transmitted to the mobile phone. In this way, interference caused to unfamiliar passengers when image 901 is transmitted to their mobile phones can be avoided.

[0156] In one embodiment, a vehicle and a mobile phone can establish a connection via a cloud server. The vehicle may upload images 901 to the cloud server, and the cloud server may be a server used by a family group of users to share information.

[0157] For example, the account logged into the vehicle is account 1, and account 1 and account 2 belong to the same family group. When generating image 901, the vehicle may upload image 901 to a cloud server. The cloud server may send image 901 to devices other than vehicle 1 under account 1, and to devices under account 2.

[0158] In one embodiment, when generating image 901, the vehicle may further upload image 901 to a server corresponding to a wallpaper mall. In this way, another user can view image 901 through a wallpaper application, and as a result, another user can download image 901 through a wallpaper application.

[0159] Figure 10 is a schematic flowchart of a control method 1000 according to one embodiment of the present application. Method 1000 may be performed by a means of transport (e.g., a vehicle), or by the aforementioned computing platform, or by a system including a computing platform and sensors, or by a system-on-a-chip (SoC) on a computing platform, or by a processor on a computing platform. Method 1000 includes the following steps:

[0160] S1010: Sensor data is acquired when input from a first user in the cockpit is detected to instruct the system to generate an image. The sensor data includes a first image acquired by a camera on the vehicle or a first audio signal acquired by a microphone on the vehicle.

[0161] For example, as shown in Figure 3(b), the input from the first user instructing the generation of an image may be an operation such as tapping control 3018.

[0162] For example, as shown in Figure 4(a), the input from the first user to instruct the generation of an image may be an operation such as tapping control 401.

[0163] For example, as shown in Figure 6(a), the input from the first user instructing the generation of an image may be an operation such as tapping control 601.

[0164] For example, as shown in Figures 8(a)-1 and 8(a)-2, the input from the first user to indicate the generation of an image could be the voice command 1 given by the user, "Xiao A, Xiao A, help me change the wallpaper to one with snow-capped mountains surrounded by white clouds."

[0165] Optionally, the cameras on the vehicle may include cameras outside the vehicle's cockpit or cameras inside the cockpit.

[0166] S1020: Controls the first display area to display a second image based on sensor data.

[0167] Optionally, the sensor data includes a first audio signal, and controlling the first display area to display a second image based on the sensor data includes controlling the first display area to display a second image based on characteristic information of the first audio signal, where the characteristic information of the first audio signal includes at least one of the pitch, timbre, and tone of the first audio signal.

[0168] For example, a vehicle may store a correspondence between characteristic information of an audio signal and an image style, and controlling a first display area to display a second image based on the characteristic information of a first audio signal includes controlling the first display area to display a second image based on the characteristic information of a first audio signal and the correspondence.

[0169] For example, the correspondence may be as shown in Table 1, Table 2, Table 3, or Table 4.

[0170] Optionally, the sensor data includes a first image, and the method further includes performing a style transformation on the first image in order to obtain a second image.

[0171] For example, as shown in Figure 4(c), the first image may be image 402, and the second image may be image 403. Image 403 can be obtained by performing a style transformation on image 402.

[0172] Optionally, the sensor data includes a first image, and the method further includes determining text description content based on the first image, wherein the text description content includes at least one of a scene, object, person, or environment corresponding to the first image, and determining a second image based on the text description content.

[0173] For example, as shown in Figures 3(c) and 3(d), the first image may be image 3019, and the second image may be image 3020. The mode of transport may input image 3019 into an image-text model to obtain text description content corresponding to image 3019. For example, the scene corresponding to the first image is "winter" and "outdoors," the objects contained in the first image are "snowy mountains" and "white clouds," and the environment in the first image is "sunny." The mode of transport may input text description content corresponding to image 3019 into a text-image model to obtain image 3020.

[0174] Optionally, the sensor data includes a first image, and the method further includes inputting the first image into an image-to-image model in order to acquire a second image.

[0175] For example, as shown in Figures 3(c) and 3(d), the first image may be image 3019, and the second image may be image 3020. The means of transport may input image 3019 into an image-to-image model to obtain image 3020.

[0176] Optionally, the cockpit further includes a second display area, the method further including controlling the second display area to display a third image based on the second audio signal and a second image when a second audio signal is detected from a second user in the cockpit, the second display area being associated with an area where the second user is located.

[0177] For example, as shown in Figures 8(c)-1 and 8(c)-2, the first display area may be area 801, and the second display area may be area 802. When the means of transport detects a speech instruction 2 from user B in the passenger seat, it may generate an image 806 based on the speech instruction 2 and image 805, and control area 802 to display the image 806.

[0178] Optionally, before acquiring sensor data, the method further includes controlling a first display area to display a fourth image, and based on the sensor data, controlling the first display area to display a second image, and based on the fourth image and sensor data, controlling the first display area to switch from displaying the fourth image to displaying the second image.

[0179] For example, as shown in Figures 7(a) to 7(c), the fourth image may be image 701, the second image may be image 702, and the sensor data may be the voice command given by the user, "Xiao A, Xiao A, add an eagle to the image." The means of transport may generate image 702 based on image 701 and the voice command, and control the content display area 301 to switch from displaying image 701 to displaying image 702.

[0180] Optionally, method 1000 further includes establishing a connection to a terminal device and transmitting a second image to the terminal device.

[0181] For example, as shown in Figures 9A and 9B, after the vehicle establishes a connection with the mobile phone, the vehicle may transmit the generated image 902 to the mobile phone.

[0182] Optionally, sending a second image to the terminal device includes sending a thumbnail of the second image to the terminal device.

[0183] Optionally, before sending a second image to the terminal device, method 1000 further includes determining that the account logged into the transportation service is associated with the account logged into the terminal device.

[0184] In this way, by transmitting a second image after it has been determined that the account logged into the transportation system and the account logged into the terminal device are associated, it is possible to avoid interference caused by the second image to unfamiliar passengers in the cockpit.

[0185] Optionally, method 1000 further includes sending a second image to a cloud server.

[0186] For example, the cloud server may be a server used by a family group of users to share information, or it may be a server that supports a wallpaper application.

[0187] Figure 11 is a block diagram of a control system 1100 according to one embodiment of the present application. The control system 1100 includes an image acquisition device 1101, a speech acquisition device 1102, an image style conversion model 1103, an image-to-text model 1104, a text-to-image model 1105, an image-to-image model 1106, an automatic speech recognition (ASR) module 1107, and an audio-to-image model 1108.

[0188] The image acquisition device 1101 is configured to acquire images acquired by cameras inside the cockpit and / or images acquired by cameras outside the cockpit. The image acquisition device 1101 can transmit the acquired images to an image style conversion model 1103, an image-to-text model 1104, or an image-to-image model 1106.

[0189] The audio acquisition device 1102 is configured to acquire the audio signal acquired by the microphone. The audio acquisition device 1102 is further configured to transmit the acquired audio signal to the speech recognition model 1107.

[0190] The image style conversion model 1103 is configured to perform style conversion on an image acquired by the image acquisition device 1101 and to acquire the image after style conversion.

[0191] The image-text model 1104 is configured to acquire corresponding text description content based on an image acquired by the image acquisition device 1101. For example, the text description content includes at least one of a scene, object, person, or environment corresponding to the image. The image-text model 1104 is further configured to transmit the determined text description content to the text-image model 1105.

[0192] The text-to-image model 1105 is configured to generate a new image based on the text description content transmitted by the image-to-text model 1104.

[0193] The image model 1106 is configured to generate a new image based on the image acquired by the image acquisition device 1101.

[0194] The ASR module 1107 is configured to analyze the audio signal acquired by the audio acquisition device 1102 and to identify the text content corresponding to the audio signal. The ASR module 1107 is further configured to transmit the recognized text content to the voice-image model 1108.

[0195] The voice-image model 1108 is configured to generate an image based on text content transmitted by the ASR module 1107.

[0196] Optionally, the voice-to-image model 1108 and the text-to-image model 1105 may be the same model or different models.

[0197] Optionally, system 1100 may further include a multistream model. The multistream model is configured to acquire image information. The image information includes images generated by the image style conversion model 1103, images generated by the text-to-image model 1105, images generated by the image-to-image model 1106, images generated by the audio-to-image model 1108, or images originally displayed on the in-vehicle display. The multistream model 1109 is further configured to receive text content transmitted by the ASR module 1107. The multistream model may generate a new image based on the text content and the acquired image information.

[0198] Optionally, system 1100 may send an image generated by text-image model 1105, an image generated by image-image model 1106, an audio-image model 1108, or an image generated by a multistream model to the head unit desktop application. The head unit desktop application controls the in-vehicle display to show the image.

[0199] Optionally, the system 1100 may transmit images generated by the image style conversion model 1103, images generated by the text-image model 1105, images generated by the image-image model 1106, images generated by the audio-image model 1108, or images generated by the multistream model 1109 to a head unit application, a terminal device (e.g., a mobile phone), or a cloud server. For example, the cloud server may be a server used by users in a family group to share information, or the cloud server may be a server corresponding to a wallpaper application.

[0200] Figure 12 is a block diagram of a control device 1200 according to one embodiment of the present application. As shown in Figure 12, the device 1200 includes a detection unit 1210, an acquisition unit 1220, and a control unit 1230. The acquisition unit 1220 is configured to acquire sensor data when the detection unit 1210 detects input from a first user in the cockpit of a means of transport to instruct it to generate an image, the sensor data including a first image acquired by a camera of the means of transport or a first audio signal acquired by a microphone of the means of transport, and the control unit 1230 is configured to control a first display area to display a second image based on the sensor data, the cockpit including the first display area.

[0201] Optionally, the sensor data includes a first audio signal, and the control unit 1230 is configured to control the first display area to display a second image based on characteristic information of the first audio signal, the characteristic information of the first audio signal includes at least one of the pitch, timbre, and tone of the first audio signal.

[0202] Optionally, the sensor data includes a first image, and the device 1200 further includes a style transformation unit configured to perform a style transformation on the first image to obtain a second image.

[0203] For example, the functions performed by the style conversion unit may be performed by the image style conversion model 1103 within the aforementioned system 1100.

[0204] Optionally, the sensor data includes a first image, and the device 1200 further includes a determination unit configured to determine text description content based on the first image, where the text description content includes at least one of a scene, object, person, or environment corresponding to the first image, and to determine a second image based on the text description content.

[0205] Optionally, the functions implemented by the decision unit may be implemented by the image-text model 1104 and the text-image model 1105 within the aforementioned system 1100.

[0206] Optionally, the sensor data includes a first image, and the device 1200 further includes an image generation unit configured to input the first image into an image-image model to obtain a second image.

[0207] For example, the functions implemented by the image generation unit can be implemented by the image-image model 1106 within the aforementioned system 1100.

[0208] Optionally, the cockpit further includes a second display area, and the control unit 1230 is further configured to control the second display area to display a third image based on the second audio signal and the second image when the detection unit 1210 detects a second audio signal from a second user in the cockpit, and the second display area is associated with the area where the second user is located.

[0209] Optionally, the control unit 1230 is configured to control the first display area to display a fourth image before the acquisition unit acquires sensor data, and to control the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and sensor data.

[0210] Optionally, the device 1200 further includes a connection establishment unit configured to establish a connection to a terminal device, and a first transmission unit configured to transmit a second image to the terminal device.

[0211] Optionally, the device 1200 further includes a second transmission unit configured to transmit a second image to a cloud server.

[0212] For example, the detection unit 1210 may be the computing platform shown in Figure 1, or a processing circuit, processor, or controller on the computing platform shown in Figure 1. For example, the detection unit 1210 is the processor 151 on the computing platform. The processor 151 acquires data obtained by a touch sensor on an in-vehicle display and, based on the data, may determine that input from a first user in the cockpit has been detected to instruct the generation of an image.

[0213] In another example, the acquisition unit 1220 may be the computing platform of Figure 1, or a processing circuit, processor, or controller on the computing platform of Figure 1. For example, the acquisition unit 1220 is a processor 152 on the computing platform. The processor 152 may acquire sensor data when the processor 151 determines that input from a first user in the cockpit has been detected to instruct the generation of an image. For example, the processor 152 may control the external camera to turn on and acquire a first image acquired by the external camera. Alternatively, the processor 152 may control the internal microphone to turn on and acquire an audio signal acquired by the microphone.

[0214] In another example, the control unit 1230 may be the computing platform of Figure 1, or a processing circuit, processor, or controller on the computing platform of Figure 1. For example, the control unit 1230 may be a processor 153 on the computing platform, and the processor 153 may generate a second image based on sensor data. For example, the processor 153 may store an image style conversion model 1103, or the processor 153 may store an image-to-text model 1104 and a text-to-image model 1105, or the processor 153 may store an image-to-image model 1106, or the processor 153 may store an ASR module 1107 and a voice-to-image model 1108.

[0215] The functions performed by the detection unit 1210, the functions performed by the acquisition unit 1220, and the functions performed by the control unit 1230 may be performed by different processors, or by the same processor, or some functions may be performed by the same processor. This is not limited to the embodiments of this application.

[0216] It should be understood that the division into units within a device is merely a logical functional division. During actual implementation, all or some of the units may be integrated into a single physical entity or may be physically separated. In addition, units within a device may be implemented in the form of software invoked by a processor. For example, the device includes a processor, the processor is connected to memory, the memory stores instructions, and the processor invokes the instructions stored in memory to perform one of the aforementioned methods or to perform the function of each unit of the device. The processor is a general-purpose processor, such as a CPU or microprocessor. The memory is either in-device memory or external memory. Alternatively, units within a device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented by designing the hardware circuits. Hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the aforementioned units are implemented by designing the logical relationships between the elements in the circuit. As another example, in another implementation, the hardware circuit may be implemented by using a PLD. Using an FPGA as an example, the hardware circuit may include a large number of logic gate circuits, and the connections between the logic gate circuits are configured by using configuration files to implement some or all of the functions of the aforementioned units. All of the aforementioned units of the device may be implemented in the form of software invoked by the processor, or all units may be implemented in the form of hardware circuits, or some units may be implemented in the form of software invoked by the processor and the rest may be implemented in the form of hardware circuits.

[0217] Each unit in the aforementioned apparatus may be one or more processors (or processing circuits) configured to carry out the method described above, for example, a CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, or FPGA, or at least two combinations of these processor forms.

[0218] In addition, all or some of the units in the aforementioned device may be integrated or implemented independently. In one embodiment, these units are integrated and implemented in the form of an SoC. The SoC may include at least one processor configured to perform any one of the methods described above or to perform the functions of the units in the device. The type of at least one processor may vary. For example, the at least one processor may include a CPU and an FPGA, a CPU and an artificial intelligence processor, or a CPU and a GPU.

[0219] One embodiment of the present application further provides a control device comprising a processing unit and a storage unit. The storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit, thereby enabling the device to perform the control method performed in the above-described embodiment.

[0220] If the device is optionally placed within a means of transport, the processing unit may be the processors 151-15n shown in Figure 1.

[0221] One embodiment of this application further provides a control system. The control system may include a computing platform and a display device. The computing platform may include the control device 1200 described above.

[0222] For example, the display device may include one or more in-vehicle displays, such as display 201, display 202, display 203, or display 204 in Figure 2.

[0223] Optionally, the means of transport may further include sensors, such as cameras or microphones.

[0224] One embodiment of the present application further provides a means of transport which may include the control device 1200 or the control system described above.

[0225] One embodiment of this application further provides a computer program product, the computer program product including computer program code. When the computer program code is executed on a computer, the computer becomes capable of performing the control method in the above-described embodiment.

[0226] One embodiment of this application further provides a computer-readable medium that stores program code. When the computer program code is executed on a computer, the computer becomes capable of executing the control method in the above-described embodiment.

[0227] One embodiment of this application further provides a chip comprising a circuit configured to perform the control method in the above-described embodiment.

[0228] In the implementation process, the steps in the aforementioned method may be completed by using hardware integrated logic circuits within the processor or by using instructions in software form. The methods disclosed with reference to embodiments of this application may be performed directly by a hardware processor or by using a combination of hardware modules and software modules within the processor. The software modules may be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage media is located in memory, and the processor reads information from memory and, in combination with the processor's hardware, completes the steps of the aforementioned method. To avoid repetition, further details are not described here.

[0229] It should be understood that the sequence numbers of the processes described above do not imply the execution sequence in the various embodiments of this application. The execution order of the processes should be determined according to the function and internal logic of the processes and should not be construed as any limitation to the implementation processes of the embodiments of this application.

[0230] Those skilled in the art will recognize, in combination with the example units and algorithmic steps described in the embodiments disclosed herein, that this application may be implemented by electronic hardware or by a combination of computer software and electronic hardware. Whether the functions are performed by hardware or by software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementations should not be considered to exceed the scope of this application.

[0231] For the sake of a convenient and concise explanation, the detailed operating processes of the aforementioned systems, apparatus, and units should be referred to the corresponding processes in the embodiments of the methods described above, and it will be readily apparent to those skilled in the art that the details are not described again here.

[0232] In some embodiments provided in this application, it should be understood that the disclosed systems, apparatus, and methods may be implemented in other ways. For example, the embodiments of the apparatus described are merely examples. For example, the division into units is merely a logical functional division, and other divisions may be used in actual implementations. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the mutual coupling, direct coupling, or communication connection shown or described may be implemented through some interfaces. Indirect coupling or communication connection between apparatus or units may be implemented in an electrical, mechanical, or other form.

[0233] Units described as separate components may or may not be physically separate, and the parts shown as units may or may not be physical units, may be located in one place, or may be distributed across multiple network units. Some or all of the units may be selected based on the actual requirements to achieve the objectives of the solution of the embodiment.

[0234] In addition, the functional units in the embodiments of this application may be integrated into a single processing unit, or each unit may exist physically independently, or two or more units may be integrated into a single unit.

[0235] When a function is implemented in the form of a software function unit and sold or used as an independent product, the function may be stored on a computer-readable storage medium. Based on such understanding, the technical solutions of this application, in essence, or in part with respect to the prior art, or a part of the technical solutions, may be implemented in the form of a software product. A computer software product is stored on a storage medium and includes several instructions for instructing a computer device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the method described in embodiments of this application. The aforementioned storage medium includes any medium capable of storing program code, such as a USB flash drive, removable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or compact disk.

[0236] The foregoing description is merely a specific implementation of the present application and is not intended to limit the scope of protection of the present application. Any modifications or substitutions that are readily conceivable by a person skilled in the art within the scope of the art disclosed herein shall fall within the scope of protection of the present application. Accordingly, the scope of protection of the present application shall be subject to the scope of protection of the claims.

Claims

1. A control method applied to the cockpit of a means of transport, the cockpit including a first display area, Acquiring sensor data when an input is detected instructing the generation of an image from a first user in the cockpit, wherein the sensor data includes a first image acquired by the camera of the means of transport, or a first audio signal acquired by the microphone of the means of transport. A method comprising controlling the first display area to display a second image based on the sensor data.

2. The sensor data includes the first audio signal, and the first display area is controlled to display the second image based on the sensor data. This includes controlling the first display area to display the second image based on the characteristic information of the first audio signal, The method according to claim 1, wherein the characteristic information of the first audio signal includes at least one of the pitch, timbre, and tone of the first audio signal.

3. The sensor data includes the first image, The method according to claim 1, further comprising performing a style transformation on the first image to obtain the second image.

4. The sensor data includes the first image, Determining text description content based on the first image, wherein the text description content includes at least one of a scene, object, person, or environment corresponding to the first image. The method according to claim 1, further comprising determining the second image based on the text description content.

5. The sensor data includes the first image, The method according to claim 1, further comprising inputting the first image into an image-image model to obtain the second image.

6. The cockpit further includes a second display area, The method according to any one of claims 1 to 5, further comprising controlling the second display area to display a third image based on the second audio signal and the second image when a second audio signal is detected from a second user in the cockpit, wherein the second display area is associated with the area where the second user is located.

7. Before acquiring the aforementioned sensor data, Controlling the first display area to display a fourth image, Based on the sensor data, the first display area is controlled to display the second image, The method according to any one of claims 1 to 6, further comprising controlling the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and the sensor data.

8. Establishing a connection to the terminal device, The method according to any one of claims 1 to 7, further comprising transmitting the second image to the terminal device.

9. The method according to any one of claims 1 to 8, further comprising transmitting the second image to a cloud server.

10. A control device comprising a detection unit, an acquisition unit, and a control unit, The acquisition unit is configured to acquire sensor data when it detects an input to instruct the detection unit to generate an image from a first user in the cockpit of the means of transport, the sensor data including a first image acquired by the camera of the means of transport, or a first audio signal acquired by the microphone of the means of transport. A control device comprising a control unit configured to control a first display area to display a second image based on sensor data, and a cockpit comprising the first display area.

11. The sensor data includes the first audio signal, and the control unit further: The system is configured to control the first display area to display the second image based on the characteristic information of the first audio signal. The apparatus according to claim 10, wherein the characteristic information of the first audio signal includes at least one of the pitch, timbre, and tone of the first audio signal.

12. The sensor data includes the first image, The apparatus according to claim 10, further comprising a style transformation unit configured to perform a style transformation on the first image to obtain the second image.

13. The sensor data includes the first image, Based on the first image, text description content is determined, and the text description content includes at least one of a scene, object, person, or environment corresponding to the first image. The apparatus according to claim 10, further comprising a determination unit configured to determine the second image based on the text description content.

14. The sensor data includes the first image, The apparatus according to claim 10, further comprising an image generation unit configured to input the first image into an image-image model to obtain the second image.

15. The cockpit further comprises a second display area, and the control unit further comprises The apparatus according to any one of claims 10 to 14, wherein when the detection unit detects a second audio signal from a second user in the cockpit, it controls the device to display a third image in the second display area associated with the area where the second user is located, based on the second audio signal and the second image.

16. The control unit is Before the acquisition unit acquires the sensor data, the first display area is controlled to display a fourth image. The apparatus according to any one of claims 10 to 15, configured to control the first display area to switch from displaying the fourth image to displaying the second image based on the fourth image and the sensor data.

17. A connection establishment unit configured to establish a connection to a terminal device, A first transmission unit configured to transmit the second image to the terminal device, The apparatus according to any one of claims 10 to 16, further comprising:

18. A second transmission unit configured to send the second image to a cloud server, The apparatus according to any one of claims 10 to 17, further comprising:

19. A control device, Memory configured to store computer programs, A device comprising a processor configured to execute the computer program stored in the memory, thereby enabling the device to perform the control method described in any one of claims 1 to 9.

20. A control system comprising a display device and a computing platform, wherein the computing platform comprises a control device according to any one of claims 10 to 19.

21. A means of transport comprising a control device according to any one of claims 10 to 19, or a control system according to claim 20.

22. The means of transportation according to claim 21, wherein the means of transportation is a vehicle.

23. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the computer is able to execute the control method described in any one of claims 1 to 9.

24. A chip comprising a processor and a data interface, wherein the processor reads instructions stored in memory via the data interface and performs the control method according to any one of claims 1 to 9.

Images