Interaction control method and device, electronic equipment, storage medium and vehicle

Abstract

Embodiments of the present disclosure provide an interaction control method and device, electronic equipment, storage medium and vehicle. The interaction control method comprises: in the case that a cross-screen playing instruction is acquired, determining a current display screen and a target display screen according to the cross-screen playing instruction; playing a guide sound, the guide sound being a stereophonic sound in which a simulated sound source position is gradually moved from the position of the current display screen to the position of the target display screen. Since the simulated sound source position of the guide sound is gradually moved from the position of the current display screen to the position of the target display screen, the user's auditory system can perceive the gradual change of the position of the guide sound, and the problem of abrupt auditory sensation caused by instantaneous switching is avoided, thereby improving the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of human-computer intelligent interaction technology, specifically to an interactive control method, device, electronic device, storage medium, and vehicle. Background Technology

[0002] With the popularization and commercialization of the smart cockpit concept, some mid-to-high-end vehicles have been equipped with multiple display screens to enable more convenient and personalized interaction between drivers and passengers and the vehicle, as well as personalized entertainment needs of passengers in various locations.

[0003] Currently, some smart cockpits can achieve cross-screen playback, meaning they can switch the audio / video content playing on the current display to the target display. However, existing smart cockpits switch the audio / video content from the current display to the target display instantly during cross-screen playback. Although users are mentally prepared for the content switching, the sudden change in the audio / video source from the current display to the target display still feels abrupt and detracts from the user experience. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this disclosure provide an interactive control method, apparatus, electronic device, storage medium, and vehicle.

[0005] In a first aspect, embodiments of this disclosure provide an interactive control method, including:

[0006] Upon receiving a cross-screen playback instruction, the current display screen and the target display screen are determined based on the cross-screen playback instruction, which includes the identifier of the current display screen and the identifier of the target display screen.

[0007] Play a guide sound, which is a stereo sound that simulates the sound source gradually moving from the current display position to the target display position.

[0008] Optionally, before playing the introductory sound, the method includes:

[0009] Determine the simulated sound source movement path based on the current display screen's location and the target display screen's location;

[0010] Play guiding sounds, including: playing guiding sounds that simulate the movement of a sound source along its path.

[0011] Optionally, play guiding sounds simulating the movement of a sound source along its path, including:

[0012] Determine the simulated playback position of each audio frame of the guide sound within the simulated sound movement path;

[0013] Based on the simulated playback position, audio frames, and the positions of each speaker in the stereo playback system, a modulation signal is generated to control the sound output of each speaker.

[0014] The corresponding loudspeakers are driven by modulation signals so that the sound waves emitted by each loudspeaker reverberate in the sound field to form guiding sounds.

[0015] Optionally, before determining the simulated sound source movement path based on the current display screen's position and the target display screen's position, the following steps are included:

[0016] Determine if there are any obstacles between the current display screen and the target display screen;

[0017] When there are obstacles between the current display screen and the target display screen, determine the path that starts from the position of the current display screen, avoids the obstacles, and ends at the position of the target display screen as the simulated sound source movement path.

[0018] Optionally, before determining the simulated sound source's movement path, the method further includes:

[0019] Determine whether the current display screen shows the first interactive image;

[0020] With the first interactive image displayed on the current screen, the simulated sound source movement path is determined based on the position of the current screen and the position of the target screen, including:

[0021] Determine the first display position of the first interactive image;

[0022] The path starting from the first display position and ending at the target display position is determined as the simulated sound source movement path.

[0023] Optionally, if it is determined that the current display screen is showing the first interactive image, the method further includes:

[0024] Determine the second display position of the second interactive image to be displayed on the target display screen;

[0025] The simulated sound source movement path is determined based on the display position of the first interactive image and the position of the target display screen, including:

[0026] The path starting from the first display position and ending at the second display position is determined as the simulated sound source movement path.

[0027] Optionally, the method may also include: obtaining the interaction operations performed by the user when interacting with the current display screen;

[0028] Determine whether the interactive operation triggers a specific playback task, which is a playback task associated with a cross-screen playback command;

[0029] When the interactive operation triggers a specific playback task, it is determined that a cross-screen playback instruction has been obtained.

[0030] Optionally, determining whether the interactive operation matches the cross-screen playback command includes:

[0031] Determine whether the interactive operation triggers a specific playback task, which is a playback task that can only be executed by the target display screen;

[0032] When the interactive operation is an operation that triggers a specific playback task, determine that the interactive operation is an operation that matches the cross-screen playback instruction.

[0033] Optionally, the method also includes:

[0034] When the interactive operation triggers a specific playback task, the corresponding prompt sound is determined based on the specific playback task;

[0035] The stereo playback system is controlled to play guiding sounds that gradually move the simulated sound source from the current display screen position to the position of the screen to be switched to, including:

[0036] Use the prompt sound as a guide sound to control the stereo playback system.

[0037] Optionally, the interaction is a voice interaction; prior to acquiring the interaction action to be performed when the user interacts with the current display screen, the following are included:

[0038] Determine the user's area of ​​interest;

[0039] Determine the current display screen based on the direction of attention.

[0040] Optionally, determine the user's focus, including:

[0041] Acquire real-time user images, which include images of the user's eye features;

[0042] Process user images to determine the user's iris and sclera regions;

[0043] The relative direction of the user's gaze is determined by the area of ​​the sclera region located around the iris region;

[0044] The direction of focus is determined based on the relative direction of the line of sight and the external parameters of the camera capturing the user's image.

[0045] Secondly, embodiments of this disclosure provide an interactive control device, including:

[0046] The screen determination unit is used to determine the current display screen and the target display screen according to the cross-screen playback instruction when a cross-screen playback instruction is obtained, wherein the cross-screen playback instruction includes the identifier of the current display screen and the identifier of the target display screen;

[0047] The sound playback unit is used to play guiding sounds, which are stereo sounds that simulate the sound source gradually moving from the current display position to the target display position.

[0048] Thirdly, embodiments of this disclosure provide an electronic device, including a processor and a memory, wherein the memory is used to store computer programs;

[0049] When a computer program is loaded by a processor, it causes the processor to execute the interactive control method described above.

[0050] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the aforementioned interactive control method.

[0051] Fifthly, embodiments of this disclosure provide a vehicle including a processor and a memory, the memory being used to store computer programs;

[0052] When a computer program is loaded by a processor, it causes the processor to execute the interactive control method described above.

[0053] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0054] By employing the solution provided in this embodiment, upon receiving a cross-screen playback command, the current display screen and the target display screen can be determined based on the identifiers of the current display screen and the target display screen in the cross-screen playback command. Subsequently, based on the positions of the current display screen and the target display screen, the stereo playback system is controlled to play a guide sound whose simulated sound source position gradually moves from the current display screen position to the target display screen position. Since the simulated sound source position of the guide sound gradually moves from the current display screen position to the target display screen position, after using stereo playback of the guide sound, the user can perceive the gradual change in the position of the guide sound without experiencing the abrupt auditory sensation caused by instantaneous switching, thus improving the user experience. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0056] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort, wherein:

[0057] Figure 1 This is a flowchart of the interactive control method provided in the embodiments of this disclosure;

[0058] Figure 2 This is a flowchart of an interactive control method provided in some embodiments of this disclosure;

[0059] Figure 3 This is a flowchart illustrating the determination of the simulated sound source movement path as defined in some embodiments of this disclosure;

[0060] Figure 4 This is a flowchart of an interactive control method provided in some embodiments of this disclosure;

[0061] Figure 5 This is a schematic diagram of the structure of the interactive control device provided in the embodiments of this disclosure;

[0062] Figure 6 This is a schematic diagram of the structure of an electronic device provided in some embodiments of this disclosure. Detailed Implementation

[0063] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0064] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0065] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0066] Figure 1 This is a flowchart of the interactive control method provided in an embodiment of this disclosure. Figure 1 As shown, the interactive control method provided in this embodiment includes S110-S120.

[0067] It should be noted that the interactive control method provided in this disclosure can be executed by various electronic devices, such as vehicle terminal devices, smart home devices, and other terminal devices that can be configured with multiple display screens. Of course, the method provided in this disclosure can also be executed collaboratively by multiple electronic devices, for example, a central control device, a passenger entertainment device, and a rear entertainment device of a vehicle terminal device can be executed collaboratively. The interactive control method provided in this disclosure will be described below using a vehicle terminal device including multiple displays.

[0068] S110: Upon receiving a cross-screen playback instruction, determine the current display screen and the target display screen based on the cross-screen playback instruction.

[0069] Cross-screen playback commands are used to instruct audio-visual content to be played across screens, that is, to switch the audio-visual content from the current display screen to the target display screen for playback.

[0070] Cross-screen playback commands can be commands issued by the user or commands generated by the vehicle terminal device based on the user's operation and preset operation rules. This disclosure does not impose any particular limitation.

[0071] In some embodiments, when a user is watching content playing on the current display screen (such as playing a movie or TV show) and wants to switch the content playing on the current display screen to another screen, the user can actively issue interactive control voice or interactive actions to issue a cross-screen playback command, thereby triggering the in-vehicle terminal device to perform a cross-screen playback operation.

[0072] In other embodiments, when a user is interacting with the current display screen and issues a specific interaction command, the in-vehicle terminal device parses the aforementioned interaction command and determines that the playback operation specified in the interaction command can only be executed by the target display screen. The in-vehicle terminal device then proactively generates a cross-screen playback command and sends it to a specific internal functional module to trigger the execution of the cross-screen playback operation.

[0073] The current display screen is the screen currently playing audio-visual content, while the target display screen is the screen that will play audio-visual content after the cross-screen operation is achieved. In practice, the current display screen can be the screen that the user is currently focusing on or watching.

[0074] In some embodiments of this disclosure, the cross-screen playback instruction includes identifiers of the current display screen and the target display screen. After obtaining the cross-screen playback instruction, the vehicle terminal device can obtain the identifiers of the current display screen and the target display screen by parsing the cross-screen playback instruction, and then determine the current display screen and the target display screen based on the identifiers.

[0075] In other embodiments of this disclosure, the cross-screen playback command includes a command type identifier. After obtaining the cross-screen playback command, the vehicle-mounted terminal device can obtain the command type identifier by parsing the command. Subsequently, the vehicle-mounted terminal device can determine the current display screen and the target display screen corresponding to the cross-screen playback command by looking up a pre-configured correspondence table based on the command type identifier.

[0076] In some embodiments of this disclosure, the cross-screen playback instruction does not include an identifier for the current display screen, and the in-vehicle terminal device does not store an identifier for the current display screen in its pre-configured mapping table. In this case, the in-vehicle terminal device can also determine the current display screen through other means. In some embodiments, the in-vehicle terminal device can use the screen on which the user is performing an interactive operation as the current display screen, or it can use the screen on which the user is currently viewing as the current display screen.

[0077] For example, if the user triggering the cross-screen playback command is a front passenger who is viewing the front passenger entertainment display, then the front passenger entertainment display can be designated as the first screen. As another example, if the user triggering the cross-screen playback command is a rear passenger who is viewing the rear entertainment display, then the rear entertainment display can be designated as the current screen. Similarly, if the user triggering the cross-screen playback command is the driver who is viewing the head-up display in front of the driver, the driver's instrument cluster display, or the control display on the steering wheel, then that head-up display, instrument cluster display, or control display on the steering wheel can be designated as the current screen.

[0078] S120: Play guide sound. The guide sound is a stereo sound that simulates the sound source gradually moving from the current display position to the target display position.

[0079] In this embodiment of the disclosure, after the vehicle terminal device determines the current display screen and the target display screen, the position of the current display screen and the position of the target display screen can then be determined.

[0080] In some embodiments, the in-vehicle terminal device stores a location information table, which includes the current location of each screen. After determining the current display screen and the target display screen, the location of the current display screen and the location of the target display screen can be determined by looking up the location information table.

[0081] A guidance sound is used to guide and prompt the user that the content being played on the current display screen will be switched to the content played on the target display screen. In some embodiments of this disclosure, the guidance sound can be the sound of the audio-visual content being played on the current display screen. For example, when the current display screen is playing a movie, the guidance sound can be the movie's narration. In other embodiments of this disclosure, the guidance sound can be a specially designed prompt sound. For example, when the in-vehicle terminal device actively generates a cross-screen playback command and determines that the content being played on the current display screen needs to be switched to the target display screen, it can generate corresponding prompt text based on the playback content and the name of the target display screen. The prompt voice is generated based on the prompt text using text-to-speech conversion.

[0082] In this embodiment, multiple speakers are configured in the vehicle compartment, each deployed in a different location within the compartment to form a stereo playback system. For example, some vehicles may have a stereo playback system with 21 speakers, which are respectively installed on the center console (e.g., in the middle and on the left and right sides of the center console), the inside of the A-pillars, the four doors, the inside of the roof, or behind the front seats or the middle row seats (in the case of a full-size SUV with three rows of seats). The in-vehicle terminal equipment can control the vibration of each speaker to produce sound, thereby creating reverberation. The user can experience stereo sound after hearing the reverberated sound.

[0083] After determining the positions of the current display screen and the target display screen, and identifying the guide sound to be played, the in-vehicle terminal equipment generates modulation signals to control the operation of each speaker based on the positions of the current display screen, the target display screen, and the speakers in the stereo playback system. The sound waves emitted by each speaker mix in the vehicle to form a stereo sound field, and this stereo sound field simulates the location of the simulated sound source, thus creating the stereo guide sound. How the in-vehicle terminal controls the operation of each speaker to produce the stereo guide sound will be analyzed later.

[0084] In this embodiment of the disclosure, the simulated sound source position of the guide sound output by the stereo playback system gradually moves from the current display screen to the target display screen. In other words, the guide sound is a stereo sound in which the simulated sound source position gradually moves from the current display screen position to the target display screen position.

[0085] Because the simulated sound source of the guiding sound moves gradually from the current display position to the target display position, after hearing the guiding sound, the user can feel that the guiding sound is coming from the current display, gradually moving towards the target display, and finally coming from the target display.

[0086] After receiving the stereo prompt, the user's auditory system can clearly feel the guiding sound gradually moving from the current display location to the target display location. In other words, the user can feel the gradual change in the location of the guiding sound, rather than the sudden change in the location of the sound source caused by an instantaneous switch. This avoids the abrupt auditory sensation caused by an instantaneous switch and improves the user experience.

[0087] Figure 2 This is a flowchart of an interactive control method provided in some embodiments of this disclosure. For example... Figure 2 As shown, in some embodiments of this disclosure, the interactive control method may include S210-S230.

[0088] S210: Upon receiving a cross-screen playback instruction, determine the current display screen and the target display screen based on the cross-screen playback instruction.

[0089] S220: Determine the simulated sound source movement path based on the current display screen's position and the target display screen's position.

[0090] The simulated sound source movement path refers to the movement path of the simulated sound source from the current position of the display screen to the position of the target display screen. The simulated sound source movement path can be a straight path, a curved path, or a combination of straight and curved paths. This disclosure does not impose any particular limitation on the embodiments.

[0091] S230: Play a guide sound that simulates the movement of a sound source along its path.

[0092] Determining the simulated sound source movement path based on the current display screen's location and the target display screen's location means determining the path the simulated sound source will take from the current display screen's location to the target display screen's location.

[0093] In some embodiments of this disclosure, both the current display screen and the target display screen are fixed in position. For example, the current display screen is a passenger entertainment display screen fixed at the passenger seat, and the target display screen is a central control display screen fixed at the center console. In this case, the vehicle terminal device can pre-store the simulated sound source movement path between the current display screen and the target display screen. After determining the current display screen and the target display screen, the vehicle terminal device can look up a table to determine the corresponding simulated sound source movement path.

[0094] In other embodiments of this disclosure, at least one of the current display screen and the target display screen is a display screen whose position is not fixed. For example, the current display screen is a rear entertainment display screen whose position can be moved, and the target display screen is a central control display screen fixed at the center console. In this case, the in-vehicle terminal device determines the simulated sound source movement path based on the current display screen and the target display screen. This can be done by determining the current position coordinates of the current display screen and the target display screen, and then planning the simulated sound source movement path based on the current position coordinates of the current display screen and the target display screen.

[0095] In this embodiment of the disclosure, playing a guide sound in S230, simulating the movement of a sound source along a sound source movement path, may include S231-S233.

[0096] S231: Determine the simulated playback position of the audio frame in the guide sound within the simulated sound movement path.

[0097] Determining the simulated playback position of the audio frame in the guide sound within the simulated sound movement path is equivalent to determining the coordinate position of the simulated sound source when playing the aforementioned audio frame.

[0098] The vehicle-mounted terminal device can determine the simulated playback position in the simulated sound movement path of all audio frames before playing the guide sound, or it can gradually determine the simulated playback position during the playing of the guide sound. This disclosure does not impose any particular limitation on the embodiments.

[0099] S232: Generate modulation signals to control the sound output of each speaker based on the analog playback position, audio frame, and the position of each speaker in the stereo playback system.

[0100] In order for the sound emitted by each speaker to reverberate in the sound field, and for the user to perceive the effect of stereo movement after reverberation, it is necessary to adaptively adjust the operating characteristics of each speaker in the stereo playback system. This includes adjusting whether each speaker is working and emitting sound, the sound intensity when the speaker is working and emitting sound, and the playback time for a specific audio frame signal.

[0101] In some embodiments, the in-vehicle terminal device needs to determine the sound characteristics (e.g., timbre and tone quality) required for reverberation based on audio frames, determine the reverberation position characteristics formed by each speaker based on the simulated playback position, and then generate modulation signals to control the sound output of each speaker based on the sound characteristics, reverberation position characteristics, and the position of each speaker. To achieve the aforementioned objectives, technicians first calibrated the system based on the vehicle cabin space, speaker installation positions, and the listening positions of the main users. Based on the calibration results, sound field reverberation tests were conducted to construct a sound field reverberation model.

[0102] When the vehicle-mounted terminal device is working, the sound field reverberation model can be used to determine the modulation signal for controlling each speaker based on the simulated playback position and audio frame.

[0103] S233: The corresponding speaker is driven by a modulation signal.

[0104] Based on the aforementioned analysis, after each speaker is driven to work by a modulation signal, the sound waves emitted by each speaker reverberate in the sound field to form a single-point sound corresponding to the audio frame. The continuous playback of the aforementioned single-point sound can form a guiding sound.

[0105] To more intuitively illustrate how to control the stereo playback system to play guiding sounds that gradually move the simulated sound source from the current display position to the target display position, the following uses an actual in-car scenario as an example.

[0106] The in-car environment features 21 speakers as described above. These 21 speakers can be divided into front speakers (including speakers located on the center console, inside the A-pillar, and in the front doors), middle speakers (including speakers located inside the B-pillar, in the rear doors, and in the roof above the middle seats), and rear speakers (including speakers located behind the middle seats and in the roof above the rear seats).

[0107] The current display screen is the rear entertainment screen located behind the front seats, in front of the rear seats, and suspended from the vehicle's roof. The target display screen is the central control display screen located in the center of the dashboard. Based on the current screen's location and the target screen's location, the simulated sound source movement path is a straight line from the rear entertainment screen to the central control display screen. The guiding voice is "Please follow me to look at the central control display screen."

[0108] In order to make the aforementioned guiding sound move along the simulated sound source movement path (that is, along the aforementioned straight path), the vehicle terminal device will determine the modulation signal to control each speaker based on the aforementioned path, the guiding sound, and the location of each speaker, and use the modulation signal to control each speaker to emit sound.

[0109] The intuitive effect of the vehicle terminal equipment controlling the sound output of each speaker is that, as the guiding sound is played: (1) the volume of the sound played by the rear speakers is always small and continues to decrease; (2) the volume of the middle speakers is large at the beginning, but gradually decreases as the guiding sound is played; (3) the volume of the front speakers is small at the beginning, but gradually increases as the guiding sound is played; (4) regardless of whether it is the front speakers, middle speakers or rear speakers, the modulation characteristics of the left and right speakers are the same (that is, relative to the aforementioned straight path), so that the reverberated guiding sound is kept in the longitudinal middle position of the vehicle.

[0110] Figure 3This is a flowchart illustrating the determination of the simulated sound source movement path as defined in some embodiments of this disclosure. For example... Figure 3 As shown, in some embodiments of this disclosure, the aforementioned S220, which determines the simulated sound source movement path based on the current display screen position and the target display screen position, may include S221-S223.

[0111] S221: Determine if there are any obstacles between the current display screen and the target display screen; if not, proceed to S222; if not, proceed to S223.

[0112] S222: Determine the simulated sound source movement path by establishing the connection path between the current display screen and the target display screen.

[0113] S223: Determine the path that starts from the current display screen's position, avoids obstacles, and ends at the target display screen's position as the simulated sound source's movement path.

[0114] In this embodiment, the obstacle is a problem located on the line connecting the current display screen and the target display screen. In practical applications, there may or may not be an obstacle between the current display screen and the target display screen. If there is no obstacle between the current display screen and the target display screen, S222 can be executed directly. However, if there is an obstacle between the current display screen and the target display screen, S223 can be executed.

[0115] For example, if the current display screen is the passenger-side entertainment screen and the target display screen is the center console screen, and there are no obstacles between the two screens, then the shortest straight line between them can be used as the simulated sound source movement path.

[0116] For example, if the current display screen is a rear entertainment screen located behind the front seats, and the target display screen is the central control screen, with the front seats obstructing the view between them, then directly using the line connecting the two screens as the simulated sound source movement path would cause it to pass through the front seats. Outputting guiding voice commands along this path would result in the guiding sound emanating from the seats, causing perceptual confusion for the user and negatively impacting their experience. To avoid this, the in-vehicle terminal device can determine the simulated sound source movement path based on the positions of the current display screen, the front seats, and the target display screen. This ensures the simulated sound source moves around obstacles like the front seats, moving in an open area, preventing perceptual confusion and guaranteeing a better user experience.

[0117] Figure 4 This is a flowchart of an interactive control method provided in some embodiments of this disclosure. For example... Figure 4As shown, in some embodiments of this disclosure, the interactive control method includes S310-S360.

[0118] S310: Upon receiving a cross-screen playback instruction, determine the current display screen and the target display screen based on the cross-screen playback instruction.

[0119] S320: Determine whether the current display screen shows the first interactive image; if so, execute S330.

[0120] S330: Determine the first display position of the first interactive image.

[0121] S340: Determine the path that starts from the first display position and ends at the position of the target display screen as the simulated sound source movement path.

[0122] S350: Controls the stereo playback system to play guide sounds as the simulated sound source moves from its current position to the target position along the simulated sound source's movement path.

[0123] In some embodiments of this disclosure, a user can interact with the current screen, which displays a first interactive avatar for interaction with the user. When the user interacts with the in-vehicle terminal device through the current display, the user's visual focus is primarily on the first interactive avatar on the current display. To provide a better interactive experience for the user interacting with the first interactive avatar, the in-vehicle terminal device uses the position of the first interactive avatar on the current display as the location of a simulated sound source for the first interactive avatar, and controls the stereo playback system to play interactive voice based on the aforementioned simulated sound source location.

[0124] To make users perceive the guiding sound as originating from the location of the first interactive avatar, the in-vehicle terminal device sets the first display position of the first interactive avatar as the starting point of the simulated sound source movement path, and the position of the target display screen as the ending point. The simulated sound source movement path is then determined based on the starting and ending points. By setting the starting point of the simulated sound source movement path to the first display position of the first interactive avatar, users perceive the guiding sound as originating from that location. This aligns the user's visual focus with the initial position of the sound source, making the prompts sound more like stereo sound from a real avatar, thus enhancing the user experience.

[0125] In some applications, when the user's focus is shifted to the target display screen, that is, after the user's visual focus is gradually guided from the current display screen to the target display screen, a second interactive avatar can be displayed on the target display screen to interact with the user through the second interactive avatar.

[0126] In the aforementioned circumstances, in some embodiments of this disclosure, the vehicle-mounted terminal device may also execute S360 before executing S350 to determine the simulated sound source movement path based on the first display position of the first interactive image and the target display screen.

[0127] S4360: Determine the second display position of the second interactive image to be displayed on the target display screen.

[0128] The second interactive avatar is displayed at the location on the target display screen when the user's visual focus is moved to that screen. In other words, the second interactive avatar will appear at the designated display location after the user's visual focus shifts to the target display screen.

[0129] When performing the aforementioned S360, the aforementioned S340 may include S341.

[0130] S351: Determine the path that starts from the first display position and ends at the second display position as the simulated sound source movement path.

[0131] In other words, when S360 is executed, the endpoint of the simulated sound source movement path is determined based on the second display position of the second interactive avatar. This ensures that once the user's visual focus moves to the target display screen, the second interactive avatar is immediately output on the target display screen, thereby shifting the user's focus to the second interactive avatar and enabling the second interactive avatar to interact with the user. In this way, the endpoints of the user's visual focus shift and auditory shift are the same, further enhancing the user experience.

[0132] It should be noted that executing S340 or S341 does not conflict with S233. That is, before executing S340 or S341, it can be determined whether there is an obstacle between the current display screen (or the first interactive image) and the display position of the second interactive image to be displayed. If there is an obstacle between them, the simulated sound source movement path can be determined based on the first display position (or the position of the obstacle) of the first interactive image and the second display position of the second interactive image, so that the simulated sound source movement path avoids the obstacle.

[0133] Optionally, in some embodiments of this disclosure, in the case of performing the aforementioned S360, the vehicle terminal device may also perform S370.

[0134] S370: When the simulated sound source position of the guiding sound transitions to or near the second display position, a second interactive image is displayed at the second display position.

[0135] When the simulated sound source moves closer to the target display screen, a second interactive avatar is displayed at the second display position, guiding the user to interact on the target display screen and improving the user experience.

[0136] Optionally, in some embodiments of this disclosure, the vehicle terminal device may also execute S380 while performing the aforementioned S350.

[0137] S380: Move and / or shrink the display of the first interactive image based on the portion of the analog sound source moving along its path, at least close to the current display screen.

[0138] In this embodiment of the disclosure, the first interactive image is moved and shrunk along the simulated sound source moving path, at least to the part that is close to the current display screen. This may be divided into several cases.

[0139] The first scenario: The current display screen and the target display screen are roughly coplanar, for example, the current display screen is the passenger entertainment display screen, and the target display screen is the central control display screen. In this case, the simulated sound source movement path is on the plane where the current display screen and the target display screen are located, or roughly on the plane where the current display screen and the target display screen are located. The in-vehicle terminal device can play guiding sounds along the simulated sound source movement path located on the current display screen while simultaneously displaying the first interactive image on the current display screen.

[0140] The second scenario: The current display screen and the target display screen are not coplanar. For example, the current display screen is the rear entertainment display screen, and the target display screen is the central control display screen. In this case, the simulated sound source location needs to move from the plane of the current display screen to the plane of the target display screen, and the simulated sound source location must be on the plane of the current display screen for the first part of the simulated sound source movement path. The in-vehicle terminal device can play guiding sounds along the simulated sound source movement path located on the current display screen while simultaneously displaying the first interactive image on the current display screen.

[0141] The third scenario: The current display screen and the target display screen are set side-by-side in parallel. For example, the current display screen is the entertainment screen located in the middle of the rear seats of the vehicle, and the target display screen is the central control screen. The simulated sound source needs to move from the plane where the current display screen is located to the plane where the target display screen is located, and the simulated sound source movement path is a straight line from the current location of the first interactive avatar to the display location of the second interactive avatar. The in-vehicle terminal device can play guiding sounds along the simulated sound source movement path while gradually shrinking the first interactive avatar to simulate the current display screen of the first interactive avatar.

[0142] The fourth scenario: The current display screen and the target display screen are misaligned. For example, the current display screen is the entertainment screen behind the front seats, and the target display screen is the central control screen. The simulated sound source needs to move from the plane of the current display screen to the plane of the target display screen, and the simulated sound source's movement path should be on the current display screen for the first part and leave the current display screen for the second part. The in-vehicle terminal device can play stereo prompts along the simulated sound source's movement path while first moving the first interactive image on the current display screen. When the simulated sound source's movement path is no longer on the current display screen, the first interactive image should be minimized until it disappears.

[0143] As previously stated, in some embodiments of this disclosure, the cross-screen playback command is a command triggered when the user performs an interactive operation with the current display screen. Correspondingly, in some embodiments of this disclosure, S130-S150 may be executed before S110.

[0144] S130: Obtain the interaction operation performed when the user interacts with the current display screen.

[0145] S140: Determine whether the interactive operation matches the cross-screen playback instruction; if so, execute S150.

[0146] S150: Confirmation that cross-screen playback command has been obtained.

[0147] In this embodiment of the disclosure, when an interaction operation between the user and the current display screen is obtained, the vehicle-mounted terminal device can perform instruction matching based on the operation type of the interaction operation to determine whether the interaction operation matches a cross-screen playback instruction. If the interaction operation matches a cross-screen playback instruction, the vehicle-mounted terminal device determines that a cross-screen playback instruction has been obtained.

[0148] In some embodiments, S140 determining whether the interactive operation matches the cross-screen playback instruction may include S141-S142.

[0149] S141: Determine whether the interactive operation is an operation that triggers a specific playback task; if so, execute S1122.

[0150] S142: Determine that the interactive operation is an operation that matches the cross-screen playback instruction.

[0151] In this embodiment of the disclosure, a specific playback task is a playback task that can only be executed by the target display screen. If the interaction operation is the operation that triggers the specific playback task, it is determined that the subsequent specific playback task can only be executed by the target display screen and cannot be executed by the current display screen. At this time, the interaction operation is an operation that matches the cross-screen playback instruction, so it can trigger the generation of the cross-screen playback instruction so that the vehicle terminal device can execute the subsequent S110-S120.

[0152] For example, the user's interaction with the current display screen involves finding a destination and determining the navigation map based on that destination. The current display screen is the rear entertainment screen. Since the navigation map can only be played on the central control screen, after the user finds the destination, determines the navigation map based on the destination, and triggers the start navigation operation, the in-vehicle electronic equipment determines that the start navigation operation is an operation that triggers a specific playback task. Therefore, it determines that the interaction operation matches the cross-screen playback command, and at this point, it confirms that the cross-screen playback command has been obtained.

[0153] Optionally, in some embodiments of this disclosure, when performing S1411 to determine that the interactive operation is an operation that triggers a specific playback task, the vehicle terminal device may also perform S143.

[0154] S143: Determine the corresponding prompt sound based on the specific playback task.

[0155] Voice prompts are sounds used to indicate specific playback tasks to be performed by the target display screen. For example, based on the aforementioned start navigation operation, the in-vehicle terminal device may generate a message such as "Play navigation map on the central control display screen."

[0156] When executing the aforementioned S143, S120 controls the stereo playback system to play a guide sound that gradually moves the position of the simulated sound source from the current display position to the target display position, which can specifically be S121.

[0157] S121: Use the prompt sound as a guide sound to control the stereo playback system.

[0158] In some embodiments of this disclosure, the interaction between the user and the current display screen is a voice interaction. When the interaction is a voice interaction, the vehicle terminal device may also execute steps S160-S170 before acquiring the interaction operation performed by the user when interacting with the current display screen.

[0159] S160: Determine the user's location and / or the direction of the user's attention.

[0160] In some embodiments of this disclosure, the vehicle-mounted terminal device can determine the user's location in various ways. For example, in some embodiments, the vehicle-mounted terminal device can determine the user's location based on pressure sensors on the seat, thereby determining the user's seat position.

[0161] In other embodiments of this disclosure, at least two microphones are configured in the carriage, and the two microphones monitor the audio signals in the carriage in real time. The vehicle terminal equipment can determine the user's location through the following steps S161-S163.

[0162] S161: Acquire audio signals from at least two microphones.

[0163] S162: Determine whether the audio signal includes a speech signal; if so, execute S163.

[0164] S163: Perform spatial positioning based on voice signals to determine the user's location.

[0165] After acquiring audio signals from at least two microphones, the in-vehicle terminal device first determines whether the audio signals include speech signals. If the audio signals include speech signals, it is determined that the user is speaking. At this point, the in-vehicle terminal device can perform spatial positioning based on the speech signals included in at least two audio signals to determine the user's location, specifically the location of the user's head area.

[0166] Specifically, the vehicle-mounted terminal device performs spatial positioning based on at least two audio signals, including voice signals. This is achieved by performing inverse calculations based on the reception time of voice signals with similar characteristics in each audio signal and the position of the microphone to determine the user's mouth position. For details on how to perform spatial positioning based on voice signals to determine the user's location, please refer to existing technical literature and products in the field of acoustics; this disclosure will not elaborate further.

[0167] Optionally, before executing the aforementioned S163, the vehicle terminal device may also execute S164: acquire the pressure signal output by the seat pressure sensor.

[0168] Correspondingly, in S163, spatial positioning is performed based on the voice signal to determine the user's location, specifically in S1631.

[0169] S1631: Spatial fusion positioning is performed based on voice signals and pressure signals to determine the user's location.

[0170] In a specific embodiment, the vehicle-mounted terminal device can use Kalman filtering or particle filtering to perform spatial fusion positioning of the location determined by the voice signal and the location determined by the pressure signal to determine the user's location. By fusing the voice signal and the pressure signal for positioning, the accuracy of determining the user's location can be improved.

[0171] In some other embodiments of this disclosure, an interior camera is also installed in the passenger compartment, which can capture images inside the passenger compartment. The vehicle-mounted terminal device can determine the user's focus direction using steps S164-S168 as follows.

[0172] S164: Acquire real-time user footage.

[0173] S165: Process the user image to determine the iris region and the sclera region.

[0174] In this embodiment, the interior camera can capture images inside the vehicle compartment in real time and determine the user image within those images based on the captured images. It should be noted that in this embodiment, the user image includes images of the user's eye features.

[0175] After acquiring the user's image, the in-vehicle terminal device can determine the iris region and sclera region of the user's eyes based on the image. In a specific embodiment, a pre-trained deep learning model can be used to determine the user's eye region, and pixel extraction can be performed based on the pixel features of the iris region and sclera region to determine the iris region and sclera region.

[0176] S166: Determine the user's relative focus direction based on the area of ​​the sclera region located around the iris region.

[0177] Common sense tells us that a user's focus is determined by eye movement. The ultimate purpose of eye movement is to change the orientation of the iris area. When a user moves their eyes, the area of ​​the sclera (white part of the eye) exposed outside the eyelid changes. For example, when a user is looking straight ahead, the areas of the sclera on both sides of the iris are roughly the same, as are the areas on the top and bottom of the iris. When the user moves their eyes to the right, the area of ​​the sclera on the left side of the iris is approximately the same as the area on the right side, and the areas on the top and bottom of the iris are roughly the same. Based on the distribution characteristics of the sclera area, the user's relative focus direction can be determined. It should be noted that the aforementioned relative focus direction can be a precise direction or a rough one.

[0178] S167: Determine the current focus based on the relative focus direction and the extrinsic parameters of the camera capturing the user's image.

[0179] S168: Determine the current focus position based on the current focus direction.

[0180] The extrinsic parameters of a camera are parameters used to characterize how the camera's position and shooting angle are calibrated in the world coordinate system. Extrinsic parameters may include a rotation matrix. Determining the current direction of interest based on the relative direction of interest and the extrinsic parameters involves performing a coordinate transformation based on the relative direction of interest and the extrinsic parameters, and then using the transformed direction of interest as the current direction of interest.

[0181] After determining the current focus direction, the display area of ​​the current screen in this direction can be determined, and this display area is the current focus position.

[0182] In the foregoing embodiments, the current focus direction is determined based on the user's image. In other embodiments, the current focus direction of the user can also be determined based on the user's location.

[0183] S170: Determine the current display screen based on location and / or direction of focus.

[0184] Based on the user's location and / or the direction of the user's attention, determine the current display screen that the user is most likely to be viewing, and use the aforementioned display screen as the current display screen.

[0185] In some embodiments of this disclosure, the current display screen that the user is focusing on may be a large display screen, and the user may only see a portion of the content on the current display screen, or the content that the user is focusing on may only be a portion of the current display screen. For example, if the current display screen is a large rear entertainment display screen located in the middle of the rear seats of a vehicle, rear passengers on the left and right sides may only focus on the area adjacent to the current display screen.

[0186] In this case, after executing S170, the vehicle-mounted terminal equipment can also execute S180-S190.

[0187] S180: Determine the third display position based on the user's location and / or the direction of the user's attention.

[0188] S190: Control the current display screen to display the first interactive image in the third display position.

[0189] Since the current display screen size is relatively large, in order to make it easier for users to determine the first interactive image to interact with, in this embodiment of the disclosure, the display position of the first interactive image (that is, the third display position) is determined according to the user's location and / or the user's direction of attention, and the current display screen is controlled to display the first interactive image at the third display position.

[0190] To better understand S180-S190, the following example illustrates the concept. In a specific application, the vehicle has a large rear entertainment display screen located in front of the center rear seat, with a passenger seated on the right side (i.e., a passenger is seated to the right rear of the entertainment display screen). At this time, the rear entertainment display screen is in standby mode. When a user wants to check navigation routes, they look towards the rear entertainment display screen and say "Ideal Student." The in-vehicle voice interaction module then receives the sound signal collected by the microphone and determines that the user has activated the interaction module of the in-vehicle terminal device. The in-vehicle terminal device, based on the sound signals collected by multiple microphones, determines that the passenger is seated on the rear side of the rear seat and is looking at the right side of the rear entertainment display screen. The rear entertainment display screen can then be designated as the current display screen, and a first interactive avatar will be displayed on the right side of the screen. This allows the first interactive avatar to visually interact with the user and execute subsequent operations that trigger the user's display task.

[0191] Optionally, in some embodiments of this disclosure, the vehicle terminal device may also execute S100 while performing the aforementioned S190.

[0192] S100: Controls the stereo playback system to output interactive voice with the analog sound source located at the third display position.

[0193] By controlling the stereo playback system to output simulated sound source location at the third display position for interactive voice, users can perceive the first interactive avatar speaking and interacting with them, thus improving the user experience. For example, after the user outputs "ideal classmate," the in-vehicle terminal device can locate the simulated sound source at the first interactive avatar display position and output a reply voice saying "What do you need?" This guides the user to continue interacting with the in-vehicle terminal device and execute any subsequent cross-screen operations that may be triggered.

[0194] The foregoing embodiments use a vehicle terminal device as the execution device to explain the method provided in the embodiments of this disclosure. In some implementations of this disclosure, the execution device can also be a smart home device. This specification will not elaborate on how smart home devices execute the method in a smart home scenario; specific practical examples can be derived by combining various hardware devices in a smart home scenario with the aforementioned method execution process.

[0195] In addition to providing the aforementioned interactive control method, embodiments of this disclosure also provide an interactive control device for implementing the aforementioned interactive control method.

[0196] Figure 5 This is a schematic diagram of the structure of the interactive control device provided in the embodiments of this disclosure. Figure 5 As shown, the interactive control device 500 provided in this embodiment includes a screen determination unit 501 and a sound playback unit 502.

[0197] The screen determination unit 501 is used to determine the current display screen and the target display screen according to the cross-screen playback instruction when a cross-screen playback instruction is obtained, wherein the cross-screen playback instruction includes the identifier of the current display screen and the identifier of the target display screen.

[0198] The sound playback unit 502 is used to play guide sounds, which are stereo sounds that simulate the sound source position gradually moving from the current display position to the target display position.

[0199] In some embodiments of this disclosure, the interactive control device 500 further includes a movement path determination unit. The movement path determination unit determines a simulated sound source movement path based on the current display screen's position and the target display screen's position. Correspondingly, the sound playback unit 502 plays a guiding sound indicating that the simulated sound source is moving along the sound source movement path.

[0200] In some embodiments of this disclosure, the sound playback unit 502 includes a simulated playback position determination subunit, a modulation signal generation subunit, and a control subunit. The simulated playback position determination subunit is used to determine the simulated playback position of the audio frame in the guide sound within the simulated sound movement path; the modulation signal generation subunit is used to generate a modulation signal to control the sound output of each speaker based on the simulated playback position, the audio frame, and the positions of each speaker in the stereo playback system; the control subunit is used to drive the corresponding speaker to operate using the modulation signal, so that the sound waves emitted by each speaker reverberate in the sound field to form the guide sound.

[0201] In some embodiments of this disclosure, the interactive control device 500 further includes an obstacle determination unit. The obstacle determination unit is used to determine whether there is an obstacle between the current display screen and the target display screen. Correspondingly, if there is an obstacle between the current display screen and the target display screen, the movement path determination unit determines a path that starts from the position of the current display screen, avoids the obstacle, and ends at the position of the target display screen as the simulated sound source movement path.

[0202] In some embodiments of this disclosure, the interactive control device 500 further includes an interactive image determination unit. The interactive image determination unit is used to determine whether the current display screen displays a first interactive image. The movement path determination unit includes a first position determination subunit and a movement path determination subunit. The first position determination subunit is used to determine a first display position of the first interactive image. The movement path determination subunit is used to determine a path starting from the first display position and ending at the position of the target display screen as the simulated sound source movement path.

[0203] In some embodiments of this disclosure, the first position determination subunit is further configured to determine a second display position of the second interactive image to be displayed on the target display screen. The movement path determination subunit determines a path starting from the first display position and ending at the second display position as the simulated sound source movement path.

[0204] In some embodiments of this disclosure, the determination unit includes an interaction operation acquisition subunit, a matching determination subunit, and an instruction determination subunit. The interaction operation acquisition subunit is used to acquire the interaction operation performed by the user when interacting with the current display screen. The matching determination subunit is used to determine whether the interaction operation triggers a specific playback task, where the specific playback task is a playback task associated with a cross-screen playback instruction.

[0205] The instruction determination subunit is used to determine whether a cross-screen playback instruction has been obtained when the interactive operation triggers a specific playback task.

[0206] In some disclosed embodiments, the interactive control device 500 further includes a sound determination unit. The sound determination unit is used to determine a corresponding prompt sound based on the specific playback task when the interactive operation triggers that task. Correspondingly, the sound playback unit 502 uses the prompt sound as a guide sound to control the stereo playback system to play.

[0207] In some embodiments of this disclosure, the interactive control device further includes a orientation determination unit and a display screen determination unit. The orientation determination unit is used to determine the user's focus direction. The display screen determination unit is used to determine the current display screen based on the focus direction.

[0208] In some embodiments of this disclosure, the orientation determination unit includes an image acquisition subunit, a processing subunit, and a direction determination subunit. The image acquisition subunit is used to acquire real-time captured user images, which are images including the user's eye features; the processing subunit is used to process the user images and determine the user's iris region and sclera region; the direction determination subunit is used to determine the relative direction of the user's gaze based on the area of ​​the sclera region located around the iris region, and to determine the direction of interest based on the relative direction of the gaze and the extrinsic parameters of the camera capturing the user images.

[0209] This disclosure also provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, can implement the interactive control method of any of the above embodiments.

[0210] Figure 6 This is a schematic diagram of the structure of an electronic device provided in some embodiments of this disclosure. See below for details. Figure 6 It shows a schematic diagram of a structure suitable for implementing the electronic device 600 in the embodiments of this disclosure. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0211] like Figure 6 As shown, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory ROM 602 or a program loaded from a storage device 608 into a random access memory RAM 603. The RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0212] Typically, the following devices can be connected to I / O interface 605: input devices 605 including, for example, touchscreens, touchpads, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0213] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0214] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0215] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0216] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0217] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0218] This disclosure also provides a vehicle, which includes a processor and a memory, the memory being used to store a computer program; when the computer program is loaded by the processor, it causes the processor to execute the previously described interactive control method.

[0219] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0220] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0221] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0222] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include, based on electrical connections of one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0223] This disclosure also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the methods of any of the above method embodiments. The execution method and beneficial effects are similar, and will not be described again here.

[0224] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0225] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interactive control method, characterized in that, include: Upon receiving a cross-screen playback instruction, the current display screen and the target display screen are determined based on the cross-screen playback instruction; Play a guide sound, which is a stereo sound simulating the sound source gradually moving from the current display screen position to the target display screen position.

2. The method according to claim 1, characterized in that, Before playing the introductory sound, the method includes: The simulated sound source movement path is determined based on the current display screen position and the target display screen position; The playback of the guiding sound includes: Play the guiding sound that simulates the movement of a sound source along the sound source's movement path.

3. The method according to claim 2, characterized in that, The playing of the guide sound, which simulates the movement of a sound source along the sound source's movement path, includes: Determine the simulated playback position of the audio frame in the guide sound within the simulated sound source's movement path; Based on the simulated playback position, the audio frame, and the positions of each speaker in the stereo playback system, a modulation signal is generated to control the sound output of each speaker; The modulation signal is used to drive the corresponding loudspeaker to work, so that the sound waves emitted by each loudspeaker reverberate in the sound field to form the guiding sound.

4. The method according to claim 2, characterized in that, Before determining the simulated sound source movement path based on the position of the current display screen and the position of the target display screen, the process includes: Determine whether there is an obstacle between the current display screen and the target display screen; If there is an obstacle between the current display screen and the target display screen, the path that starts from the position of the current display screen, avoids the obstacle, and ends at the position of the target display screen is determined as the simulated sound source movement path.

5. The method according to claim 1, characterized in that, Before playing the introductory sound, the method further includes: Determine whether the current display screen shows the first interactive image; When the first interactive image is displayed on the current display screen, determine the first display position of the first interactive image; The path starting from the first display position and ending at the position of the target display screen is determined as the simulated sound source movement path.

6. The method according to claim 5, characterized in that, If it is determined that the current display screen is displaying the first interactive image, the method further includes: Determine the second display position of the second interactive image to be displayed on the target display screen; Determining the path from the first display position to the target display position as the simulated sound source movement path includes: The path starting from the first display position and ending at the second display position is determined as the simulated sound source movement path.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Get the interaction operations performed when the user interacts with the current display screen; Determine whether the interactive operation is an operation that triggers a specific playback task, wherein the specific playback task is a playback task associated with the cross-screen playback instruction; When the interactive operation is an operation that triggers the specific playback task, it is determined that the cross-screen playback instruction has been obtained.

8. The method according to claim 7, characterized in that, The method further includes: When the interactive operation is an operation that triggers the specific playback task, a corresponding prompt sound is determined according to the specific playback task; Controlling the stereo playback system to play guiding sounds that gradually move the simulated sound source from the current display screen position to the target display screen position includes: The prompt sound is used as the guide sound to control the stereo playback system to play.

9. The method according to claim 1, characterized in that, The method further includes: Determine the user's focus; The current display screen is determined based on the direction of attention.

10. The method according to claim 9, characterized in that, Determining the user's focus includes: Acquire real-time captured user images, which include images of the user's eye features; Process the user image to determine the user's iris region and sclera region; The relative direction of the user's gaze is determined based on the area of ​​the sclera region located around the periphery of the iris region; The direction of attention is determined based on the relative direction of the line of sight and the extrinsic parameters of the camera that captures the user's image.

11. An interactive control device, characterized in that, include: A screen determination unit is used to determine the current display screen and the target display screen according to the cross-screen playback instruction when a cross-screen playback instruction is obtained; The sound playback unit is used to play a guiding sound, which is a stereo sound simulating the sound source position gradually moving from the current display position to the target display position.

12. An electronic device, characterized in that, Includes a processor and a memory, the memory being used to store computer programs; When the computer program is loaded by the processor, it causes the processor to execute the interactive control method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, causes the processor to implement the interactive control method as described in any one of claims 1-10.

14. A vehicle, characterized in that, Includes a processor and a memory, the memory being used to store computer programs; When the computer program is loaded by the processor, it causes the processor to execute the interactive control method as described in any one of claims 1-10.

Images