Virtual voice image interaction method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of intelligent automobiles, and provides a virtual voice image interaction method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a voice instruction; according to the voice instruction, collecting a scene in front of the vehicle in a vehicle parking state and converting the scene into a first image; acquiring a preset virtual voice image and audio according to the voice instruction, and analyzing and fusing the virtual voice image and the first image to generate a second image; projecting the second image on a front windshield displayable area of the vehicle; and playing an audio corresponding to the second image according to an audio playing device of the vehicle. Therefore, the scene in front of the vehicle and the virtual voice image are combined and projected on the front windshield of the vehicle, so that the problems that the traditional virtual voice image is inconvenient to observe and interact and the display space is limited to a large central control screen of the vehicle are solved, and the immersion and interestingness of interaction between a user and the virtual voice image are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent automobile technology, and in particular to a virtual voice image interaction method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of smart cockpits and voice technology, voice-based command interaction is becoming more and more popular. The car system has launched a personalized voice assistant virtual image, which brings users a more humanized interactive experience.

[0003] Currently, the mainstream car systems on the market limit the display of voice images to the central control screen. However, since the central control screen is small in size and is often located between the driver and the front passenger, the user's actual observation angle is poor and the interactive experience is poor. Summary of the invention

[0004] To solve the above problems, the present invention provides a virtual voice image interaction method, device, electronic device and storage medium to solve the problem that traditional virtual voice image observation and interaction are inconvenient and the display space is limited to the large screen of the vehicle's central control, thereby enhancing the immersion and fun of the user's interaction with the virtual voice image, thereby enhancing the interactive experience of the virtual voice image.

[0005] The first aspect of the present disclosure provides a virtual voice image interaction method, comprising: obtaining a voice command; according to the voice command, collecting the scene in front of the vehicle when the vehicle is parked and converting it into a first image; according to the voice command, obtaining a preset virtual voice image and audio, analyzing and fusing the virtual voice image with the first image to generate a second image; projecting the second image onto a displayable area of ​​the front windshield of the vehicle; and playing audio corresponding to the second image according to an audio playback device of the vehicle.

[0006] With such a configuration, the scene in front of the vehicle can be collected as the first image by initiating a voice command, and the first image is analyzed and fused with a preset virtual voice image to generate a second image, so that the second image is generated by the virtual voice image based on the scene in front of the vehicle, further making the second image more flexible and meeting the real-time picture requirements of the vehicle scene, and helping to make the second image more intelligent and easier to observe after being projected on the front windshield of the vehicle, and playing the sound of the second image through an audio playback device to solve the problem that traditional virtual voice image observation and interaction are inconvenient, and the display space is limited to the large central control screen of the vehicle computer, thereby enhancing the immersion and fun of the user's interaction with the virtual voice image.

[0007] In some embodiments, collecting the scene in front of the vehicle and converting it into a first image when the vehicle is parked according to the voice command includes: performing data analysis on the first image to determine whether there is a main object in the scene in front of the vehicle; if there is a main object, obtaining the type, position parameters and scene brightness of the main object, and converting the scene in front of the vehicle into the first image according to the type, position parameters and scene brightness of the main object; if there is no main object, directly converting the scene in front of the vehicle into the first image.

[0008] With such a configuration, the first image is analyzed to determine whether there is a main object in the scene in front of the vehicle. If there is a main object, the second image is generated on the front windshield of the vehicle based on the type, position parameters and scene brightness of the main object, avoiding the main object to prevent the main object from blocking the second image, thereby achieving flexible playback of the second image. If there is no main object, the second image is played directly, thereby effectively avoiding the influence of the type, position and scene brightness of the main object on the projection of the second image.

[0009] In some embodiments, if the main object exists, the type, position parameters and scene brightness of the main object are obtained, and the scene in front of the vehicle is converted into a first image according to the type, position parameters and scene brightness of the main object, including: the type of the main object includes buildings, trees and mountains; the position parameters of the main object include an area near the main driver and an area near the co-driver on the front windshield of the vehicle; the scene brightness includes a first brightness level, a second brightness level and a third brightness level, the first brightness level is greater than the second brightness level, and the second brightness level is greater than the third brightness level.

[0010] With such a setting, by taking buildings, trees and mountains as the main objects, it is possible to effectively avoid obstruction of the second image projected on the vehicle's windshield. Combined with the position of the main object, the projection of the second image can avoid the position of the main object on the vehicle's windshield. At the same time, the brightness of the second image on the vehicle's windshield can be adjusted according to the scene brightness, so as to improve the clarity of the second image on the vehicle's windshield, thereby facilitating the projection of a clearer second image.

[0011] In some embodiments, obtaining a preset virtual voice image and audio according to the voice command, and analyzing and fusing the virtual voice image with the first image to generate a second image includes: if there is no main object in the scene in front of the vehicle, obtaining whether the sound source comes from the driver's seat, the co-driver's seat or the back row according to the voice command; if the sound source comes from the driver's seat, the second image is located in the area on the front windshield of the vehicle facing the driver's seat; if the sound source comes from the co-driver's seat, the second image is located in the area on the front windshield of the vehicle facing the co-driver's seat; if the sound source comes from the back row, the second image is located in the area on the front windshield of the vehicle facing the central control screen of the vehicle.

[0012] With such a setting, when there is no main object in the scene in front of the vehicle, the display position of the second image on the front windshield of the vehicle can be changed according to the position of the sound source that issues the voice command, so that the user at the sound source position can view the second image after it is projected, thereby enhancing the immersion and fun of the user's interaction with the virtual voice image.

[0013] In some embodiments, obtaining a preset virtual voice image and audio according to the voice command, and analyzing and fusing the virtual voice image with the first image to generate a second image includes: if the main object is in the scene in front of the vehicle, obtaining whether the sound source comes from the driver's seat, the co-driver's seat or the back row according to the voice command; if the sound source comes from the driver's seat, the second image is located in the area of ​​the front windshield of the vehicle close to the driver's seat and avoids the main object; if the sound source comes from the co-driver's seat, the second image is located in the area of ​​the front windshield of the vehicle close to the co-driver's seat and avoids the main object; if the sound source comes from the back row, the second image is located in the area of ​​the front windshield of the vehicle close to the vehicle's central control screen.

[0014] With such a configuration, when there is a main object in the scene in front of the vehicle, the position where the second graphic is displayed on the front windshield of the vehicle is avoided by the main object based on the position of the sound source that issues the voice command and in combination with the situation of the main object, so as to prevent the main object from blocking the projection of the second image. This helps to move the second image away from the main object while approaching the sound source, which helps the user who issues the voice command to interact with the virtual voice image.

[0015] In some embodiments, if the sound source comes from the main driver's seat, the second image is located in the area close to the main driver's seat on the front windshield of the vehicle and avoids the main object, including; when the second image avoids the main object and approaches the co-driver's seat of the vehicle, the second image is deflected toward the position of the main driver's seat of the vehicle.

[0016] With this arrangement, when the user in the main driver's seat issues a voice command, the second image is in the area of ​​the vehicle's front windshield, close to the main driver's position, avoiding the main object, and facing the vehicle's main driver's seat, so that the user in the main driver's seat can face the virtual voice image on the vehicle's front windshield, thereby facilitating the interaction between the user in the main driver's seat and the virtual voice image.

[0017] In some embodiments, if the sound source comes from the co-pilot, the second image is located in the area close to the co-pilot on the front windshield of the vehicle and avoids the main object, including: when the second image avoids the main object and approaches the main driver of the vehicle, the second image is deflected toward the position of the co-pilot of the vehicle.

[0018] With this arrangement, when the user on the passenger seat issues a voice command, the second image is in the area of ​​the vehicle's front windshield, close to the passenger seat, avoiding the main object, and facing the passenger seat of the vehicle, so that the user on the passenger seat can face the virtual voice image on the vehicle's front windshield, thereby facilitating the interaction between the user on the passenger seat and the virtual voice image.

[0019] The second aspect of the present disclosure provides a virtual voice image interaction device, which includes: a sound source processing module for obtaining voice commands; an image acquisition module for collecting the scene in front of the vehicle in a parking state according to the voice commands and converting it into a first image; an image generation and processing module for obtaining a preset virtual voice image and audio according to the voice commands; analyzing and fusing the virtual voice image with the first image to generate a second image; a projection module for projecting the second image onto a displayable area of ​​the front windshield of the vehicle; and a sound broadcast module for playing audio corresponding to the second image according to the audio playback device of the vehicle.

[0020] The technical effects brought about by any possible implementation method of the second aspect can refer to the technical effects brought about by the above-mentioned first aspect, and will not be repeated here.

[0021] The third aspect of the present disclosure provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing a virtual voice image interaction method as described in the first aspect when executing the computer program.

[0022] The technical effects brought about by any possible implementation of the third aspect can refer to the technical effects brought about by the above-mentioned first aspect, and will not be repeated here.

[0023] A fourth aspect of the present disclosure provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, a virtual voice image interaction method as described in the first aspect is implemented.

[0024] The technical effects brought about by any possible implementation of the fourth aspect can refer to the technical effects brought about by the above-mentioned first aspect, and will not be repeated here.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] Parameters such as the size, position and brightness of the virtual voice image will be adjusted accordingly based on the relevant parameter information of the main object in the first image; the position and rotation angle of the virtual voice image will be adjusted accordingly based on the sound source positioning of the voice command, thereby enhancing the immersion and fun of the user's interaction with the virtual voice image.

[0027] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or be understood by implementing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 is a flow chart of a virtual voice image interaction method provided by an embodiment of the present disclosure;

[0030] Figure 2 is a schematic diagram of the corresponding relationship between the ambient brightness of the first image and the brightness level of the virtual voice image provided by the embodiment of the present disclosure;

[0031] Figure 3 It is a schematic diagram of a default projection area of ​​a virtual voice image on a windshield provided by an embodiment of the present disclosure;

[0032] Figure 4 It is a schematic diagram of the offset zoom area of ​​the projection position of the virtual voice image on the windshield provided by the embodiment of the present disclosure;

[0033] Figure 5 is a schematic diagram of a model rotation angle offset corresponding to a position offset of a virtual voice image provided by an embodiment of the present disclosure;

[0034] Figure 6 It is a structural block diagram of a virtual voice image interaction device provided by an embodiment of the present disclosure;

[0035] Figure 7is a schematic diagram of an image of a virtual voice image projected on a passenger seat provided by an embodiment of the present disclosure;

[0036] Figure 8 It is a schematic diagram of an image of a virtual language image projected on a main vehicle provided by an embodiment of the present disclosure;

[0037] Fig. 9 It is a structural block diagram of an electronic device 3000 provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0039] The following is a brief introduction to some concepts that may be involved in the embodiments of the present disclosure.

[0040] HUD (Head Up Display) technology can project important driving information into the driver's front field of view in the form of virtual images, allowing the driver to quickly obtain the required information while maintaining normal driving vision.

[0041] With the rapid development of HUD technology, how to use this technology to enhance the entertainment expansion of smart cockpits has become a hot topic in current research. At present, the mainstream car system in the market limits the display of voice images to the central control screen. Since the central control screen is small in size and is often located between the driver and the co-driver, the user's actual viewing angle is poor and the interactive experience is poor.

[0042] To this end, the present disclosure provides a virtual voice image interaction method, such as Figure 1 As shown, Figure 1 This is a flow chart of a virtual voice image interaction method provided by an embodiment of the present disclosure, the method comprising the following steps:

[0043] S100: Acquire voice instructions.

[0044] In this embodiment, the vehicle is aroused to interact according to the user's voice command, and the voice command includes the user's voice information and the sound source location information.

[0045] S200: According to the voice command, a scene in front of the vehicle is captured when the vehicle is parked and converted into a first image.

[0046] In this embodiment, after the scene in front of the vehicle is converted into the first image, data analysis is performed on the first image to determine whether there is a subject in the scene in front of the vehicle. If there is a subject, the type, position parameters and scene brightness of the subject are obtained, and the scene in front of the vehicle is converted into the first image according to the type, position parameters and scene brightness of the subject; if there is no subject, the scene in front of the vehicle is directly converted into the first image.

[0047] It can be understood that the first image can be a scene with no obstructions in front of the vehicle, or a scene with obstructions in front of the vehicle, and recording and converting the data of the obstructed image can help plan the visible area of ​​the vehicle's front windshield in combination with the data of the first image.

[0048] Exemplarily, the types of main objects include buildings, trees and mountains; the location parameters of the main objects include the area near the driver's seat and the area near the co-driver's seat on the front windshield of the vehicle; the scene brightness includes a first brightness level, a second brightness level and a third brightness level, the first brightness level is greater than the second brightness level, and the second brightness level is greater than the third brightness level.

[0049] For example, Figure 2 As shown, Figure 2 : is a schematic diagram of the corresponding relationship between the ambient brightness of the first image and the brightness level of the virtual voice image provided by the embodiment of the present disclosure. The first image is the image file A in the figure, and the first brightness level is shown as Figure 2 The brightness level is 76-100; the second brightness level is shown as Figure 2 The darker and brighter levels are 51-75 for the brighter level and 26-50 for the darker level. The third brightness level is shown as Figure 2 Medium dark, brightness level is 0-25.

[0050] S300: Obtain a preset virtual voice image and audio according to a voice instruction, and analyze and fuse the virtual voice image with the first image to generate a second image.

[0051] In this embodiment, the position and rotation angle of the virtual voice image will be adjusted accordingly according to the sound source positioning of the voice command, so that the user can clearly see the virtual voice image at the location where the command is issued. The size, position, brightness and other parameters of the virtual voice image will be adjusted accordingly according to the relevant parameter information of the main object in the first image, so that after the user issues the command, the virtual voice image can form a complete and clear second image according to the relevant parameter information of the main object in the first image, and display it on the front windshield of the vehicle.

[0052] Virtual voice image interactive animation is a set of voice image animation clips built into the 3D engine program, which are mainly divided into two categories: command feedback and emotional feedback. Different voice commands call different animation clips. For example, when the user issues a command: "Turn on the air conditioner", the program will call the ok gesture action animation clip in the library to express the meaning of completing the command and display it at the target location.

[0053] For example, Figure 3 As shown, Figure 3 It is a schematic diagram of the default projection area of ​​the virtual voice image on the windshield provided by the embodiment of the present disclosure. If there is no main object in the scene in front of the vehicle, the initial default position of the projection area of ​​the virtual voice image on the windshield. According to the voice command, it is obtained whether the sound source comes from the main driver, the co-driver or the back row; if the sound source comes from the main driver, the second image is located in the area facing the main driver on the front windshield of the vehicle; if the sound source comes from the co-driver, the second image is located in the area facing the co-driver on the front windshield of the vehicle; if the sound source comes from the back row, the second image is located in the area facing the central control screen of the vehicle on the front windshield of the vehicle.

[0054] For example, Figure 4 As shown, Figure 4 It is a schematic diagram of the offset zoom area of ​​the projection position of the virtual voice image on the windshield provided by the embodiment of the present disclosure. If there is a main object in the scene in front of the vehicle, the sound source is obtained from the main driver, the co-driver or the back row according to the voice command; if the sound source comes from the main driver, the second image is located in the area near the main driver on the front windshield of the vehicle and avoids the main object; if the sound source comes from the co-driver, the second image is located in the area near the co-driver on the front windshield of the vehicle and avoids the main object; if the sound source comes from the back row, the second image is located in the area near the central control screen of the vehicle on the front windshield of the vehicle.

[0055] For example, when there is a main object in the scene in front of the vehicle, the display position of the virtual voice image awakened on the main driver's side is the default initial position, and the size is the initial default initial size (such as Figure 3 When a building that occupies a large space on the left side of the front windshield display area is detected in the first image, the display position of the virtual voice image awakened on the main driver's side is moved to the right accordingly and is reduced accordingly (as shown in FIG. Figure 4 When the ambient brightness of the first image scene is detected to be dark, the brightness of the virtual voice image is adjusted to brightness level B accordingly, wherein the corresponding relationship between the ambient brightness of the first image and the brightness level of the virtual voice image is as shown in FIG. Figure 2 shown.

[0056] For example, Figure 4 and Figure 5 As shown, Figure 5It is a schematic diagram of the model rotation angle offset corresponding to the position offset of the virtual voice image provided by the embodiment of the present disclosure. If the sound source comes from the main driver's seat, when the second image avoids the main object and approaches the co-driver's seat of the vehicle, the second image is deflected 30° toward the position of the main driver's seat of the vehicle and is reduced. If the sound source comes from the co-driver's seat, when the second image avoids the main object and approaches the main driver's seat of the vehicle, the second image is deflected 30° toward the position of the co-driver's seat of the vehicle and is reduced. If the sound source comes from the back row, the second image is located in the middle area of ​​the windshield and is enlarged.

[0057] For example, Figure 2 and Figure 4 As shown, the brightness level of the second image corresponds to and is the same as the brightness level of the scene, that is, Figure 2 The brightness level division of the virtual voice image in the image is the same as the brightness level division of the first image (image file A), the first brightness level corresponds to level A, the second brightness level corresponds to level B, and the third brightness level corresponds to level C, so that after the second image is projected on the windshield, the brightness of the second image is higher than that of the first image, so that the second image can be projected clearly.

[0058] S400: Projecting a second image onto a displayable area of ​​a front windshield of the vehicle.

[0059] In this embodiment, the second image is projected onto the displayable area of ​​the vehicle's windshield through HUD technology, so that the processed second image can be clearly and completely projected on the vehicle's windshield, so that the virtual voice image can be played on the vehicle's windshield.

[0060] S500: Play the audio corresponding to the second image according to the audio playback device of the vehicle.

[0061] In this embodiment, while the second image is projected on the front windshield of the vehicle, the audio of the preset virtual voice image is played through the vehicle's audio playback device to achieve interaction between the virtual voice image and the user, thereby enhancing the user's sense of immersion in the interaction with the virtual voice image.

[0062] Based on the above method, the embodiment of the present disclosure also provides a virtual voice image interaction device 2000 corresponding to the above method, such as Figure 6 As shown, Figure 6 It is a structural block diagram of a virtual voice image interaction device provided in an embodiment of the present disclosure, and the device includes a sound source processing module 210, an image acquisition module 220, an image analysis module 230, an image generation processing module 240, a projection module 250, a sound broadcasting module 260 and a human-computer interaction module 270.

[0063] Among them, the sound source processing module 210 is used to obtain voice commands, receive voice commands and perform sound source positioning to obtain the position information of the speaker, and the position information of the speaker includes the main driver, the co-driver and the back seat.

[0064] The image acquisition module 220 is used to capture the scene in front of the vehicle and convert it into a first image according to the voice command when the vehicle is parked. It should be noted that the image acquisition module 220 is mainly composed of a camera and lighting equipment, and is responsible for collecting the image data of the scene in front of the vehicle and converting it into an image file, which is transmitted to the cloud server for processing.

[0065] The image analysis module 230 is used to identify and analyze the first image collected, obtain information such as the type, position, and ambient brightness of the main object in the image scene, and output the parameter information of the type, position, and ambient brightness of the main object to the image generation processing module 240. It should be noted that the identification and analysis of the first image collected depends on image recognition technology. Image recognition technology is based on cloud computing services that integrate cloud computing + big data + artificial intelligence, and uses deep learning and large-scale image training to accurately identify objects, positions, and ambient brightness in the picture. It can be understood that the main identification includes objects such as buildings, trees, and mountains that may cause large-scale occlusion of the virtual voice image, so as to adjust the position, size, and rotation angle of the virtual voice image during image generation processing. The main identification scene ambient brightness includes multiple brightness levels, so as to adjust the display brightness of the virtual voice image during image generation processing.

[0066] The image generation processing module 240 is used to obtain preset virtual voice image and audio according to voice commands, receive parameter information of the first image and sound source data of the voice command, analyze and fuse the virtual voice image with the first image to generate a second image; in the fusion algorithm, the size, position and rotation angle parameters of the virtual voice image will be processed and adjusted according to the position parameters of the main object in the first image and the sound source position data of the voice command.

[0067] For example, voice commands are acquired and sound source localization is performed, and the sound source location data is output to the image generation processing module 240. Sound source localization can accurately identify whether the voice command comes from the driver's seat, the co-driver's seat, or the back seat, so as to adjust the position and rotation angle of the voice image during the image generation processing, such as Figure 7 and Figure 8 As shown, Figure 7 is a schematic diagram of an image of a virtual voice image projected on the passenger seat provided by an embodiment of the present disclosure, Figure 8 Schematic diagram of an image of a virtual language image projected on the main driver's seat provided by an embodiment of the present disclosure. Figure 7 In the car, the co-pilot gave a voice command, and the virtual voice image was displayed on the co-pilot side and avoided obstacles with strong interference. Figure 8In the process, the driver issues a voice command, and the virtual voice image is displayed on the driver's side and avoids obstacles with strong interference.

[0068] For example, image processing and generation algorithms rely on artificial intelligence technology. Data analysis is performed through artificial neural networks, which consist of an input layer, an intermediate layer, and an output layer. In theory, any nonlinear function can be infinitely fitted through a three-layer neural network. The advantages of the artificial neural network recognition method are that it is insensitive to noise and deformation of characters and has strong fault tolerance. Figure 7 In the video, the scene is an outdoor sunny environment. The obstruction in the box on the right side of the picture is a tree. The virtual voice image avoids the tree and is displayed in a relatively open area with a brighter display brightness. Figure 8 In the figure, the scene is an outdoor cloudy environment in the evening. The obstruction in the box on the side of the picture is a building. The virtual voice image avoids the obstruction of the building and is displayed in a relatively open area, and the display brightness is dim.

[0069] The projection module 250 is used to project the second image onto the displayable area of ​​the front windshield of the vehicle. The fused second image is projected onto the windshield in the direction of the speaker through the HUD technology, and the feedback audio corresponding to the interactive animation is played in TTS voice.

[0070] The sound broadcast module 260 is used to play the audio corresponding to the second image according to the audio playback device of the vehicle, mainly by playing TTS (Text-to-Speech, text-to-speech technology) voice feedback. In intelligent voice interaction, TTS voice feedback can convert the user's voice command into text for processing, and feedback the processing results to the user in the form of voice. The sound broadcast can also be regarded as a part of the human-computer interaction module 270 in the smart cockpit for outputting audio. The HUD technology can project important driving information in the form of a virtual image into the driver's front field of view. For example, the user issues a command: "Please open the main driving window". When the car program completes the command, the TTS voice feedback will give a broadcast feedback in the form of voice: "OK, it's open."

[0071] The human-computer interaction module 270 is the basic configuration of the smart cockpit, which is used to receive voice commands and execute the animation sequence of the preset virtual voice image, and transmit the animation image to the image generation processing module 240.

[0072] In summary, according to the virtual voice image interaction method and device provided in the above embodiments, the scene image in front of the vehicle is obtained; the type and position of the object contained in the image and the ambient brightness and other information are analyzed; the position information of the speaker is obtained through sound source positioning; the animation sequence of the virtual voice image is obtained through human-computer interaction; a new fused image is generated through an image algorithm; the new fused image is projected onto the windshield in the direction of the speaker, and the feedback audio corresponding to the interactive animation is played by TTS voice. At this point, the voice interaction in the cockpit will become more immersive and interesting, and the user experience will be enhanced.

[0073] Based on the same inventive concept as the above disclosed content, the present disclosure also provides an electronic device 3000. Fig. 9 As shown, Fig. 9 1 is a structural block diagram of an electronic device 3000 provided by an embodiment of the present disclosure. The electronic device 3000 of the embodiment of the present disclosure includes at least one processor 310 and at least one memory 320 that are electrically connected. The memory 320 is electrically connected to the processor 310, wherein the memory 320 stores instructions that can be executed by the at least one processor 310, and the instructions are executed by the at least one processor 310 so that the at least one processor 310 can execute the method described above.

[0074] It should be noted that the electrical connection between the above-mentioned units does not necessarily mean the connection between lines. An indirect connection method can be applied to the embodiments of the present disclosure as long as the purpose of the present disclosure is achieved.

[0075] Based on the same inventive concept, the present disclosure also provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described above is performed. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0076] Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A virtual voice image interaction method, characterized in that: The method comprises: Get voice commands; According to the voice command, the scene in front of the vehicle is captured and converted into a first image when the vehicle is parked; Acquire a preset virtual voice image and audio according to the voice command, and analyze and fuse the virtual voice image with the first image to generate a second image; Projecting the second image onto a displayable area of ​​a front windshield of the vehicle; The audio corresponding to the second image is played according to the audio playing device of the vehicle.

2. The method according to claim 1, characterized in that The collecting the scene in front of the vehicle and converting it into a first image according to the voice command in a parking state of the vehicle comprises: Performing data analysis on the first image to determine whether there is a main object in the scene in front of the vehicle; If the subject object exists, obtaining the type, position parameters and scene brightness of the subject object, and converting the scene in front of the vehicle into a first image according to the type, position parameters and scene brightness of the subject object; If the main object does not exist, the scene in front of the vehicle is directly converted into a first image.

3. The method according to claim 2, characterized in that If the subject object exists, obtaining the type, position parameters and scene brightness of the subject object, and converting the scene in front of the vehicle into a first image according to the type, position parameters and scene brightness of the subject object includes: The types of the subject objects include buildings, trees and mountains; The position parameters of the main object include an area close to the main driver's seat and an area close to the co-driver's seat on the front windshield of the vehicle; The scene brightness includes a first brightness level, a second brightness level, and a third brightness level, the first brightness level is greater than the second brightness level, and the second brightness level is greater than the third brightness level.

4. The method according to claim 2, characterized in that: The step of acquiring a preset virtual voice image and audio according to the voice command, and analyzing and fusing the virtual voice image with the first image to generate a second image comprises: If there is no subject in the scene in front of the vehicle, obtaining, according to the voice command, whether the sound source is from the driver's seat, the co-driver's seat, or the rear seat; If the sound source comes from the driver's seat, the second image is located in the area of ​​the front windshield of the vehicle facing the driver's seat; If the sound source comes from the passenger seat, the second image is located in the area of ​​the front windshield of the vehicle facing the passenger seat; If the sound source comes from the back row, the second image is located in the area on the front windshield of the vehicle facing the central control screen of the vehicle.

5. The method according to claim 2, characterized in that: The step of acquiring a preset virtual voice image and audio according to the voice command, and analyzing and fusing the virtual voice image with the first image to generate a second image comprises: If the subject is in the scene in front of the vehicle, obtaining, according to the voice command, whether the sound source is from the driver's seat, the co-driver's seat, or the back seat; If the sound source comes from the main driver's seat, the second image is located on the front windshield of the vehicle near the main driver's seat and avoids the main object; If the sound source comes from the passenger seat, the second image is located in the area near the passenger seat on the front windshield of the vehicle and avoids the main object; If the sound source comes from the back row, the second image is located in an area on the front windshield of the vehicle close to the central control screen of the vehicle.

6. The method according to claim 5, characterized in that If the sound source comes from the main driver's seat, the second image is located in the area of ​​the front windshield of the vehicle close to the main driver's seat and avoids the main object, including: When the second image avoids the main object and approaches the co-driver's seat of the vehicle, the second image deflects toward the position of the main driver's seat of the vehicle.

7. The method according to claim 5, characterized in that If the sound source comes from the front passenger seat, the second image is located in an area close to the front passenger seat on the front windshield of the vehicle and avoids the main object, including: When the second image avoids the main object and approaches the main driver's seat of the vehicle, the second image deflects toward the position of the auxiliary driver's seat of the vehicle.

8. A virtual voice image interaction device, characterized in that: The device comprises: The sound source processing module is used to obtain voice commands; The image acquisition module is used to acquire the scene in front of the vehicle in a parking state according to the voice command and convert it into a first image; The image generation and processing module is used to obtain a preset virtual voice image and audio according to the voice command; analyze and fuse the virtual voice image with the first image to generate a second image; The projection module is used to project the second image onto a displayable area of ​​the front windshield of the vehicle; The sound broadcast module is used to play the audio corresponding to the second image according to the audio playback device of the vehicle.

9. An electronic device, characterized in that: The electronic device comprises: Memory for storing computer programs; A processor, used to implement a virtual voice image interaction method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the virtual voice image interaction method according to any one of claims 1 to 7 is implemented.

Images