Audio content projection method and device, equipment and storage medium

Abstract

The invention relates to an audio content projection method and device, equipment and a storage medium, and is applied to the technical field of intelligent interaction. The method comprises the steps that in response to user audio of a target type, target projection content corresponding to the user audio and a target projection assembly in a vehicle are determined, and the user audio of the target type represents that the user audio is audio showed out of the vehicle by a user in the vehicle; determining an out-of-vehicle projection position corresponding to the target projection content; and controlling the target projection component to project the target projection content to the projection position outside the vehicle, and playing the user audio outside the vehicle. According to the method, the target projection content and the target projection component corresponding to the user audio showed out of the vehicle by the user in the vehicle and the projection position outside the vehicle corresponding to the target projection content are determined, the target projection content is projected, and the user audio is played outside the vehicle; the method can transmit the call content of the user in the vehicle to the person outside the vehicle through projection in a noisy environment, and improves the efficiency and accuracy of transmitting information to the outside of the vehicle by the user in the vehicle.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent interactive technology, and in particular to an audio content projection method, apparatus, device, and storage medium. Background Technology

[0002] When a vehicle is moving or parked, occupants may need to communicate with pedestrians or other vehicle occupants outside, such as reminding pedestrians to be careful or coordinating parking locations with other vehicles. Currently, this communication is primarily achieved by the occupant speaking to others outside the vehicle. In quiet environments, those outside can clearly hear the occupant's messages, enabling effective communication. However, in noisy environments, the noise can interfere with communication, making it difficult for those outside to hear clearly. This not only affects communication efficiency but may also lead to misunderstandings due to poor information transmission, potentially causing traffic hazards. Therefore, improving the efficiency and accuracy of information transmission from the occupant to others outside the vehicle is a technical problem that needs to be solved. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides an audio content projection method, apparatus, device, and storage medium.

[0004] A first aspect of this disclosure provides an audio content projection method applied to a vehicle, the method comprising: In response to user audio of target type, determine the target projection content corresponding to the user audio and the target projection component in the vehicle. User audio of target type means that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle. Determine the external projection position of the target projection content; The target projection component is controlled to project the target content to the projection position outside the vehicle and play the user's audio outside the vehicle.

[0005] In some embodiments, before determining the target projection content corresponding to the user audio and the target projection component in the vehicle in response to the user audio of the target type, the method includes: Based on whether the shout function is enabled, determine the matching priority of at least one intent type, wherein at least one intent type includes a target type, and when the shout function is enabled, the matching priority of the target type is the highest priority; Based on the matching priority corresponding to each intent type, the user audio is matched with each intent type to determine the intent type of the user audio as the target type.

[0006] In some embodiments, determining the external projection position corresponding to the target projection content includes: If the current projection content of the target projection component includes a projected image, determine the external projection position of the target projection content based on the projected image and the target projection content. And / or, If the current projection content of the target projection component does not include a projected image, the external projection position of the target projection component is determined based on the correspondence between the projection component and the preset projection position.

[0007] In some embodiments, determining the external projection position corresponding to the target projection content based on the projected image and the target projection content includes: Based on the projected image and the target projected content, determine the embedding position of the target projected content in the projected image; The embedded position is transformed based on the first transformation relationship between the image coordinate system and the first coordinate system to obtain the external projection position of the vehicle. The first coordinate system is the coordinate system corresponding to the target projection component.

[0008] In some embodiments, after determining the external projection position corresponding to the target projection content, the method further includes: The external projection position is transformed based on the second transformation relationship between the first coordinate system and the second coordinate system to obtain the interface display position of the target projection content in the car model display interface. The first coordinate system is the coordinate system corresponding to the target projection component, and the second coordinate system is the coordinate system corresponding to the vehicle 3D model in the car model display interface. The target projection content is displayed on the car model display interface on the in-vehicle screen based on the interface display position.

[0009] In some embodiments, after displaying the target projected content in the vehicle model display interface shown on the in-vehicle screen based on the interface display position, the method further includes: In response to the detection of a projection position adjustment operation in the car model display interface, the target projection content in the car model display interface is adjusted to the target display position for display. The target display position is the display position corresponding to the projection position adjustment operation. The target display position is transformed based on the second transformation relationship to obtain the target vehicle external projection position; Control the target projection component to project the target content to the projection position outside the target vehicle.

[0010] In some embodiments, the target projection content includes text content, and before controlling the target projection component to project the target projection content to the projection position outside the vehicle, the method further includes: Determine the projection style information of the text content based on the semantic information of the text content; Controlling the target projection component to project target content to a projection position outside the vehicle includes: The control target projection component projects the target content to the projection position outside the vehicle according to the projection style information.

[0011] A second aspect of this disclosure provides an audio content projection device applied to a vehicle, the device comprising: The first determining module is used to determine the target projection content corresponding to the user audio and the target projection component in the vehicle in response to the user audio of the target type. The user audio of the target type means that the user audio is the audio of the user inside the vehicle shouting to the outside of the vehicle. The second determining module is used to determine the external projection position of the target projection content; The first control module is used to control the target projection component to project the target content to the projection position outside the vehicle and play the user's audio outside the vehicle.

[0012] In some embodiments, the audio content projection device further includes: The third determining module is used to determine the matching priority of at least one intent type based on whether the shouting function is enabled. The at least one intent type includes a target type. When the shouting function is enabled, the matching priority of the target type is the highest priority. The matching module is used to match user audio with each intent type based on the matching priority corresponding to each intent type, and determine the intent type of the user audio as the target type.

[0013] In some embodiments, the second determining module is specifically used to determine the vehicle exterior projection position corresponding to the target projection content based on the projected image and the target projection content when the current projection content of the target projection component includes a projected image; and to determine the vehicle exterior projection position corresponding to the target projection component based on the correspondence between the projection component and the preset projection position when the current projection content of the target projection component does not include a projected image.

[0014] In some embodiments, the second determining module is specifically used to determine the embedding position of the target projection content in the projection image based on the projection image and the target projection content; and to transform the embedding position based on the first transformation relationship between the image coordinate system and the first coordinate system to obtain the external projection position, wherein the first coordinate system is the coordinate system corresponding to the target projection component.

[0015] In some embodiments, the audio content projection device further includes: The first transformation module is used to transform the external projection position of the vehicle based on the second transformation relationship between the first coordinate system and the second coordinate system to obtain the interface display position of the target projection content in the vehicle model display interface. The first coordinate system is the coordinate system corresponding to the target projection component, and the second coordinate system is the coordinate system corresponding to the vehicle 3D model in the vehicle model display interface. The display module is used to display the target projection content on the car model display interface displayed on the in-vehicle screen based on the interface display position.

[0016] In some embodiments, the audio content projection device further includes: The adjustment module is used to respond to the detection of a projection position adjustment operation in the car model display interface, and adjust the target projection content in the car model display interface to the target display position for display. The target display position is the display position corresponding to the projection position adjustment operation. The second conversion module is used to convert the target display position based on the second conversion relationship to obtain the target vehicle exterior projection position; the second control module is used to control the target projection component to project the target projection content to the target vehicle exterior projection position.

[0017] In some embodiments, the target projection content includes text content, and the audio content projection device further includes: The fourth determining module is used to determine the projection style information of the text content based on the semantic information of the text content; The third control module is used to control the target projection component to project the target content to the projection position outside the vehicle according to the projection style information.

[0018] A third aspect of this disclosure provides a computer device including a memory and a processor, and a computer program, wherein the memory stores the computer program, and when the computer program is executed by the processor, it implements the audio content projection method of the first aspect described above.

[0019] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the audio content projection method of the first aspect described above.

[0020] The technical solution provided in this disclosure has the following advantages compared with the prior art: In the audio content projection method, apparatus, device, and storage medium provided in this disclosure, by responding to user audio of a target type, the target projection content corresponding to the user audio and the target projection component in the vehicle are determined. The target type of user audio indicates that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle. The external projection position corresponding to the target projection content is determined, the target projection component is controlled to project the target projection content to the external projection position, and the user audio is played outside the vehicle. This allows for accurate determination of the corresponding target projection content, target projection component, and external projection component based on the user audio after receiving the user audio of a user inside the vehicle shouting to the outside of the vehicle. The projection position controls the target projection component to project the target content to the external projection position of the vehicle, and simultaneously plays the user's audio outside the vehicle. Therefore, when the external environment is noisy and people outside the vehicle cannot directly hear what the user inside the vehicle is saying, they can not only obtain information through the user's audio played outside the vehicle, but also intuitively and accurately obtain the information that the user inside the vehicle wants to convey by viewing the target projection content. This eliminates the need for the user inside the vehicle to repeatedly shout, lean out of the vehicle, or adjust the vehicle's position to communicate with people outside the vehicle, improving the efficiency and accuracy of information transmission, avoiding misunderstandings caused by poor information transmission, and reducing traffic hazard risks. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0022] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an audio content projection method provided in an embodiment of this disclosure; Figure 2 This is a flowchart of a method for determining user audio as a target type, provided in an embodiment of this disclosure; Figure 3 This is a flowchart of a method for determining the external projection position of a vehicle according to an embodiment of this disclosure; Figure 4 This is a flowchart illustrating a method for displaying target projection content according to an embodiment of this disclosure; Figure 5 This is a flowchart of a method for adjusting the position of an external projection provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of an audio content projection device provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0024] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0025] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] Figure 1 This is a flowchart of an audio content projection method provided in an embodiment of this disclosure. The method can be executed by an audio content projection device, which can be implemented in software and / or hardware, and can be configured in a vehicle. Figure 1 As shown, the audio content projection method provided in this embodiment includes the following steps: S101. In response to user audio of target type, determine the target projection content corresponding to the user audio and the target projection component in the vehicle. User audio of target type means that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle.

[0028] The user audio in this embodiment can be understood as the user audio collected by an audio acquisition device inside the vehicle, such as a microphone. For example, the user audio can be the audio generated by a user speaking inside the vehicle, or the audio played by electronic devices inside the vehicle, and is not limited here.

[0029] The target projection content in this embodiment can be understood as the specific information content corresponding to the user's audio that needs to be projected and displayed. For example, the target projection content may include text content, image content, video content, etc., and is not limited thereto.

[0030] The projection component in this disclosure can be understood as one or more components installed on a vehicle that have an external projection function. Different projection components can be installed in different locations on the vehicle, such as in front of and behind the vehicle. The target projection component can be understood as a component used to project target content onto the exterior of the vehicle, thereby displaying the target content to people outside the vehicle. For example, the target projection component can be a projector, a projection window with projection function, a projection headlight, etc. Among them, the projection headlight is an automotive lighting system that combines lighting function with projection display function. It emits light through a high-brightness light source (such as a light-emitting diode or laser), which is focused and reflected by an optical system (such as a lens or reflector) and illuminates an imaging element. The imaging element modulates the light according to the input image signal, and finally projects the modulated light onto the road surface or other target surface outside the vehicle through a projection lens to form clear text or images.

[0031] In this embodiment, the audio content projection device can determine the target projection content and the target projection component to be projected based on the collected user audio when it detects user audio indicating a target type of user speaking from inside the vehicle to outside. Specifically, for the target projection content, the audio content projection device can parse the user audio according to a pre-set type of target projection content and generate target projection content of the corresponding type. For the target projection component, if there is one projection component for projection outside the vehicle, the audio content projection device can determine that projection component as the target projection component. If there are multiple projection components for projection outside the vehicle, the device can determine one or more target projection components from the multiple projection components according to the user's settings for the projection function, or it can determine one or more target projection components from the multiple projection components according to scene information.

[0032] In one exemplary embodiment of this disclosure, when the type of the target projection content is text, the audio content projection device can perform audio-to-text processing on the user's audio to obtain the text corresponding to the user's audio, and determine the target projection content based on the text corresponding to the user's audio.

[0033] Optionally, the audio content projection device can convert user audio into corresponding text using a pre-trained text conversion model. The text conversion model first preprocesses the user audio to remove silence periods, then performs frame segmentation to extract sound features from the obtained audio, and then obtains the text corresponding to the audio through an acoustic model and a language model. For example, the language model can be based on a recurrent neural network (RNN), and the acoustic model can be based on deep neural networks (DNN) or long short-term memory networks (LSTM), without any limitation.

[0034] Optionally, the audio content projection device can convert the user's audio into corresponding text, correct the text based on a preset vocabulary, and determine the corrected text as the target projection content. The preset vocabulary can include everyday phrases and traffic terms, such as "be careful," "pull over," "attention," "turn left," and "go straight," as well as words that might be used when a user shouts from inside the vehicle, such as "parking space," "occupied," and "malfunction." Users can also customize the preset vocabulary by adding potentially used shouting words. When correcting the text, the audio content projection device can segment the text, remove interjections, and then match the segmentation results with words in the preset vocabulary. Specifically, it can replace dialect words, words with similar pronunciations, and common mistranslated words in the segmentation results with words from the preset vocabulary. For segmentation results that do not match words in the preset vocabulary, it can check and correct them based on contextual semantic logic and common collocations. The corrected segmentation results are then combined to obtain the target projection content. Alternatively, a pre-set vocabulary can be used to train the natural language model, and the trained natural language model can be used to correct the text and obtain the target projection content.

[0035] In another exemplary embodiment of the present disclosure, when the type of the target projection content is an image, the audio content projection device can search for a target image that matches the text corresponding to the user's audio in a preset projection image library and determine the target image as the target projection content. Alternatively, it can use a pre-trained image generation model to generate a target image based on the text corresponding to the user's audio and determine the target image as the target projection content.

[0036] In another exemplary embodiment of the present disclosure, when the type of the target projection content is video, the audio content projection device can search for a target video that matches the text corresponding to the user's audio in a preset projection video library and determine the target video as the target projection content. Alternatively, it can use a pre-trained video generation model to generate a target video based on the text corresponding to the user's audio and determine the target video as the target projection content.

[0037] In another exemplary embodiment of the present disclosure, when a vehicle is provided with a plurality of projection components for projecting to the outside of the vehicle, the audio content projection device can determine the projection component specified by the user for projecting target projection content corresponding to the target type of user audio as the target projection component.

[0038] In another exemplary embodiment of this disclosure, when multiple projection components for projecting onto the outside of the vehicle are installed on the vehicle, the audio content projection device can acquire scene information of the vehicle's location and determine the number and location of people outside the vehicle within a preset range based on the scene information. If the number of people outside the vehicle within the preset range is one, or if all people outside the vehicle within the preset range are located in the same direction as the vehicle, then a corresponding target projection component is selected from the multiple projection components based on the location of the people outside the vehicle. If the number of people outside the vehicle within the preset range is multiple, and the multiple people outside the vehicle are located in different directions as the vehicle, then a target projection component is determined from the multiple projection components based on user instructions. The user instructions can be voice instructions, gesture instructions, etc., and are not limited thereto.

[0039] In another exemplary embodiment of the present disclosure, when multiple projection components for projecting onto the outside of the vehicle are provided on the vehicle, the audio content projection device can extract semantic information of the target type of user audio, determine whether the semantic information contains information related to the projection component, projection direction or projection position, and if so, determine the corresponding projection component as the target projection component based on the semantic information.

[0040] S102. Determine the external projection position of the target projection content.

[0041] In this embodiment of the present disclosure, after determining the target projection content and the target projection component, the audio content projection device can further determine the external projection position for projecting the target projection content. Specifically, the audio content projection device can determine the projection range based on the target projection component and determine the external projection position within the projection range.

[0042] In one exemplary embodiment of this disclosure, the audio content projection device can determine the optimal external projection position within the projection range based on external environmental information such as light and surface flatness, thereby ensuring the projection effect of the target content.

[0043] S103, Control the target projection component to project the target projection content to the projection position outside the vehicle, and play the user's audio outside the vehicle.

[0044] In this embodiment, the audio content projection device, after determining the target projection content, the target projection component, and the external projection position, converts the target projection content into a projection signal format supported by the target projection component. Based on the external projection position, it determines the projection parameters of the target projection component, such as angle and focal length. Based on the format-converted target projection content and projection parameters, it generates a control command and sends the control command to the target projection component, causing the target projection component to execute upon receiving the control command and project the target projection content to the external projection position. Simultaneously, the audio content projection device can also play user audio outside the vehicle. Specifically, it can send the user audio to an external audio playback device, such as a speaker, and control the speaker to play the user audio outside the vehicle.

[0045] This embodiment of the disclosure determines the target projection content corresponding to the user audio and the target projection component in the vehicle in response to the user audio of the target type. The target type of user audio indicates that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle. It determines the external projection position corresponding to the target projection content, controls the target projection component to project the target projection content to the external projection position, and plays the user audio outside the vehicle. After receiving the user audio of a user inside the vehicle shouting to the outside of the vehicle, it can accurately determine the corresponding target projection content, target projection component, and external projection position based on the user audio. It controls the target projection component to project the target projection content to the external projection position and simultaneously plays the user audio outside the vehicle. Therefore, when the external environment is relatively noisy and people outside the vehicle cannot directly hear the user shouting from inside the vehicle, they can not only obtain information through the user audio played outside the vehicle, but also intuitively and accurately obtain the information that the user inside the vehicle wants to convey by viewing the target projection content. This eliminates the need for the user inside the vehicle to repeatedly shout, lean out of the vehicle, or adjust the vehicle position to communicate with people outside the vehicle, improving the efficiency and accuracy of information transmission, avoiding misunderstandings caused by poor information transmission, and reducing the risk of traffic hazards.

[0046] Figure 2 This is a flowchart of a method for determining user audio as a target type, provided in an embodiment of this disclosure. Figure 2 As shown, based on the above embodiments, the user's audio can be determined to be of the target type using the following method.

[0047] S201. Based on the activation status of the shout function, determine the matching priority corresponding to at least one intent type, wherein at least one intent type includes a target type, and when the shout function is activated, the matching priority corresponding to the target type is the highest priority.

[0048] In this embodiment of the disclosure, the shouting function can correspond to a physical button or a virtual control. Users can switch the on / off state of the shouting function by operating the physical button or the virtual control.

[0049] The intent type in this embodiment can be understood as different types after classifying and dividing the user's behavioral purpose and demand tendency carried by the user's audio. For example, the intent type may include not only the target type of the user in the car shouting to the outside of the car, but also the intent types of communication between people in the car, making and receiving phone calls in the car, and entertainment in the car, etc., without limitation.

[0050] In this embodiment of the disclosure, the on and off states of the announcement function correspond to different matching priority rules for each intent type. For example, when the announcement function is on, the target type corresponds to the first priority (i.e., the highest priority), the intent type of communication between people in the vehicle corresponds to the second priority, the intent type of making or receiving phone calls in the vehicle corresponds to the third priority, and the intent type of in-vehicle entertainment corresponds to the fourth priority. When the announcement function is off, the intent type of communication between people in the vehicle corresponds to the first priority, the intent type of making or receiving phone calls in the vehicle corresponds to the second priority, the intent type of in-vehicle entertainment corresponds to the third priority, and the target type corresponds to the fourth priority.

[0051] In this embodiment of the disclosure, the audio content projection device can obtain the activation status of the shouting function after detecting the user's audio, and determine the matching priority of at least one intent type corresponding to the current activation status.

[0052] In one exemplary embodiment of this disclosure, the audio content projection device can acquire the noise level outside the vehicle when it is determined that the current announcement function is off. Specifically, it can acquire the current noise level in decibels outside the vehicle and compare it with a preset threshold. If the noise level is higher than the preset threshold, the matching priority of the target type is increased based on the matching priority of the target type corresponding to the off announcement function. For example, if the announcement function is off, the intention type of communication between people inside the vehicle corresponds to the first priority, the intention type of making or receiving phone calls inside the vehicle corresponds to the second priority, the intention type of in-vehicle entertainment corresponds to the third priority, and the target type corresponds to the fourth priority. When the noise level is detected to be higher than the preset threshold, the priority of the target type can be increased to the first priority, and the matching priority of other intention types is adjusted according to the original priority order, that is, the intention type of communication between people inside the vehicle corresponds to the second priority, the intention type of making or receiving phone calls inside the vehicle corresponds to the third priority, and the intention type of in-vehicle entertainment corresponds to the fourth priority.

[0053] S202. Based on the matching priority corresponding to each intent type, match the user audio with each intent type to determine the intent type of the user audio as the target type.

[0054] In this embodiment of the disclosure, the audio content projection device, after determining the matching priority corresponding to each intent type, sequentially matches the user audio with each intent type in descending order of matching priority. If the user audio successfully matches the currently matched intent type, the intent type of the user audio is determined to be the currently matched intent type. If the user audio fails to match the currently matched intent type, the device continues to match the user audio with the intent type of the next matching priority until an intent type that successfully matches the target projection content is found. When the user audio successfully matches the target type, the intent type of the user audio is determined to be the target type.

[0055] In one exemplary embodiment of this disclosure, the audio content projection device can first convert user audio into text and then match the converted text with various intent types. Each intent type corresponds to a pre-trained matching model, which determines whether the intent of the text matches the corresponding intent type. When matching user audio with intent types, the audio content projection device can sequentially input the converted text into the matching model corresponding to the intent type with the current matching priority, according to the matching priority from high to low. Based on the output of the matching model, it determines whether the user audio intent type matches the current intent type. When the text converted from user audio is input into the matching model corresponding to the target type, and a successful match is obtained from the matching model's output, the user audio intent type is determined to be the target type.

[0056] This embodiment of the disclosure determines the matching priority of at least one intent type based on whether the shout function is enabled. The at least one intent type includes a target type. When the shout function is enabled, the matching priority of the target type is the highest priority. Based on the matching priority of each intent type, the user audio is matched with each intent type to determine the intent type of the user audio as the target type. Different matching priorities can be set for different intent types according to whether the shout function is enabled, thereby improving the accuracy of identifying the intent type of the user audio, reducing the probability of misprojection caused by recognition errors, and improving the user experience.

[0057] In some embodiments, when user audio of the target type is received, the target projection component may be in a working state. The audio content projection device can obtain the current projection content of the target projection component. If the current projection content of the target projection component includes a projected image, the device determines the external projection position corresponding to the target projection content based on the projected image and the target projection content. And / or, if the current projection content of the target projection component does not include a projected image, the device determines the external projection position corresponding to the target projection component based on the correspondence between the projection component and a preset projection position.

[0058] Specifically, when the audio content projection device determines that target projection content exists, it can acquire the current projection content of the target projection component. For example, the current projection content may include a projected image, specifically including driver assistance images and entertainment images. The driver assistance images may include lane boundary lines, turn guidance lines, distance warning lines, route guidance information, etc. When the current projection content of the target projection component includes a projected image, the audio content projection device can determine the external projection position for projecting the target projection content based on the projected image and the target projection content, thereby controlling the target projection component to project both the projected image and the target projection content simultaneously.

[0059] Optionally, the audio content projection device can determine the unoccupied area of ​​the projected image as the vehicle exterior projection position corresponding to the target projected content within the projection range of the target projection component, or it can determine a specific area in the projected image as the vehicle exterior projection position corresponding to the target projected content, so that the projected image and the target projected content are superimposed and displayed.

[0060] When the current projection content of the target projection component does not include a projected image, the audio content projection device can directly project the target projection content. Based on the pre-acquired correspondence between the projection component and the preset projection position, the target projection position corresponding to the target projection component is determined, and the target projection position is determined as the external projection position of the vehicle.

[0061] This disclosure, through its embodiments, determines the vehicle exterior projection position corresponding to the target projection content based on the projected image and the target projection content when the current projection content of the target projection component includes a projected image, and / or, when the current projection content of the target projection component does not include a projected image, determines the vehicle exterior projection position corresponding to the target projection component based on the correspondence between the projection component and a preset projection position. This allows for optimization of the overall display effect of the projected image and target projection content when both projected images and target projection content need to be displayed, by combining the projected image to determine the vehicle exterior projection position. Furthermore, when projected images are not required, the optimal vehicle exterior projection position corresponding to the target projection component is directly used to project the target projection content, shortening processing time and further improving the user experience.

[0062] Figure 3 This is a flowchart of a method for determining the external projection position of a vehicle according to an embodiment of this disclosure, such as... Figure 3 As shown, based on the above embodiments, the external projection position can be determined by the following method.

[0063] S301. Based on the projected image and the target projected content, determine the embedding position of the target projected content in the projected image.

[0064] The embedding position in the embodiments of this disclosure can be understood as the position of the target projection content in the image coordinate system corresponding to the projected image.

[0065] In this embodiment of the disclosure, the audio content projection device can first determine the coordinates of the target projection content in the image coordinate system corresponding to the projection image when the current projection content of the target projection component includes the projection image.

[0066] In one exemplary embodiment of the present disclosure, the audio content projection device can input the projected image and the target projected content into a pre-trained embedding model to obtain the embedding position output by the embedding model. The embedding model can be trained based on pre-collected model training data, which may include images, embedded content, and annotation results used to characterize the optimal embedding position.

[0067] In another exemplary embodiment of the present disclosure, when the current projection content includes assisted driving images, the embedding position corresponding to the target projection content can be determined according to the preset correspondence between the type of assisted driving images and the embedding position; when the current projection content includes entertainment images, the entertainment images can be divided into regions, and the target projection content can be matched with each image region, and the location of the successfully matched image region can be determined as the embedding position.

[0068] S302. Based on the first transformation relationship between the image coordinate system and the first coordinate system, the embedded position is transformed to obtain the external projection position of the vehicle. The first coordinate system is the coordinate system corresponding to the target projection component.

[0069] The transformation relationship in the embodiments of this disclosure may include transformation parameters such as translation, rotation, and scaling. Optionally, the first transformation relationship may be obtained in advance or calibrated and corrected in real time, which is not limited here.

[0070] In this embodiment of the present disclosure, after obtaining the embedding position of the target projection content in the projection image, the audio content projection device can transform the embedding position using a first transformation relationship between the image coordinate system and the first coordinate system, and determine the transformed result as the projection position outside the vehicle.

[0071] This embodiment of the disclosure determines the embedding position of the target projection content in the projected image based on the projected image and the target projection content. The embedding position is transformed based on a first transformation relationship between the image coordinate system and the first coordinate system to obtain the external projection position of the vehicle. The first coordinate system is the coordinate system corresponding to the target projection component. It can comprehensively determine the optimal embedding position in the image coordinate system by combining the projected image and the target projection content. Then, the embedding position is mapped to the first coordinate system where the target projection component is located, which improves the accuracy of the external projection position of the target projection content and ensures that the final projection effect is consistent with the expected effect corresponding to the embedding position, thereby further improving the user experience.

[0072] Figure 4 This is a flowchart illustrating a method for displaying target projection content according to an embodiment of this disclosure. For example... Figure 4 As shown, based on the above embodiments, the target projection content can be displayed using the following method.

[0073] S401. Based on the second transformation relationship between the first coordinate system and the second coordinate system, the external projection position is transformed to obtain the interface display position of the target projection content in the car model display interface. The first coordinate system is the coordinate system corresponding to the target projection component, and the second coordinate system is the coordinate system corresponding to the vehicle 3D model in the car model display interface.

[0074] The vehicle model display interface in this embodiment can be understood as an interface used to display the three-dimensional appearance model of the vehicle.

[0075] In this embodiment of the disclosure, the audio content projection device can determine the interface display position of the target projection content in the vehicle model display interface after determining the vehicle exterior projection position corresponding to the target projection content. Specifically, the vehicle exterior projection position can be transformed by the second transformation relationship between the first coordinate system corresponding to the target projection component and the second coordinate system corresponding to the vehicle three-dimensional model, and the transformed result can be determined as the interface display position.

[0076] S402. Display the target projection content in the car model display interface displayed on the in-vehicle screen based on the interface display position.

[0077] The in-vehicle screen in this embodiment can be any screen in the vehicle, such as the instrument panel screen, the central control display screen, the passenger entertainment screen, the rear entertainment screen, the head-up display screen, etc., and is not limited thereto.

[0078] In this embodiment of the present disclosure, after determining the interface display position, the audio content projection device can add the target projection content to the second coordinate system corresponding to the vehicle 3D model based on the interface display position, so that the combined effect of the vehicle 3D model and the target projection content is consistent with the real projection effect, and send the vehicle 3D model with the added target projection content to the in-vehicle screen for display on the vehicle model display interface displayed on the in-vehicle screen.

[0079] In one exemplary embodiment of the present disclosure, the audio content projection device can display the target projection content using different display effects on the vehicle model display interface displayed on different in-vehicle screens. Specifically, it can acquire multi-angle images of the vehicle 3D model and the target projection content from multiple perspectives and send the multi-angle images to different in-vehicle screens for display.

[0080] This embodiment transforms the external projection position based on a second transformation relationship between a first coordinate system and a second coordinate system to obtain the interface display position of the target projection content in the vehicle model display interface. The first coordinate system is the coordinate system corresponding to the target projection component, and the second coordinate system is the coordinate system corresponding to the vehicle 3D model in the vehicle model display interface. Based on the interface display position, the target projection content is displayed in the vehicle model display interface on the in-vehicle screen, which can simulate the real projection effect of the target projection content outside the vehicle. This allows in-vehicle users to quickly understand the presentation of the external projection through the in-vehicle screen, further improving the user experience.

[0081] In some embodiments, after determining the external projection position of the target projection content, the audio content projection device can send the external projection position to the 3D engine in the vehicle system. The 3D engine then determines the corresponding interface display position of the external projection position in the vehicle model display interface and generates a corresponding display image to be displayed in the vehicle model display interface.

[0082] Figure 5 This is a flowchart of a method for adjusting the external projection position of a vehicle according to an embodiment of this disclosure, such as... Figure 5 As shown, based on the above embodiments, the position of the vehicle's external projection can be adjusted using the following method.

[0083] S501. In response to detecting a projection position adjustment operation in the car model display interface, the target projection content in the car model display interface is adjusted to the target display position for display. The target display position is the display position corresponding to the projection position adjustment operation.

[0084] The projection position adjustment operation in this embodiment can be understood as an operation to adjust the display position of the target projection content in the car model display interface. For example, the projection position adjustment operation can be implemented by voice commands, gesture commands, touch commands, etc., which are not limited here.

[0085] In this embodiment of the present disclosure, the audio content projection device can obtain the target display position corresponding to the projection position adjustment operation after detecting the user's projection position adjustment operation, and adjust the display position of the target projection content in the car model display interface from the original interface display position to the target display position for display.

[0086] In one exemplary embodiment of this disclosure, the projection position adjustment operation can be a sliding operation. After the audio content projection device detects the sliding operation in the car model display interface, it can obtain the sliding start point and sliding end point of the sliding operation, determine whether the sliding start point is located at the current interface display position of the target projection content, and if so, determine that the sliding operation is a projection position adjustment operation, and determine the position of the sliding end point as the target display position corresponding to the projection position adjustment operation, and then adjust the target projection content to the target display position for display.

[0087] In another exemplary embodiment of the present disclosure, the projection position adjustment operation can be an operation on the position switching control in the car model display interface. After the audio content projection device detects the sliding operation in the car model display interface, it determines the next preset display position as the target display position according to the switching order of multiple preset display positions obtained in advance, and then adjusts the target projection content to the target display position for display.

[0088] S502. Based on the second transformation relationship, the target display position is transformed to obtain the target vehicle external projection position.

[0089] In this embodiment of the present disclosure, the audio content projection device can determine the target vehicle exterior projection position of the target projection content when it is actually projected based on the target display position after determining the adjusted target display position. Specifically, it can use the second transformation relationship between the first coordinate system corresponding to the target projection component and the second coordinate system corresponding to the vehicle three-dimensional model to transform the target display position and determine the transformed result as the target vehicle exterior projection position.

[0090] S503, Control the target projection component to project the target projection content to the target vehicle's external projection position.

[0091] In this embodiment of the present disclosure, the audio content projection device can determine the target projection parameters of the target projection component based on the target projection position after determining the target projection position outside the vehicle, generate a control command based on the target projection parameters, and send the control command to the target projection component so that the target projection component can execute the control command after receiving it and project the target projection content to the target projection position outside the vehicle.

[0092] This embodiment of the disclosure, in response to a projection position adjustment operation detected in the vehicle model display interface, adjusts the target projection content in the vehicle model display interface to a target display position for display. The target display position is the display position corresponding to the projection position adjustment operation. Based on a second conversion relationship, the target display position is converted to obtain the target vehicle exterior projection position. The target projection component is controlled to project the target projection content to the target vehicle exterior projection position. This allows for quick adjustment through the vehicle model display interface when the user wants to adjust the projection position of the target projection content, improving the smoothness and convenience of user interaction. It also enhances the intelligence level and practicality of the vehicle projection component, further improving the user experience.

[0093] In some embodiments, the target projection content includes text content. Before controlling the target projection component to project the target projection content to the projection position outside the vehicle, the audio content projection device can determine the projection style information of the text content based on the semantic information of the text content, and when controlling the target projection component to project the target projection content to the projection position outside the vehicle, control the target projection component to project the target projection content to the projection position outside the vehicle according to the projection style information.

[0094] The projection style information can be understood as visual parameters related to the text presentation effect. For example, the projection style information may include font, font size, text thickness, text color, alignment, static or dynamic display effect, etc., without limitation.

[0095] Specifically, when the target projection content includes text, the audio content projection device can convert the user's audio into text information, correct the text information to obtain the text to be projected, extract the semantic information of the text to be projected, determine the projection style information based on the semantic information, match the semantic information with preset semantic types, and determine the projection style information corresponding to the successfully matched semantic type as the projection style information of the text content. For example, when the successfully matched semantic type is a warning semantic type, the corresponding projection style information can be a bold red font (size 3) displayed statically. When the successfully matched semantic type is an emotional expression semantic type, the corresponding projection style information can be a green font (size 4) displayed dynamically, such as with flashing or fade-in effects. The correspondence between semantic types and projection style information can be customized by the user. After determining the projection style information, the text to be projected is rendered using this projection style information to generate the final text content to be projected. Then, when the target projection component projects the target content, it controls the target projection component to project the text content with the corresponding projection style to the projection position outside the vehicle.

[0096] This embodiment of the disclosure determines the projection style information of the text content based on the semantic information of the text content, and controls the target projection component to project the target projection content to the projection position outside the vehicle according to the projection style information. It can present the corresponding display effect by combining the semantics of the text content, enhance the fun of interaction between users inside the vehicle and people outside the vehicle, and further improve the user experience.

[0097] Figure 6 This is a schematic diagram of the structure of an audio content projection device provided in an embodiment of this disclosure. Figure 6 As shown, the audio content projection device 600 includes: a first determining module 610, a second determining module 620, and a first control module 630. The first determining module 610 is used to determine the target projection content corresponding to the user audio and the target projection component in the vehicle in response to the user audio of the target type. The target type of user audio indicates that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle. The second determining module 620 is used to determine the projection position outside the vehicle corresponding to the target projection content. The first control module 630 is used to control the target projection component to project the target projection content to the projection position outside the vehicle and play the user audio outside the vehicle.

[0098] This embodiment of the disclosure determines the target projection content corresponding to the user audio and the target projection component in the vehicle in response to the user audio of the target type. The target type of user audio indicates that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle. It determines the projection position outside the vehicle corresponding to the target projection content, controls the target projection component to project the target projection content to the projection position outside the vehicle, and plays the user audio outside the vehicle. After receiving the user audio of a user inside the vehicle shouting to the outside of the vehicle, it can accurately determine the corresponding target projection content, target projection component, and projection position outside the vehicle based on the user audio. It controls the target projection component to project the target projection content to the projection position outside the vehicle and simultaneously plays the user audio outside the vehicle. Therefore, when the outside environment is relatively noisy and people outside the vehicle cannot directly hear the user shouting inside the vehicle, they can not only obtain information through the user audio played outside the vehicle, but also intuitively and accurately obtain the information that the user inside the vehicle wants to convey by viewing the target projection content. This eliminates the need for the user inside the vehicle to repeatedly shout, lean out of the vehicle, or adjust the vehicle position to communicate with people outside the vehicle, improving the efficiency and accuracy of information transmission, avoiding misunderstandings caused by poor information transmission, and reducing the risk of traffic hazards.

[0099] Optionally, the audio content projection device 600 further includes: a third determining module, used to determine the matching priority corresponding to at least one intent type based on the activation status of the shouting function, wherein at least one intent type includes a target type, and when the shouting function is activated, the matching priority corresponding to the target type is the highest priority; and a matching module, used to match the user audio with each intent type based on the matching priority corresponding to each intent type, and determine that the intent type of the user audio is the target type.

[0100] Optionally, the second determining module 620 is specifically used to determine the vehicle exterior projection position corresponding to the target projection content based on the projected image and the target projection content when the current projection content of the target projection component includes a projected image; and to determine the vehicle exterior projection position corresponding to the target projection component based on the correspondence between the projection component and the preset projection position when the current projection content of the target projection component does not include a projected image.

[0101] Optionally, the second determining module 620 is specifically used to determine the embedding position of the target projection content in the projection image based on the projection image and the target projection content; and to transform the embedding position based on the first transformation relationship between the image coordinate system and the first coordinate system to obtain the external projection position, wherein the first coordinate system is the coordinate system corresponding to the target projection component.

[0102] Optionally, the audio content projection device 600 further includes: a first conversion module, used to convert the external projection position based on a second conversion relationship between a first coordinate system and a second coordinate system to obtain the interface display position of the target projection content in the vehicle model display interface, wherein the first coordinate system is the coordinate system corresponding to the target projection component and the second coordinate system is the coordinate system corresponding to the vehicle 3D model in the vehicle model display interface; and a display module, used to display the target projection content in the vehicle model display interface displayed on the in-vehicle screen based on the interface display position.

[0103] Optionally, the audio content projection device 600 further includes: an adjustment module, used to adjust the target projection content in the vehicle model display interface to a target display position for display in response to a projection position adjustment operation detected in the vehicle model display interface, wherein the target display position is the display position corresponding to the projection position adjustment operation; a second conversion module, used to convert the target display position based on a second conversion relationship to obtain a target vehicle exterior projection position; and a second control module, used to control the target projection component to project the target projection content to the target vehicle exterior projection position.

[0104] Optionally, the target projection content includes text content, and the audio content projection device 600 further includes: a fourth determining module, used to determine the projection style information of the text content based on the semantic information of the text content; and a third control module, used to control the target projection component to project the target projection content to the projection position outside the vehicle according to the projection style information.

[0105] The audio content projection device provided in this embodiment can perform the methods described in any of the above embodiments, and its execution method and beneficial effects are similar, so they will not be described again here.

[0106] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure.

[0107] like Figure 7 As shown, the computer device may include a processor 710 and a memory 720 storing computer program instructions.

[0108] Specifically, the processor 710 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this disclosure.

[0109] Memory 720 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 720 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 720 may include removable or non-removable (or fixed) media. Where appropriate, memory 720 may be internal or external to the integrated gateway device. In a particular embodiment, memory 720 is a non-volatile solid-state memory. In a particular embodiment, memory 720 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0110] The processor 710 reads and executes computer program instructions stored in the memory 720 to perform the steps of the audio content projection method provided in this embodiment of the disclosure.

[0111] In one example, the computer device may also include a transceiver 730 and a bus 740. Wherein, as... Figure 7 As shown, the processor 710, memory 720 and transceiver 730 are connected via bus 740 and communicate with each other.

[0112] Bus 740 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 740 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this disclosure, this disclosure contemplates any suitable bus or interconnect.

[0113] This disclosure also provides a computer-readable storage medium that can store a computer program that, when executed by a processor, enables the processor to implement the audio content projection method provided in this disclosure.

[0114] The aforementioned storage medium may, for example, include a memory 720 for computer program instructions, which can be executed by the processor 710 of the audio content projection device to complete the audio content projection method provided in this embodiment. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), compact disc-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device. The aforementioned computer program may be written in any combination of one or more programming languages ​​to perform the operations of this embodiment, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code may be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0115] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for projecting audio content, characterized in that, The method is applied to a vehicle, and the method includes: In response to user audio of a target type, determine the target projection content corresponding to the user audio and the target projection component in the vehicle, wherein the user audio of the target type indicates that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle; Determine the external projection position of the target projection content; The target projection component is controlled to project the target projection content to the projection position outside the vehicle, and the user audio is played outside the vehicle.

2. The method according to claim 1, characterized in that, Before determining the target projection content corresponding to the user audio and the target projection component in the vehicle in response to the user audio of the target type, the method includes: Based on whether the shout function is enabled, determine the matching priority corresponding to at least one intent type, wherein the at least one intent type includes the target type, and when the shout function is enabled, the matching priority corresponding to the target type is the highest priority; Based on the matching priority corresponding to each intent type, the user audio is matched with each intent type to determine the intent type of the user audio as the target type.

3. The method according to claim 1, characterized in that, Determining the external projection position corresponding to the target projection content includes: When the current projection content of the target projection component includes a projection image, the external projection position corresponding to the target projection content is determined based on the projection image and the target projection content. And / or, If the current projection content of the target projection component does not include a projected image, the external projection position corresponding to the target projection component is determined based on the correspondence between the projection component and the preset projection position.

4. The method according to claim 3, characterized in that, Determining the external projection position of the target projection content based on the projected image and the target projection content includes: Based on the projected image and the target projected content, determine the embedding position of the target projected content in the projected image; The embedded position is transformed based on the first transformation relationship between the image coordinate system and the first coordinate system to obtain the vehicle exterior projection position, where the first coordinate system is the coordinate system corresponding to the target projection component.

5. The method according to claim 1, characterized in that, After determining the external projection position corresponding to the target projection content, the method further includes: The external projection position is transformed based on the second transformation relationship between the first coordinate system and the second coordinate system to obtain the interface display position of the target projection content in the car model display interface. The first coordinate system is the coordinate system corresponding to the target projection component, and the second coordinate system is the coordinate system corresponding to the vehicle 3D model in the car model display interface. The target projection content is displayed on the car model display interface on the in-vehicle screen based on the interface display position.

6. The method according to claim 5, characterized in that, After displaying the target projection content on the car model display interface shown on the in-vehicle screen based on the interface display position, the method further includes: In response to detecting a projection position adjustment operation in the car model display interface, the target projection content in the car model display interface is adjusted to a target display position for display, wherein the target display position is the display position corresponding to the projection position adjustment operation; Based on the second conversion relationship, the target display position is converted to obtain the target vehicle exterior projection position; The target projection component is controlled to project the target projection content onto the target vehicle's external projection position.

7. The method according to claim 1, characterized in that, The target projection content includes text content. Before controlling the target projection component to project the target projection content to the external projection position, the method further includes: The projection style information of the text content is determined based on the semantic information of the text content; The control of the target projection component to project the target projection content to the external projection position includes: The target projection component is controlled to project the target projection content to the external projection position according to the projection style information.

8. An audio content projection device, characterized in that, The device is applied to a vehicle, and the device includes: The first determining module is used to determine the target projection content corresponding to the user audio and the target projection component in the vehicle in response to the user audio of the target type, wherein the user audio of the target type indicates that the user audio is the audio of a user inside the vehicle shouting to the outside of the vehicle; The second determining module is used to determine the vehicle exterior projection position corresponding to the target projection content; The first control module is used to control the target projection component to project the target projection content to the projection position outside the vehicle and to play the user audio outside the vehicle.

9. A computer device, characterized in that, include: Memory; processor; And a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the audio content projection method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the audio content projection method as described in any one of claims 1-7.

Images