Reminding method, system, and apparatus, vehicle, and related product

Abstract

A reminding method, system, and apparatus, a vehicle, a computer-readable storage medium, and a computer program product, which relate to the field of intelligent vehicles. According to the reminding method, a masking result of a sound masking system on a target sound at a sound masking location is determined, and the masking result is displayed to a user, so that user confidence in using the sound masking system can be improved. The reminding method further provides a plurality of modes for receiving user instructions and a plurality of modes for presenting reminder information to the user, so that various possible situations can be promptly and effectively fed back to the user, thereby improving user confidence in using the sound masking system, and further improving the usability of the sound masking system.

Description

A reminder method, system, device, vehicle, and related products Technical Field

[0001] This application relates to the field of acoustic technology, and more particularly to a reminder method, system, device, vehicle, and related products. Background Technology

[0002] Privacy Sound Shield technology is a technology that uses acoustic principles to protect sound privacy in a specific area. This technology effectively masks speech in the target area by playing noise at a sufficiently high volume, significantly reducing the intelligibility of the speech and thus achieving the purpose of confidentiality.

[0003] A system that uses privacy-masking technology to mask sound can be called a sound masking system. The level of confidentiality or speech intelligibility in a secure area is not stable in practice, as it is affected by various factors, such as the volume of the target sound or whether the masking speaker is blocked. This can lead to a lack of confidence among users when using the sound masking system, affecting its practicality. Summary of the Invention

[0004] This application provides a reminder method, system, device, vehicle, and related products, relating to the field of intelligent vehicles. The reminder method provided by this application can effectively detect the masking result of an in-cabin sound masking system on a target sound, and can display the masking result in a timely manner, thereby increasing user confidence in using the sound masking system and thus improving the practicality of the sound masking system.

[0005] Firstly, this application provides a warning method, including determining the sound source location, the sound masking location, and the sound observation signal within the cockpit; determining a masking result based on the sound source location, the sound masking location, and the sound observation signal; and determining a first warning message based on the masking result. The sound source location is the location from which the target sound is emitted. For example, the sound source location could be the location of the first user's mouth, and the sound emitted by the first user's mouth is called the target sound. Alternatively, the sound source location could be the location of a speaker, and the sound emitted by the speaker is called the target sound. The sound masking location is the location from which the target sound needs to be masked, for example, the sound masking location could be the location of the second user's head. The sound observation signal can be a signal obtained by a sound pickup device near (within a certain range) the sound masking location, or a signal obtained by a sound pickup device at the sound masking location. Here, "obtained" can refer to a signal directly acquired by the sound pickup device; if the sound pickup device has a certain preprocessing function, "obtained" can also refer to a signal obtained after certain processing by the sound pickup device. The “masking” described in this application means that a user at the sound masking location cannot understand or hear the sound from the sound source location.

[0006] In this application, the alerting device determines the masking result based on the sound source location, the sound masking location, and the sound observation signal. For example, when the sound observation signal is a signal obtained by a sound pickup device at the sound masking location, the alerting device can directly analyze the sound observation signal to determine the signal strength of the target sound at the sound masking location, thereby determining the masking result. For example, when the sound observation signal is a signal obtained by a sound pickup device near the sound masking location, the alerting device can determine the transfer function of the target sound and the transfer function of the masking sound, and, in conjunction with the sound observation signal, evaluate the signal strength of the target sound at the sound masking location, thereby determining the masking result. After determining the masking result, the alerting device determines a first alert message based on the masking result, which helps the user understand the masking effect of the sound masking system on the target sound at the sound masking location, thereby improving the user's understanding of the sound masking system and enhancing the practicality of the masking system.

[0007] Optionally, the transfer function of the target sound and / or the transfer function of the masking sound can be data predetermined and stored in the storage unit. Therefore, determining the transfer function of the target sound and / or the transfer function of the masking sound in this application can also be understood as acquiring the transfer function of the target sound and / or the transfer function of the masking sound. The transfer function of the target sound refers to the sound transfer function from the source location of the target sound to the sound masking location or the sound pickup device. The transfer function of the masking sound refers to the sound transfer function from the source location of the masking sound to the sound masking location or the sound pickup device. The alerting device can acquire the transfer function of the target sound and / or the transfer function of the masking sound by reading data from the storage unit. This application does not limit how the transfer function of the target sound and the transfer function of the masking sound are determined; for example, the transfer functions can be determined by theoretical modeling or experimental measurement. The storage unit storing the transfer functions can be the storage unit of the alerting device or an independent storage unit; this application does not limit this as well.

[0008] Optionally, the aforementioned first reminder information can be used to remind / display the masking result to the user. The masking result is an evaluation of the masking effect on the target sound at the masking location. For example, the masking result can be normal, warning, or ineffective. Furthermore, the masking result can also be classified in other ways. For example, displaying the masking result to the user when the masking result is in a normal state allows the user to speak normally and confidently. As another example, displaying the masking result to the user when the masking result is in a warning state allows the user to adjust their speaking volume in time, thereby preventing information leakage. As yet another example, displaying the masking result to the user when the masking result is in an ineffective state allows the user to adjust their speaking content in time, thereby preventing information leakage. In some possible implementations, an ineffective masking result may include the target sound volume exceeding a certain threshold, and / or the distance between the sound source location and the sound masking location being less than a certain threshold. In summary, determining appropriate alerts based on the masking results allows users to promptly understand the masking effect of the sound masking system on the target sound at the masking location, thereby increasing users' confidence in using the masking sound system and ultimately enhancing its practicality.

[0009] In one possible implementation, the above-mentioned reminder method further includes: determining a second reminder message when the volume of the target sound is greater than a first threshold, and / or when the distance between the sound source location and the sound masking location is less than a second threshold.

[0010] The volume of the target sound can refer to the volume of the target sound at any location within the cockpit. For example, the volume of the target sound can be the volume at the sound source location, or the volume of the target sound at the sound masking location. The first threshold can be preset. For example, the first threshold can be preset according to the performance of the sound shielding system, and the first threshold can be 60dB, 70dB, or 80dB, etc. This application does not limit this. Similarly, the second threshold can also be preset. For example, the second threshold is 30cm, 40cm, or 50cm, etc. This application does not limit this.

[0011] In the above embodiments, when the volume of the target sound exceeds the first threshold, the volume may approach or exceed the upper limit of the sound shielding system's shielding performance, rendering the sound shielding system ineffective in shielding the target sound. Similarly, when the distance between the sound source location and the target shielding location is less than the second threshold, on the one hand, the sound shielding system may not be able to effectively shield the target sound due to the close distance; on the other hand, the close distance may also cause the sound shielding system to interfere with the target sound, affecting normal communication between users outside the target shielding location. In short, if the volume of the target sound exceeds the first threshold, and / or the distance between the sound source location and the sound masking location is less than the second threshold, the sound shielding system may fail to shield the target sound, resulting in information leakage. In this case, determining the second reminder information allows the user to adjust the volume of the target sound in a timely manner, and / or adjust the distance between the sound source location and the sound masking location, ensuring that the sound shielding system can effectively shield the target sound.

[0012] For example, the scenario corresponding to "the volume of the target sound is greater than the first threshold" is: the user who makes the target sound is the first user, and the first user speaks too loudly due to emotions or personal characteristics during a telephone conversation with other passengers.

[0013] For example, the scenario corresponding to "the distance between the sound source location and the target shielding location is less than the second threshold" is as follows: the user who makes the target sound is the first user (located in the second row), the passenger in the target shielding location is the second user (located in the front row), the first user leans forward while speaking, or the second user leans back.

[0014] In another possible implementation, if the volume of the target sound exceeds a first threshold, the second reminder information is used to alert the user that the target sound volume is too high, and / or, the second reminder information is used to remind the user to lower the volume of the target sound. And / or, if the distance between the sound source location and the sound masking location is less than a second threshold, the second reminder information indicates that the distance between the sound source location and the sound masking location is too close, and / or, the second reminder information is used to remind the user to increase the distance between the sound source location and the sound masking location.

[0015] In the above implementation, when the volume of the target sound exceeds a first threshold, the second reminder information is used to alert the user that the target sound volume is too high, and / or to remind the user to lower the target sound volume, enabling the user to promptly detect that the target sound is too loud and thus reduce the volume in time, thereby preventing information leakage. When the distance between the sound source location and the sound masking location is less than a second threshold, the second reminder information indicates that the distance between the sound source location and the sound masking location is too close, and / or to remind the user to increase the distance between the sound source location and the sound masking location, enabling the user to promptly detect that the distance between the sound source location and the sound masking location is too close and thus increase the distance in time, thereby preventing information leakage. In summary, the above implementation allows users to promptly detect the possible causes of masking failure and prompts them to correctly and effectively improve related problems, thereby ensuring that the sound masking system can effectively mask the target sound and thus prevent information leakage. On the other hand, by providing timely feedback to users on potential problems, the above-described implementation can also enhance users' confidence in using the sound masking system, thereby improving the practicality of the sound masking system.

[0016] In another possible implementation, after determining the second reminder information to the user, the method further includes: determining the third reminder information when the volume of the target sound is less than or equal to a first threshold, and / or when the distance between the sound source location and the sound masking location is greater than or equal to a second threshold.

[0017] In the above embodiments, the volume of the target sound being less than or equal to a first threshold indicates that the volume of the target sound is within a suitable range. Similarly, the distance between the sound source location and the sound masking location being greater than or equal to a second threshold indicates that the distance between the sound source location and the sound masking location is within a suitable range. Therefore, the above embodiments can, when a user finds that the volume of the target sound is too loud, and / or that the distance between the sound source location and the sound masking location is too close, clearly indicate to the user whether the adjusted volume of the target sound is within a suitable range, and / or whether the adjusted distance between the sound source location and the sound masking location is within a suitable range. Accordingly, based on the third reminder information determined by the reminder device, the user can further improve the practicality of using the sound masking system, thereby enhancing the usability of the sound masking system.

[0018] Optionally, if the volume is less than or equal to a first threshold, the third alert information is used to indicate that the volume of the target sound is within an appropriate range. And / or, if the distance between the sound source location and the sound masking location is greater than or equal to a second threshold, the distance between the sound source location and the sound masking location is within an appropriate range.

[0019] Optionally, when the volume of the target sound is less than or equal to a first threshold, and / or when the distance between the sound source location and the sound masking location is greater than or equal to a second threshold, the alerting device determines a third alert message according to a first rule. The first rule can be preset or custom-defined; for example, it may periodically determine the third alert message. In the above embodiment, determining the third message according to the first rule when the volume of the target sound is within a suitable range, and / or the distance between the sound source location and the sound masking location is within a suitable range, allows the user to periodically determine whether the volume of the target sound is within a suitable range, and / or whether the distance between the sound source location and the sound masking location is within a suitable range, thereby increasing the user's confidence in using the sound masking device and thus improving the practicality of the sound masking device.

[0020] Optionally, the first, second, and third reminder messages can be sent to the user via video, images, sound, light, or haptic feedback. Furthermore, all reminder messages in this application can be sent to the user via video, images, sound, light, or haptic feedback. For example, reminder messages can be sent to the user through the vehicle's infotainment screen, HUD, ambient lighting, seat vibration, voice announcements, or through the vehicle's associated devices.

[0021] In another possible implementation, the method further includes: receiving a first instruction from a user. Based on the first instruction, identifying at least one target object within the cockpit, wherein the sound emitted by the target object is a target sound. And / or, based on the first instruction, identifying at least one masking object within the cockpit, wherein the masking object is the object whose target sound is to be masked. Both the target object and the masking object include the user within the cockpit, and / or, a spatial area within the cockpit. The first instruction is used to indicate at least one target object within the cockpit, and / or at least one masking object within the cockpit.

[0022] In the above embodiments, the alerting device receives a first instruction from the user and can determine at least one target object and / or at least one masking object within the cabin based on the first instruction. Both the target object and the masking object include users within the cabin and / or spatial areas within the cabin. Users within the cabin may be, for example, the driver, front passenger, or second-row passengers. Spatial areas within the cabin may be, for example, the space corresponding to the driver's seat, the front passenger seat, the left rear seat, or the right rear seat. Therefore, the above embodiments can determine the object emitting the target sound and / or the object whose target sound needs to be masked based on the user's first instruction, allowing the user to flexibly select the sound to be masked and / or the object to be masked.

[0023] Optionally, any target object can be different from any masking object.

[0024] Optionally, the target object can also be the user's mouth, and the object to be covered can also be the user's head or the user's ears.

[0025] In another possible implementation, the method further includes: displaying at least one target object determined based on the first instruction, and / or at least one masking object.

[0026] In the above embodiments, the reminder device can enable the user to determine whether the selection has been completed or whether the selection was correct by displaying at least one target object and / or at least one masking object determined based on the first instruction. This increases the user's confidence in using the sound masking system and thus improves the practicality of the sound masking system.

[0027] Optionally, assuming that the at least one target object and / or at least one masking object displayed to the user by the alerting device does not match the user's intentions, the user can also issue a new instruction to the alerting device to modify the at least one target object and / or at least one masking object, so that the modified at least one target object and / or at least one masking object is consistent with the user's intentions. Clearly, this method of providing timely feedback to the user significantly increases the user's confidence in using the sound masking system, thereby enhancing the practicality of the sound masking system.

[0028] In another possible implementation, the method further includes: receiving a second instruction from a user; adjusting at least one target object based on the second instruction, wherein the at least one target object before adjustment is different from the at least one target object after adjustment; and / or adjusting at least one masking object based on the second instruction, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment.

[0029] In the above embodiments, the reminder device adjusts at least one target object and / or at least one masking object based on a second instruction from the user. This allows the user to make timely adjustments if an error occurs in selecting the target object and / or the masking object. Furthermore, it allows the user to more flexibly select the target sound to be masked and / or the masking object to be masked, thereby improving the practicality of the sound masking system.

[0030] In another possible implementation, both the first and second instructions described above are user instructions, which include one or more of touch instructions, voice instructions, or gesture instructions. Touch instructions are, for example, instructions generated by a user through touching a human-computer interaction device; voice instructions are, for example, instructions generated by a user through voice control; and gesture instructions are, for example, instructions generated by a user through gesture control.

[0031] Optionally, the voice commands may also include semantic commands, such as commands obtained by analyzing the user's voice. For example, if user 1 says, "I'm going to make a phone call," the corresponding semantic command could be, "Identify at least one target object as user 1 and identify at least one cover object as other personnel in the cockpit."

[0032] In another possible implementation, determining the sound source location and the sound masking location includes: if the target is a user inside the cabin, determining the spatial area where the target's mouth is located as the sound source location; or, if the target is a spatial area inside the cabin, determining a portion or all of the target's spatial area as the sound source location. And / or, if the masking target is a user inside the cabin, determining the spatial area where the masking target's head is located as the sound masking location; or, if the masking target is a spatial area inside the cabin, determining a portion or all of the masking target's spatial area as the sound masking location.

[0033] In the above embodiments, when the target is a user inside the cabin, the alerting device determines the spatial area where the target's mouth is located as the sound source location. Alternatively, when the target is a spatial area inside the cabin, the alerting device determines part or all of the spatial area of ​​the target as the sound source location, making the sound source location more precise and improving the accuracy of determining the masking effect. Similarly, when the masked target is a user inside the cabin, the alerting device determines the spatial area where the masked target's head is located as the sound masking location. Alternatively, when the masked target is a spatial area inside the cabin, the alerting device determines part or all of the spatial area of ​​the masked target as the sound masking location, also making the sound masking location more precise and similarly improving the accuracy of determining the masking effect.

[0034] For example, when the target object is the driver's seat area, the reminder device can determine the sound source location as the area in the driver's seat area where the user's mouth frequently moves.

[0035] For example, when the target of the sound masking is the driver's seat area, the alerting device can determine the area in the driver's seat area where the user's head frequently moves as the sound masking location.

[0036] In another possible implementation, determining the spatial region where the head of the object being masked is located as the sound masking location includes: determining a first spatial region where the head of the object being masked is located based on perception technology, and determining the first spatial region as the sound masking location. The perception technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0037] In the above embodiments, the reminder device uses sensing technology to determine the location of the head of the object to be masked in the first spatial region and determines the first spatial region as the sound masking location. This allows the sound masking system to mask the target sound of the object by generating a masking region on the head of the object, thereby achieving a better masking effect.

[0038] In another possible implementation, after determining the first spatial region as the sound masking location, the method further includes: if the head of the masked object moves to the second spatial region, the second spatial region is determined as the sound masking location, and the first spatial region is different from the second spatial region.

[0039] In the above embodiments, when the head of the object being masked moves, by determining the second spatial region to which the head moves as the sound masking position, the sound masking system can mask the target sound in the second spatial region. This allows the system to mask the target sound of the object in real time when the head of the object being masked moves, thereby achieving a better masking effect.

[0040] In another possible implementation, determining the spatial region where the mouth of the target object is located as the sound source location includes: determining a third spatial region where the mouth of the target object is located based on perception technology, and determining the third spatial region as the sound source location. The perception technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0041] In the above embodiments, the reminder device uses sensing technology to determine the third spatial region where the mouth of the target object is located, and determines the third spatial region as the sound source location, so that the sound masking system can determine the sound source location and the target sound in a timely manner, and the calculated masking effect has high accuracy.

[0042] In another possible implementation, after determining the third spatial region as the sound source location, the method further includes: if the mouth of the target object moves to the fourth spatial region, the fourth spatial region is determined as the sound source location, and the third spatial region is different from the fourth spatial region.

[0043] In the above embodiments, when the mouth of the target object moves, the alerting device determines the fourth spatial region to which the mouth has moved as the sound source location, enabling the sound masking system to correctly identify the target sound and thus mask it. The above embodiments also prevent the sound masking system from identifying irrelevant sounds as the target sound, thereby masking incorrect sounds. Furthermore, by determining the fourth spatial region as the sound source location and assisting the sound masking system in correctly identifying the target sound, the alerting device achieves high accuracy in calculating the masking effect (since the masking effect is determined based on the sound source location), thereby improving the practicality of the sound masking system.

[0044] In another possible implementation, the method further includes: determining the sound masking location, and / or determining the sound source location.

[0045] In the above embodiments, the alerting device, by determining the sound masking location and / or the sound source location, enables users to promptly grasp the sound masking system's identification of the target masking location and / or the sound source location. This helps users judge whether the sound masking system can effectively mask the target sound, thereby increasing user confidence in using the sound masking system and enhancing its practicality. It should be noted that the alerting device is used to determine the real-time location information of the sound masking location and / or the sound source location. For example, if the head of the masked object moves from a first spatial region to a second spatial region, the alerting device will determine the first and second spatial regions sequentially.

[0046] In another possible implementation, the method further includes: sending a masking sound based on the target sound, the masking sound being used to mask the target sound at a sound masking location.

[0047] In the above embodiments, the reminder device sends a masking sound based on the target sound. The masking sound is used to mask the target sound at the sound masking position, so that the head movement of the masked object and / or the mouth movement of the target object can generate a suitable masking sound based on the target sound, thereby masking the target sound for the masked object and having a good masking effect.

[0048] In another possible implementation, the cockpit includes N sound pickup devices, where N is a positive integer. Determining the sound observation signal within the cockpit includes: picking up the sound observation signal through the N sound pickup devices, where the sound observation signal includes M sub-sound observation signals picked up by M of the N sound pickup devices, where M is a positive integer less than or equal to N. The sound pickup device is a device that has the function of converting sound (sound waves) into electrical signals; for example, the sound pickup device is a microphone or a pickup unit.

[0049] In the above embodiments, the sound observation signals include M sub-sound observation signals picked up by M sound pickup devices. It should be noted that the M sound pickup devices are M different sound pickup devices in the cockpit, and these M sound pickup devices can be set in different positions in the cockpit. Correspondingly, the M sub-sound observation signals can be signals observed at M different positions in the cockpit. Therefore, the M sub-sound observation signals can more accurately and comprehensively reflect the sound status in the cockpit, enabling the alerting device to determine the accurate masking effect based on the M sub-sound observation signals and send the masking effect to the user.

[0050] In another possible implementation, one or more of the M sound pickup devices are located within a first range of the sound masking location.

[0051] Optionally, when the sound masking location is a spatial point, the first range can be a spatial range centered on the sound masking location with a radius of a first length, wherein the first length is, for example, 10cm, 20cm, or 30cm. When the sound masking location is a spatial region, the spatial region represented by the first range includes the spatial region represented by the sound masking location.

[0052] In the above embodiments, one or more of the M sound pickup devices are located within a first range of the sound masking location, such that the observation sub-signals picked up by one or more of the M sound pickup devices have a high degree of similarity with the sound observation signals at the sound masking location. This enables the alerting device to determine an accurate masking result based on the observation sub-signals and send the masking result to the user, thereby increasing the user's confidence in using the sound masking system.

[0053] In another possible implementation, M=1, and the M sound pickup devices are the first sound pickup device, which is the sound pickup device closest to the sound masking position among the N sound pickup devices.

[0054] In the above embodiment, the first sound pickup device is the sound pickup device closest to the sound masking position among the N sound pickup devices, indicating that the sound observation signal picked up by the first sound pickup device has the highest similarity with the sound observation signal at the sound masking position. This enables the reminder device to determine the accurate masking result based on the observation sub-signal and send the masking result to the user, thereby increasing the user's confidence in using the sound masking system.

[0055] In another possible implementation, determining the masking result based on the sound source location, the sound masking location, and the sound observation signals includes: determining a first signal based on the sound source location, the sound masking location, and M sub-sound observation signals. This first signal describes the signal related to the target sound at the sound masking location. The masking result is then determined based on the first signal. The first signal describes the signal related to the target sound at the sound masking location; for example, it describes the signal strength of the target sound at the sound masking location. It is understood that the first signal is calculated, not actually measured, and therefore there may be errors between the first signal and the actually measured signal strength of the target sound.

[0056] In the above embodiment, the alerting device determines a first signal based on the sound source location, the sound masking location, and M sub-sound observation signals. Since the M sub-sound observation signals are sound observation signals acquired by M sound pickup devices in the cockpit, the alerting device, based on the sound source location, the sound masking location, and the M sub-sound observation signals, and in conjunction with acoustic principles, can evaluate the signal (i.e., the first signal) related to the target sound at the sound masking location. After the alerting device evaluates the first signal, it can analyze the first signal to determine whether the masking sound has effectively masked the sound signal at the sound masking location, thereby determining the masking effect.

[0057] Optionally, when M=1, the alerting device can directly determine the first signal based on the sound source location, the sound masking location, and M sub-sound observation signals. When M>1, the alerting device can determine M sub-signals based on the sound source location, the sound masking location, and the M sub-sound observation signals, wherein the first sub-sound observation signal among the M sub-sound observation signals is used to determine the first sub-signal among the M sub-signals, and there is a one-to-one correspondence between the M sub-sound observation signals and the M sub-signals. Further, the alerting device determines the first signal based on the M sub-signals, for example, by weighting the M sub-signals.

[0058] In another possible implementation, the cockpit includes a masking sound generating device for playing a masking sound to mask target sound at a sound masking location. Determining a masking result based on a first signal includes determining a second signal describing a signal related to the masking sound at the sound masking location. The masking result is then determined based on the first and second signals.

[0059] In the above embodiments, the alerting device can determine the second signal based on the masking sound, the location of the masking sound generating device, and the sound masking location. For example, the alerting device can determine / obtain the transfer function of the masking sound from the masking sound generating device to the sound masking location, thereby calculating the relevant signal of the masking sound propagating to the sound masking location. It is understood that when the second signal is calculated using a correlation algorithm, there may be a deviation between the masking sound-related signal described by the second signal and the actual signal of the masking sound at the sound masking location. After calculating the second signal, the masking result can be determined based on the first signal (the relevant signal used to describe the target sound at the sound masking location) and the second signal. For example, the masking result can be determined by calculating the ratio between the signal strength of the first signal and the signal strength of the second signal.

[0060] In another possible implementation, the above-mentioned determination of the masking result based on the sound source location, the sound masking location, and the sound observation signal includes: based on the sound observation signal S o Masking acoustic signal S m First transfer function H mo The second transfer function H sd and the third transfer function H so Determine the first signal S sd Sound observation signal S o Masking acoustic signal S m First transfer function H mo The second transfer function H sd The third transfer function H so and the first signal S sd S satisfies the following relationship: sd =(S o -S m *H mo )H sd / H so Among them, the masking acoustic signal S m Used to generate masking sound, the first transfer function H mo The second transfer function H is used to represent the transfer function between the masking sound generation device and the sound pickup device. sd The third transfer function H is used to represent the transfer function between the sound source location and the sound masking location. so This is used to represent the transfer function between the location of the sound source and the sound pickup device.

[0061] In the above embodiments, the masking acoustic signal S m Used to generate masking sound, for example, a loudspeaker based on the masking sound signal S m Capable of generating masking sound. First transfer function H mo The second transfer function H sdand the third transfer function H so The transfer function can be predetermined; for example, it can be determined based on theoretical modeling or experimental measurement methods. This application does not limit this. The above S... m *H mo Used to represent the sound observation signal S o The signal related to the masking sound, (S) o -S m *H mo ) is used to represent the sound observation signal S o Related signals other than masking sound, or, (S o -S m *H mo ) is used to represent the sound observation signal S o The signal related to the target sound. Combining the above, the second transfer function H... sd and the third transfer function H so From the description, we can know that (S) o -S m *H mo )H sd / H so This can be used to describe the signal related to the target sound at the sound masking location, i.e., the first signal. It is understood that the calculation accuracy of the first signal obtained above is related to the first transfer function H. mo The second transfer function H sd and the third transfer function H so The accuracy is related to that.

[0062] Optionally, the alerting device can alert the user via real-time masking sound signal S. m and sound observation signal S o Adaptive filtering calculations are performed to obtain the first transfer function H. mo Updated value H mo Furthermore, the reminder device can be based on the fourth transfer function H. md and the first transfer function H mo Calculate the fourth transfer function H md The updated value, for example, H md '=H md *H mo ' / H mo Among them, the fourth transfer function H md Used to represent the transfer function between the masking sound generation device and the sound masking location.

[0063] In another possible implementation, determining the masking result based on the first and second signals includes: the ratio R of the signal strength of the second signal to the signal strength of the first signal is less than R0. LIn the case of R, the masking result is determined to be in a failed state. Alternatively, if the ratio R is greater than R... H In the case where the ratio R is greater than or equal to R, the masking result is determined to be valid. L And the ratio R is less than or equal to R H In this case, the masking result is determined to be a warning state. Among them, R L ≤R H .

[0064] In the above embodiments, the ratio R is the ratio between the signal strength of the second signal and the signal strength of the first signal. A larger ratio R indicates a greater ratio between the signal strength of the second signal and the first signal, meaning the masking sound's signal strength is greater than the target sound's signal strength, resulting in a better masking effect. Conversely, a smaller ratio R indicates a smaller ratio between the signal strength of the second signal and the first signal, meaning the masking sound's signal strength is smaller than the target sound's signal strength, resulting in a worse masking effect. Therefore, the alerting device compares R and R... L and R H The size relationship between them can accurately determine the effect of masking the target sound at the target location.

[0065] Optionally, the above R L and R H It can be deterministic, regarding how to determine R. L and R H This application does not impose any restrictions.

[0066] In another possible implementation, the aforementioned ratio R is, for example, the first signal S. sd With the second signal S md The short-time energy ratio between them. For example, R, the first signal S sd With the second signal S md The following relationship exists between them:

[0067] Optionally, the first signal S sd With the second signal S md The short-time energy between them can be calculated by superimposing the entire frequency band. In this case, R and the first signal S sd With the second signal S md The following relationship exists between them:

[0068] Optionally, when the first signal S is... sd With the second signal S md When dividing the signal into multiple sub-bands, weighting coefficients can be set for each sub-band to highlight the importance of certain sub-bands. R, First signal S sd With the second signal Smd The following relationship can be satisfied between them:

[0069] Among them, G subband represents the weighting coefficient for each spectral energy.

[0070] Optionally, the division of the sub-bands is not limited in this application. For example, the sub-bands can be divided according to octaves or 1 / 2, 1 / 3 octaves, etc.

[0071] In another possible implementation, R L The value range of R includes [5dB, 18dB]. H The value range includes [10dB, 30dB].

[0072] In another possible implementation, the cockpit includes a sound masking system for masking target sound at a masking location. Determining the masking result based on a first signal and a second signal includes:

[0073] If the signal strength of the first signal is greater than or equal to the third threshold, the masking result is determined to be invalid. And / or,

[0074] If the signal strength of the first signal is less than or equal to the fourth threshold, the masking result is determined to be valid. And / or,

[0075] If the signal strength of the first signal is between the third and fourth thresholds, the masking result is determined to be a warning state. And / or,

[0076] If the signal strength of the second signal is less than or equal to the fifth threshold, the masking result is determined to be invalid. And / or,

[0077] If the signal strength of the second signal is greater than or equal to the sixth threshold, the masking result is determined to be valid. And / or,

[0078] If the signal strength of the second signal is between the fifth and sixth thresholds, the masking result is determined to be a warning state.

[0079] In the above embodiments, the alerting device can determine the masking result by comparing the signal strength of the first signal with the relationship between the third threshold and the fourth threshold. For example, if the signal strength of the first signal is greater than or equal to the third threshold, it indicates that the signal strength of the target sound at the masking location is too high, thus determining the masking result as invalid. As another example, if the signal strength of the first signal is less than or equal to the fourth threshold, it indicates that the signal strength of the target sound at the masking location is low, thus determining the masking result as valid. As yet another example, if the signal strength of the first signal is between the third and fourth thresholds, it indicates that the signal strength of the target sound at the masking location may cause information leakage, thus determining the masking result as a warning. The alerting device can also determine the masking result by comparing the signal strength of the second signal with the relationship between the fifth threshold and the sixth threshold. For example, if the signal strength of the second signal is less than or equal to the fifth threshold, it indicates that the signal strength of the masked sound at the masking location is too low, thus determining the masking result as invalid. As yet another example, if the signal strength of the second signal is greater than or equal to the sixth threshold, it indicates that the signal strength of the masked sound at the masking location is high, thus determining the masking result as valid. For example, if the signal strength of the second signal is between the fifth and sixth thresholds, it indicates that the signal strength of the masking sound at the sound masking location may not be sufficient to effectively mask the target sound, posing a risk of information leakage. Therefore, the masking result is determined to be a warning state.

[0080] Optionally, if the third threshold equals the fourth threshold, the masking result determined by the alerting device based on the third and fourth thresholds does not include a warning state. If the fifth threshold equals the sixth threshold, the masking result determined by the alerting device based on the fifth and sixth thresholds does not include a warning state.

[0081] In another possible implementation, the aforementioned first reminder information is used to remind the user of the masking result.

[0082] In the above embodiments, the first reminder information is used to remind the user of the masking result, so that the user knows the masking result of the target sound at the sound masking location, thereby increasing the user's confidence in using the sound masking system and improving the practicality of the sound masking system.

[0083] Secondly, this application provides a notification method, comprising: receiving a first instruction from a user, the first instruction being used to indicate at least one target object within the cockpit, the sound emitted by the target object being a target sound; and / or, the first instruction being used to indicate at least one masking object within the cockpit, the masking object being an object whose target sound is to be masked. Both the target object and the masking object include a user within the cockpit or a spatial area within the cockpit. In response to the first instruction, at least one target object is displayed, and / or at least one masking object is displayed.

[0084] In the above embodiments, the first instruction is used to indicate at least one target object and / or at least one masking object within the cockpit. For example, the first instruction is used to indicate that the space area in the right rear seat of the cockpit is at least one target object, and / or that the space area in the driver's seat of the cockpit is at least one target object. In response to the first instruction, the reminder device displays the at least one target object indicated by the first instruction, and / or displays at least one masking object. By displaying at least one target object and / or at least one masking object, the reminder device allows the user to clearly identify the currently selected at least one target object and / or at least one masking object, thereby increasing the user's confidence in using the sound masking system and thus improving the practicality of the sound masking system. Furthermore, the content of the reminder from the reminder device also allows the user to clearly understand whether the currently selected at least one target object and / or at least one masking object is correct, and to reselect if an incorrect selection is made, thereby further improving the practicality of the sound masking system.

[0085] In one possible implementation, the method further includes: receiving a second instruction from a user, the second instruction indicating adjustment of at least one target object, wherein the at least one target object before adjustment is different from the at least one target object after adjustment. And / or, the second instruction indicating adjustment of at least one masking object, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment. In response to the second instruction, displaying the at least one target object after adjustment, and / or displaying the at least one masking object after adjustment.

[0086] In the above embodiments, the second instruction is used to instruct the adjustment of at least one target object and / or at least one masking object. Optionally, the at least one target object before adjustment is different from the at least one target object after adjustment. For example, the at least one target object before adjustment includes objects 1 to 5, and the at least one target object after adjustment may include objects 1 to 6, objects 1 to 4, or objects 6 to 8, etc., ensuring that the at least one target object before and after adjustment is not completely identical. The at least one masking object before adjustment is also different from the at least one masking object after adjustment. For example, the at least one masking object before adjustment includes masking objects 1 to 5, and the at least one masking object after adjustment may include masking objects 1 to 6, masking objects 1 to 4, or masking objects 6 to 8, etc., ensuring that the at least one masking object before and after adjustment is not completely identical. The reminder device, in response to the second instruction, displays the at least one target object and / or at least one masking object after adjustment, enabling the user to clearly understand whether the adjustment is complete or whether the adjustment is correct. When a user is informed of at least one adjusted target object and / or at least one masked object through a notification device, it helps to increase the user's confidence in using the sound masking system, thereby improving the practicality of the sound masking system.

[0087] In another possible implementation, the first instruction is a user instruction, which includes one or more of touch instructions, voice instructions, or gesture instructions.

[0088] In the above embodiments, the first instruction is a user instruction, which includes one or more of touch instructions, voice instructions, or gesture instructions. In other words, the user can send the first instruction through one or more of touch, voice, or gesture. Similarly, the second instruction is also a user instruction, and the user can also send the second instruction through one or more of touch, voice, or gesture. Sending instructions through one or more of touch, voice, or gesture offers high practicality and convenience.

[0089] In another possible implementation, the method further includes displaying the volume information of the target sound. Alternatively, if the volume of the target sound is less than or equal to a first threshold, information indicating that the volume of the target sound is within a suitable range is displayed; or if the volume of the target sound is greater than the first threshold, information indicating that the volume of the target sound is too high is displayed. The information indicating that the volume of the target sound is too high includes information reminding the user to lower the volume of the target sound.

[0090] In the above embodiments, the reminder device displays the volume of the target sound, which helps the user determine whether the target sound volume is too loud, thus allowing the user to consciously control the target sound within an appropriate range. This also increases the user's confidence in using the target masking system. The reminder device can display the target sound volume in real time or according to certain rules. For example, the reminder device can periodically display the target sound volume. Furthermore, the reminder device can also display information indicating that the target sound volume is within an appropriate range when the target sound volume is less than or equal to a first threshold, allowing the user to directly determine whether the target sound volume is within an appropriate range based on the displayed information, thereby increasing the user's confidence in using the sound masking system. Correspondingly, the reminder device can also display information indicating that the target sound volume is too loud when the target sound volume exceeds the first threshold, allowing the user to adjust the target sound volume in a timely manner to avoid information leakage, which also helps to increase the user's confidence in using the sound masking system.

[0091] Optionally, the alert device may also display information such as the sound shielding being too weak, or that the sound shielding system is malfunctioning.

[0092] In another possible implementation, the at least one target object includes a first object, the location where the first object emits sound is the sound source location, and the at least one masking object includes a second object, the location where the second object receives sound is the sound masking location. The method further includes: displaying information about the distance between the sound source location and the sound masking location. And / or, if the distance between the sound source location and the sound masking location is greater than or equal to a second threshold, displaying information that the distance between the sound source location and the sound masking location is within a suitable range; or, if the distance between the sound source location and the sound masking location is less than the second threshold, displaying information that the distance between the sound source location and the sound masking location is too small. The information about the distance between the sound source location and the sound masking location being too small includes information reminding the user to increase the distance between the sound source location and the sound masking location.

[0093] It is understandable that if the distance between the sound source location and the target masking location is too close, the sound masking system will be unable to effectively mask the target sound, or the sound masking system will cause unnecessary interference to the target sound. When the distance between the sound source location and the target masking location is within a suitable range, it helps the sound masking system to better mask the target sound. In the above embodiment, the reminder device displays the distance information between the sound source location and the target masking location, which helps the user to know the distance information between the sound source location and the target masking location in a timely manner, thereby increasing the user's confidence in using the sound masking system. In addition, when the distance between the sound source location and the sound masking location is greater than or equal to the second threshold, displaying information that the distance between the sound source location and the sound masking location is within a suitable range allows the user to know whether the current distance between the sound source location and the sound masking location is within a suitable range, thereby increasing the user's confidence in using the sound masking system. When the distance between the sound source location and the sound masking location is less than a second threshold, displaying information indicating that the distance is too small alerts the user. This allows the user to adjust the distance promptly, ensuring the sound masking system effectively masks the target sound. In summary, by displaying the distance between the sound source location and the sound masking location, along with related information, the alert device helps users understand the operational status of the sound masking system, thereby improving user experience and ultimately enhancing its practicality.

[0094] In another possible implementation, when the second object is a user inside the cabin, the method further includes: displaying the spatial area where the head is located after the head movement, in the event that the second object's head moves. Here, "movement of the second object's head" includes the head moving a range greater than a second range.

[0095] In the above embodiments, when the head of the second object moves, the spatial area where the head of the second object is located after the head moves is displayed. This allows the user to know whether the sound perception system has timely obtained information about the movement of the head of the second object. This lets the user know that even if the head of the second object moves, the sound masking system can still mask the spatial area where the head of the second object is located, thereby increasing the user's confidence in using the sound masking system and improving the practicality of the sound masking system.

[0096] Optionally, if the head of the second object moves, the alerting device displays the sound masking position after the head movement and the spatial area where the head of the second object is located. By displaying the sound masking position after the head movement and the spatial area where the head of the second object is located, the alerting device allows users to more intuitively determine whether the sound masking system has masked the second object after its head movement, thereby increasing user confidence in using the sound masking system and improving its practicality.

[0097] In another possible implementation, the method further includes: displaying the masking result of masking the target sound for the masking object, the masking result including one or more of masking success, masking failure, or masking warning.

[0098] In the above embodiments, the reminder device displays the masking result of the target sound for the masking object, which allows the user to know the masking result of the target sound at the sound masking location, thereby increasing the user's confidence in using the sound masking system and improving the practicality of the sound masking system.

[0099] Thirdly, this application provides a warning device, including a processing unit and an interaction unit. The processing unit is used to determine the sound source location, the sound masking location, and the sound observation signal inside the cockpit, based on the sound source location, the sound masking location, and the sound observation signal. The processing unit is also used to determine a masking result based on the sound source location, the sound masking location, and the sound observation signal. The interaction unit is used to determine a first warning message based on the aforementioned masking result. Wherein, the sound source location is the location from which the target sound is emitted.

[0100] In one possible implementation, the interaction unit determines the second reminder information when the volume of the target sound is greater than a first threshold, and / or when the distance between the sound source location and the sound masking location is less than a second threshold.

[0101] In another possible implementation, if the volume of the target sound exceeds a first threshold, the second reminder information is used to alert the user that the target sound volume is too high, and / or, the second reminder information is used to remind the user to lower the volume of the target sound. And / or, if the distance between the sound source location and the sound masking location is less than a second threshold, the second reminder information indicates that the distance between the sound source location and the sound masking location is too close, and / or, the second reminder information is used to remind the user to increase the distance between the sound source location and the sound masking location.

[0102] In another possible implementation, the interaction unit is further configured to determine a third reminder message when the volume of the target sound is less than or equal to a first threshold, and / or when the distance between the sound source location and the sound masking location is greater than or equal to a second threshold.

[0103] In another possible implementation, the interaction unit is further configured to receive a first instruction from the user. The processing unit is configured to, based on the first instruction, determine at least one target object within the cockpit, wherein the sound emitted by the target object is the target sound. And / or, the processing unit is configured to, based on the first instruction, determine at least one masking object within the cockpit, wherein the masking object is the object whose target sound is to be masked. Both the target object and the masking object include the user within the cockpit, and / or, a spatial area within the cockpit. The first instruction is used to indicate at least one target object within the cockpit, and / or at least one masking object within the cockpit.

[0104] In another possible implementation, the interaction unit is also used to display at least one target object determined based on the first instruction, and / or at least one masking object.

[0105] In another possible implementation, the interaction unit is further configured to receive a second instruction from the user. The processing unit is configured to adjust at least one target object based on the second instruction, wherein the at least one target object before adjustment is different from the at least one target object after adjustment. And / or, the processing unit is configured to adjust at least one masking object based on the second instruction, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment.

[0106] In another possible implementation, both the first and second instructions described above are user instructions, which include one or more of touch instructions, voice instructions, or gesture instructions. Touch instructions are, for example, instructions generated by a user through touching a human-computer interaction device; voice instructions are, for example, instructions generated by a user through voice control; and gesture instructions are, for example, instructions generated by a user through gesture control.

[0107] In another possible implementation, when the target is a user inside the cockpit, the processing unit is specifically configured to determine the spatial area where the target's mouth is located as the sound source location; or, when the target is a spatial area inside the cockpit, the processing unit is specifically configured to determine part or all of the spatial area of ​​the target as the sound source location. And / or, when the masking target is a user inside the cockpit, the processing unit is specifically configured to determine the spatial area where the masking target's head is located as the sound masking location; or, when the masking target is a spatial area inside the cockpit, the processing unit is specifically configured to determine part or all of the spatial area of ​​the masking target as the sound masking location.

[0108] In another possible implementation, the processing unit is specifically used to determine a first spatial region where the head of the object to be masked is located based on perception technology, and to determine the first spatial region as the sound masking location. The perception technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0109] In another possible implementation, when the head of the object being masked moves to the second spatial region, the processing unit is specifically used to determine the second spatial region as the sound masking location, which is different from the first spatial region.

[0110] In another possible implementation, a third spatial region where the mouth of the target object is located is determined based on perception technology, and the processing unit is specifically used to determine the third spatial region as the location of the sound source. The perception technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0111] In another possible implementation, when the mouth of the target object moves to the fourth spatial region, the processing unit is specifically used to determine the fourth spatial region as the sound source location, which is different from the third spatial region.

[0112] In another possible implementation, the interaction unit is also used to determine the sound masking location, and / or, determine the sound source location.

[0113] In another possible implementation, the interactive device further includes a masking unit for transmitting a masking sound based on the target sound, the masking sound being used to mask the target sound at a sound masking location.

[0114] In another possible implementation, the cockpit includes N sound pickup devices, where N is a positive integer. The processing unit is specifically used to pick up sound observation signals through the N sound pickup devices. The sound observation signals include M sub-sound observation signals picked up by M of the N sound pickup devices, where M is a positive integer less than or equal to N. The sound pickup device is a device that has the function of converting sound (sound waves) into electrical signals; for example, the sound pickup device is a microphone or a pickup unit.

[0115] In another possible implementation, one or more of the M sound pickup devices are located within a first range of the sound masking location.

[0116] In another possible implementation, M=1, and the M sound pickup devices are the first sound pickup device, which is the sound pickup device closest to the sound masking position among the N sound pickup devices.

[0117] In another possible implementation, the processing unit is specifically configured to determine a first signal based on the sound source location, the sound masking location, and M sub-sound observation signals. This first signal describes the signal associated with the target sound at the sound masking location. The processing unit is further configured to determine a masking result based on the first signal. The first signal describes the signal associated with the target sound at the sound masking location; for example, it describes the signal strength of the target sound at the sound masking location. It is understood that the first signal is calculated, not actually measured, and therefore there may be errors between the first signal and the actually measured signal strength of the target sound.

[0118] In another possible implementation, the cockpit includes a masking sound generating device for playing a masking sound to mask target sound at a sound masking location. The processing unit is specifically configured to determine a second signal describing a signal related to the masking sound at the sound masking location. The processing unit is further configured to determine a masking result based on the first and second signals.

[0119] In another possible implementation, the processing unit is specifically used for processing based on the sound observation signal S o Masking acoustic signal S m First transfer function H mo The second transfer function H sd and the third transfer function H so Determine the first signal S sd Sound observation signal S o Masking acoustic signal S m First transfer function H mo The second transfer function H sd The third transfer function H so and the first signal S sd S satisfies the following relationship: sd =(S o -S m *H mo )H sd / H so Among them, the masking acoustic signal S m Used to generate masking sound, the first transfer function H mo The second transfer function H is used to represent the transfer function between the masking sound generation device and the sound pickup device. sd The third transfer function H is used to represent the transfer function between the sound source location and the sound masking location. so This is used to represent the transfer function between the location of the sound source and the sound pickup device.

[0120] In another possible implementation, the processing unit is specifically configured such that the ratio R of the signal strength of the second signal to the signal strength of the first signal is less than R0. L In the case of R, the masking result is determined to be in a failed state. Alternatively, if the ratio R is greater than R... H In the case where the ratio R is greater than or equal to R, the masking result is determined to be valid. L And the ratio R is less than or equal to R H In this case, the masking result is determined to be a warning state. Among them, R L ≤R H .

[0121] In another possible implementation, the aforementioned ratio R is, for example, the first signal S. sd With the second signal S md The short-time energy ratio between them. For example, R, the first signal S sd With the second signal S md The following relationship exists between them:

[0122] In another possible implementation, R L The value range of R includes [5dB, 18dB]. H The value range includes [10dB, 30dB].

[0123] In another possible implementation, the cockpit includes a sound masking system for masking target sounds at a masking location. Specifically, the processing unit is configured to: determine that the masking result is invalid if the signal strength of the first signal is greater than or equal to a third threshold; and / or determine that the masking result is valid if the signal strength of the first signal is less than or equal to a fourth threshold; and / or determine that the masking result is a warning if the signal strength of the first signal is between the third and fourth thresholds; and / or determine that the masking result is invalid if the signal strength of the second signal is less than or equal to a fifth threshold; and / or determine that the masking result is valid if the signal strength of the second signal is greater than or equal to a sixth threshold; and / or determine that the masking result is a warning if the signal strength of the second signal is between the fifth and sixth thresholds.

[0124] In another possible implementation, the aforementioned first reminder information is used to remind the user of the masking result.

[0125] Fourthly, this application provides another alerting device, including a receiving unit and an interaction unit. The receiving unit is used to receive a first instruction from a user, the first instruction indicating at least one target object within the cockpit, the sound emitted by the target object being a target sound. And / or, the first instruction is used to indicate at least one masking object within the cockpit, the masking object being the object whose target sound is to be masked. Both the target object and the masking object include the user within the cockpit or a spatial area within the cockpit. The interaction unit is also used to display at least one target object and / or, at least one masking object, in response to the first instruction.

[0126] In one possible implementation, the receiving unit is configured to receive a second instruction from a user, the second instruction indicating adjustment of at least one target object, wherein the at least one target object before adjustment is different from the at least one target object after adjustment. And / or, the second instruction indicates adjustment of at least one masking object, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment. The interaction unit is configured to, in response to the second instruction, display the at least one target object after adjustment, and / or display the at least one masking object after adjustment.

[0127] In another possible implementation, the first instruction is a user instruction, which includes one or more of touch instructions, voice instructions, or gesture instructions.

[0128] In another possible implementation, the interaction unit is also used to display volume information of the target sound. Alternatively, if the volume of the target sound is less than or equal to a first threshold, information indicating that the volume of the target sound is within a suitable range is displayed; or if the volume of the target sound is greater than the first threshold, information indicating that the volume of the target sound is too high is displayed. The information indicating that the volume of the target sound is too high includes information reminding the user to lower the volume of the target sound.

[0129] In another possible implementation, the at least one target object includes a first object, the location from which the first object emits sound is the sound source location, and the at least one masking object includes a second object, the location from which the second object receives sound is the sound masking location. The interaction unit is also configured to display information about the distance between the sound source location and the sound masking location. And / or, if the distance between the sound source location and the sound masking location is greater than or equal to a second threshold, information indicating that the distance between the sound source location and the sound masking location is within a suitable range is displayed; or, if the distance between the sound source location and the sound masking location is less than the second threshold, information indicating that the distance between the sound source location and the sound masking location is too small is displayed. The information indicating that the distance between the sound source location and the sound masking location is too small includes information reminding the user to increase the distance between the sound source location and the sound masking location.

[0130] In another possible implementation, when the second object is a user inside the cabin, the interaction unit is further configured to display the spatial area where the head is located after the head movement, in the event that the second object's head moves. Here, "movement of the second object's head" includes the head moving a range greater than a second range.

[0131] In another possible implementation, the interactive unit is also used to display the masking result for masking the target sound of the masking object, the masking result including one or more of the following: masking success, masking failure, or masking warning.

[0132] Fifthly, this application provides an interactive device including a processor and a communication interface. The communication interface is used for outputting and / or outputting data (including instructions), and / or for receiving and / or sending data. When the processor executes program instructions in memory, it implements the method described in the first aspect or any possible embodiment of the first aspect, or implements the method described in the second aspect or any possible embodiment of the second aspect.

[0133] Sixthly, this application provides an interactive device including a processor and a memory. The memory stores a computer program, and the processor invokes the computer program to implement the method described in the first aspect or any possible implementation of the first aspect, or to implement the method described in the second aspect or any possible implementation of the second aspect.

[0134] Optionally, the interactive device described in the third or fourth aspect above includes a chip, or the computing device includes a chip system of multiple chips.

[0135] Alternatively, the computing device may be a computing platform.

[0136] In a seventh aspect, this application provides a reminder system, including the interactive device of the fourth or fifth aspect, which can be installed in a vehicle.

[0137] Eighthly, this application provides a vehicle that includes the alerting device described in the third or fourth aspect, or the alerting device described in the fifth aspect, or the alerting device described in the sixth aspect, or the alerting system described in the seventh aspect.

[0138] Ninthly, this application provides a computer program product, including program instructions or executable computer program code, which, when executed by at least one processor, implements the method described in the first aspect or any possible implementation of the first aspect, or implements the method described in the second aspect or any possible implementation of the second aspect.

[0139] In a tenth aspect, this application provides a computer-readable storage medium storing program instructions that, when executed by a processor, implement the method described in the first aspect or any possible implementation of the first aspect, or implement the method described in the second aspect or any possible implementation of the second aspect. Attached Figure Description

[0140] The accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0141] Figure 1 is a schematic diagram of the working process of a sound masking system provided in this application;

[0142] Figure 2 is a schematic diagram of a sound masking system provided in this application;

[0143] Figures 3A and 3B are schematic diagrams of the control interface of an interactive device provided in this application;

[0144] Figure 4 is a schematic diagram of the seat distribution in a vehicle cabin provided in this application;

[0145] Figure 5 is a structural schematic diagram of a reminder system provided in this application;

[0146] Figure 6 is a schematic diagram of the distribution of a sound pickup device provided in this application within the cockpit;

[0147] Figure 7 is a schematic diagram of an interactive device provided in this application;

[0148] Figure 8 is a schematic diagram of a transfer function provided in this application;

[0149] Figure 9 is a schematic diagram of another transfer function provided in this application;

[0150] Figure 10 is a schematic diagram of another transfer function provided in this application;

[0151] Figure 11 is a flowchart illustrating a reminder method provided in this application;

[0152] Figures 12A to 12C are schematic diagrams of the operation of a reminder device provided in this application;

[0153] Figure 13 is a schematic diagram of another reminder device provided in this application;

[0154] Figures 14A to 14C are schematic diagrams illustrating the operation of another reminder device provided in this application;

[0155] Figures 15A and 15B are schematic diagrams of a scenario provided in this application;

[0156] Figures 16A and 16B are schematic diagrams of another scenario provided in this application;

[0157] Figure 17 is a flowchart illustrating another reminder method provided in this application;

[0158] Figures 18A to 18G are schematic diagrams of a method for sending user instructions provided in this application;

[0159] Figure 19 is a schematic diagram of the interface of a reminder device provided in this application;

[0160] Figure 20 is a schematic diagram of two scenarios for users to send voice commands provided in this application;

[0161] Figure 21 is a schematic diagram of a gesture command provided in this application;

[0162] Figures 22A and 22B are schematic diagrams of two more gesture commands provided in this application;

[0163] Figures 23A to 23D are schematic diagrams of the interface of another reminder device provided in this application;

[0164] Figure 24 is a schematic diagram of the interface of another reminder device provided in this application;

[0165] Figures 25A to 25C are schematic diagrams of another method for sending user instructions provided in this application;

[0166] Figure 26 is a schematic diagram of the interface of another reminder device provided in this application;

[0167] Figure 27 is a schematic diagram of another application scenario provided in this application;

[0168] Figures 28A and 28B are schematic diagrams of the interface of another reminder device provided in this application;

[0169] Figure 29 is a schematic diagram of the interface of another reminder device provided in this application;

[0170] Figures 30A and 30B are schematic diagrams of another application scenario provided by this application;

[0171] Figures 31A and 31B are schematic diagrams of the interface of another reminder device provided in this application;

[0172] Figure 32 is a flowchart illustrating another reminder method provided in this application;

[0173] Figure 33 is a structural schematic diagram of an interactive device provided in this application;

[0174] Figure 34 is a schematic diagram of another interactive device provided in this application. Detailed Implementation

[0175] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described below with reference to the accompanying drawings.

[0176] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0177] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0178] It should be understood that in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0179] The following is an explanation of some of the terms used in this application. It should be noted that these explanations are for the convenience of those skilled in the art and are not intended to limit the scope of protection claimed in this application.

[0180] (1) Privacy Voice Shield Technology.

[0181] Privacy Sound Shield technology is a technique that utilizes acoustic principles to achieve voice confidentiality. It works by playing masking noise at a sufficiently high volume in the masked area / location, significantly reducing the intelligibility of the target voice and thus achieving confidentiality. One possible implementation utilizes anti-phase sound wave technology to substantially weaken or cancel the target voice at the masked location / location, thereby achieving the effect of masking the target voice.

[0182] (2) Sound masking system.

[0183] As shown in Figure 1, a sound masking system typically includes a sound pickup device, a masking signal generation device, and a masking sound generation device. The sound pickup device picks up sound signals and outputs the target sound signal; the masking signal generation device generates a masking sound signal based on the target sound signal; and the masking sound generation device plays a masking sound based on the masking sound signal, which is used to mask the target sound. The target sound is the sound that needs to be masked at the sound masking location (e.g., the location of the driver's ear inside the vehicle). The sound source location shown in Figure 1 is the location from which the target sound is emitted; for example, the sound source location is the location of the mouth of a rear passenger inside the vehicle.

[0184] The working process of the sound masking system shown in Figure 1 is as follows: The "sound pickup device" picks up the sound signal (direct sound wave) from the sound source location and detects the target sound signal from the sound signal. Then, the "masking signal generating device" generates a masking signal and outputs the masking sound signal to the "masking sound generating device" near the sound masking location. Finally, the "masking sound generating device" near the sound masking location plays the masking sound. At this time, at the sound masking location, the masking sound masks and interferes with the sound signal (direct sound wave) from the sound source location, thereby achieving the effect of sound masking. This application does not limit how the "masking signal generating device" generates the masking sound signal based on the target sound signal. For example, the "masking signal generating device" can determine a masking sound with the same spectrum as the target sound but opposite phase (180 degrees out of phase).

[0185] Understandably, sound masking systems have a wide range of applications, including vehicles, offices, medical facilities, financial institutions, educational institutions, and other locations requiring confidentiality. The following section will provide an example of a sound masking system in a vehicle scenario.

[0186] Please refer to Figure 2, which is a schematic diagram of a sound masking system provided in this application. As shown in Figure 2, the sound masking system includes a sound pickup device, a masking signal generation device, and a masking sound generation device. The sound pickup device is positioned close to the passenger emitting the target sound (or other audio-emitting devices, such as mobile phones, speakers, etc.) to pick up the target sound. The passenger emitting the target sound can be a passenger in any seat in the vehicle; for example, in the scenario shown in Figure 2, the passenger sitting to the right rear of the seat is the passenger emitting the target sound. The masking signal generation device receives the signal of the target sound and outputs a masking sound to the masking sound generation device. The masking sound generation device then plays the masking sound according to the masking sound signal, thereby generating a masking area to mask the target sound. The masking area shown in Figure 2 is, for example, the area where the driver's ear / head is located in the front seat of the vehicle. It should be noted that the positions and sizes of the sound pickup device, masking signal generation device, masking sound generation device, and masking area in Figure 2 are exemplary and should not be construed as limiting this application. Understandably, multiple masking sound generating devices can be installed inside the vehicle, and one or more masking areas can be generated simultaneously.

[0187] Optionally, the sound masking system shown in Figure 2 may further include an interactive device, such as a mobile terminal or a display device in the vehicle cabin, for interacting with the driver and passengers. The display device in the vehicle cabin can be deployed on the center console, behind the seat headrests, or on the rear center console. Please refer to Figure 3A, which is a schematic diagram of the control interface of an interactive device provided in this application. On the control interface shown in Figure 3A, the user can select to mask a specified area according to the text prompts. For example, if the user clicks the "Mask Passenger Seat" control, the control interface will adjust as shown in Figure 3B. In this state, the sound masking system will mask the voices of the driver and passenger seats, preventing them from hearing / understanding conversations of the rear passengers. It should be noted that in the control interfaces shown in Figures 3A and 3B, when a control is "lit up," it indicates that sound masking is enabled for the specified area / location; when a control is grayed out, it indicates that sound masking for the specified area / location is disabled.

[0188] Please refer to Figure 4, which is a schematic diagram of the seating arrangement in a vehicle cabin provided in this application. As shown in Figure 4, the vehicle 400 has front and rear seats, including a driver's seat 401, a front passenger seat 402, a rear left seat 403, a rear middle seat 404, and a rear right seat 405. Combined with the sound masking system shown in Figure 2, passengers can make private calls within the vehicle 400 via an interactive device. For example, when passengers in the rear left seat 403 and rear right seat 405 are talking, the rear passengers can use the interactive device of the sound masking system to mask the conversation from the passengers in the front driver's seat 401 and front passenger seat 402. Similarly, when the passenger in the driver's seat 401 (i.e., the driver) is making a phone call, the driver can use the interactive device of the sound masking system to prevent the conversation from being heard / understood by the passenger in the front passenger seat 402 and the rear passengers.

[0189] However, as described in the background section, the level of confidentiality or speech intelligibility in secure areas achieved by sound masking systems in practice is not stable. It is affected by various factors, such as the volume of the target sound or the obstruction of the masking sound generation device. This leads to a lack of confidence among users when using sound masking systems, affecting their practicality.

[0190] In view of this, this application provides a reminder method, system, device, vehicle and related products that can analyze and determine the masking effect of a target sound in a masking area, and remind the user based on the analyzed masking effect, so that the user can obtain the current masking effect in a timely manner, thereby enhancing the user's confidence in using the sound masking system and improving the practicality of the sound masking system.

[0191] The solution provided in this application embodiment can be implemented by the reminder system 500 shown in FIG5. As shown in FIG5, the reminder system 500 includes, but is not limited to: a control device 510, a sensor device 520, a sound pickup device 530, a communication device 540, and an interaction device 550.

[0192] The control device 510 can be connected to the sensor device 520, the sound pickup device 530, the communication device 540, and the interaction device 550. The control device 510 can be a platform with processing capabilities, such as hardware, software, or a combination of hardware and software, for example, it can be implemented as an in-vehicle computing platform and / or a cockpit domain control platform. This application does not limit this.

[0193] As an example of hardware implementation, the control device 510 may include at least one processor, which is a circuit with processing capabilities. In one implementation, the processor may be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, microcontroller unit (MCU), graphics processing unit (GPU), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits, which may be fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement a response function. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In some implementations, the control device 510 includes at least one processor integrated as a system-on-chip (SOC), which is commonly referred to as an SOC by those skilled in the art. The SOC may include at least one processor, and when the SOC includes multiple processors, the types of processors can be different, such as including a CPU and an NPU.

[0194] For example, the alert system 500 can be applied to human-vehicle interaction scenarios, and the control device 510 includes, but is not limited to, a mobile data center (MDC), a domain controller (DC), an electronic control unit (ECU), and a vehicle integrated / integration unit (VIU). Alternatively, the control device 510 may also include one or more of the following: a handheld terminal, a wearable terminal, a vehicle, smart home devices (such as smart refrigerators, televisions, or air conditioners), or network devices. Among these, network devices include, for example, gateways (GW).

[0195] As an example of software implementation, the control device 510 may include software functional units. As another example of a software functional unit, the control device 510 may include computing instances, such as virtual machines, containers, and the cloud. A virtual machine is a computer system with complete hardware system functionality simulated by software and running in an isolated environment. A container is an isolated environment obtained by packaging applications and their dependencies. The cloud is a software platform using application virtualization technology, enabling one or more software applications to be developed and run in an independent virtualization environment. Optionally, the cloud may be deployed on a public cloud, private cloud, or hybrid cloud. As yet another example of a software functional unit, the control device 510 may include code that can run on a computing instance, wherein the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container.

[0196] The sensor device 520 may include, but is not limited to, some or all of the following: image sensors, radar sensors, pressure sensors, or touch sensors.

[0197] The image sensor can be a camera (e.g., an optical camera, a depth camera). An image sensor may include multiple photosensitive units, which can sense light signals and form an image. Examples of photosensitive units include complementary metal-oxide-semiconductor (CMOS), charge-coupled devices (CCD), and Live MOS. Image sensors are typically classified based on the type of photosensitive units they contain. For example, an image sensor with CMOS photosensitive units is called a CMOS image sensor (CIS). A CIS may include one or more of the following: a color sensor, a mono sensor, etc. A color sensor may include a red-green-blue (RGB) sensor. Image sensors can be used to acquire images inside a vehicle cabin. This application does not specifically limit the number or deployment location of image sensors.

[0198] Radar sensors are devices that detect objects by emitting detection signals and receiving the echoes returned from targets in space. Based on the echo processing, radar sensors can obtain relevant information about the target in the perceived space, such as distance, position, angle, speed, reflectivity, reflection intensity, image, color, texture, and material, among other things. Depending on the type of detection signal emitted, radar sensors can be categorized into millimeter-wave radar, centimeter-wave radar, ultrasonic radar, and lidar. Radar sensors can be used to detect passengers inside vehicle cabins.

[0199] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor can be installed in devices accessible to the user, such as steering wheels, displays, seat armrests, seat backs, or headrests. The pressure sensor can process the collected pressure signals to obtain touch operations or gestures made by the user on these devices. Pressure sensors can include resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may consist of at least two parallel plates with conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes.

[0200] Touch sensors, also known as "touch devices," can be placed on displays or steering wheels. A touchscreen, also called a "touch panel," consists of touch sensors and a display screen. Touch sensors are used to detect touch operations applied to or near the device.

[0201] The sound pickup device 530 can be a device that converts sound waves into electrical signals, also known as a microphone or transducer. As shown in Figure 6, the sound pickup device 530 can include sound pickup devices 530-1 to 530-12, wherein sound pickup devices 530-1 to 530-3 can be deployed in front of the driver, sound pickup devices 530-4 and 530-5 can be deployed to the side of the driver, sound pickup devices 530-6 and 530-7 can be deployed at the headrest position of the driver, sound pickup devices 530-8 and 530-9 can be deployed to the side of the rear driver, and sound pickup devices 530-10 to 530-12 can be deployed at the headrest position of the rear driver. It should be understood that the deployment locations and quantities of the aforementioned sound pickup devices are merely illustrative and do not represent all deployment methods. This application does not specifically limit the deployment locations and quantities of sound pickup devices. However, it should be understood that all deployment methods that meet the requirements of this application are within the protection scope of this application.

[0202] Communication device 540 refers to a device capable of receiving wireless signals, and optionally also capable of transmitting wireless signals. The wireless technology used by communication device 540 includes one or more of the following: (long term evolution-based communication technologies or fifth-generation mobile networks or 5th-generation wireless systems, 5G, etc.). In some implementations, alert system 500 may not include communication device 540. For example, if alert system 500 includes the masking signal generating device shown in Figure 2 above, alert system 500 may not include communication device 540.

[0203] The interactive device 550 has a reminder function. For example, the interactive device 550 can send reminder information to the user through one or more of the following: images, text, animation, lights, voice broadcasts, or haptic stimulation. The interactive device 550 may include a display, lights, speakers, seats, or steering wheel in the cockpit. The interactive device 550 may also include linked wearable devices or linked personal electronic terminal devices, such as mobile phones, watches, bracelets, smart glasses, or smartwatches linked to the vehicle. Please refer to Figure 7, which is a schematic diagram of an interactive device provided by this application. As shown in Figure 7, the interactive device 550 includes ambient lights, screen A, screen B, and screen C. The ambient lights surround the driver and passengers, including front ambient lights and side ambient lights. Screen A may be a central control screen, screen B may be a screen behind the driver's headrest, and screen C may be a screen behind the passenger's headrest. The ambient lights may flash different colors and / or different frequencies of light to indicate different information. Screens A to C may also display text or patterns to indicate different information. It should also be understood that the interactive device shown in Figure 7 above is illustrative and not intended to limit the scope of this application. However, it should be understood that all deployment methods that meet the requirements of this application are within the protection scope of this application.

[0204] The interactive device 550 can also receive various commands from the user, including one or more of touch commands, voice commands, or gesture commands. For example, the interactive device 550 can acquire touch commands from the user through the touch sensor in the sensor device 520, including one or more of taps, double taps, or swipes. As another example, the interactive device 550 can also acquire voice commands from the user through the voice pickup device 530. As yet another example, the interactive device 550 can also acquire gesture commands from the user through the image sensor, radar sensor, or pressure sensor in the sensor device 520. Furthermore, the interactive device can also acquire fused commands through one or more of the sensor device 520 or the voice pickup device 530, such as a fused command of voice and gesture commands, or a fused command of touch and gesture commands.

[0205] The aforementioned alert system 500 may also include the sound masking system shown in FIG1 or FIG2. The interactive device 550 in the aforementioned alert system 500 and the interactive device in the aforementioned sound masking system may be the same device. For example, the interactive device 550 and the interactive device in the sound masking system may both include screen A, screen B and screen C shown in FIG7.

[0206] Optionally, the multiple modules included in the reminder system 500 can be set independently, integrated together, or partially designed independently and partially integrated. For example, sensor device 520 and interaction device 550 can be integrated together. Another example is the integration of sound pickup device 530 and sensor device 520. Yet another example is the integration of sound pickup device 530 and interaction device 550. Another example is the integration of communication device 540 and interaction device 550. Yet another example is the integration of sensor device 520 and sound pickup device 530 in one device, and control device 510, communication device 540, and interaction device 550 in another device. Yet another example is the integration of control device 510 and communication device 540 in device A, sensor device 520 and sound pickup device 530 in device B, and interaction device 550 in device C, where device A, device B, and device C are three different devices.

[0207] It should be understood that the following description is for ease of understanding and explanation only, using a control device in a vehicle as the executing entity to illustrate the method provided in the embodiments of this application. The control device can be, executively, the control device 510 in Figure 5 above. This control device can be a component in the vehicle, such as a chip, chip system, or other functional module capable of calling and executing programs. However, it should be understood that this does not constitute any limitation on the executing entity of the method provided in this application. In this application, the control device 510 can also be referred to as a reminder device. Optionally, the reminder device may also include ambient lighting, speakers, display devices, or motion-sensing devices, etc. The display device can be a central control screen, driver and passenger entertainment screen, or rear entertainment screen in the vehicle, etc. The motion-sensing device can be the steering wheel, seat, or wearable devices for the driver and passengers in the vehicle, etc.

[0208] To better understand the reminder scheme provided in this application, the following will provide an illustrative introduction to the terms and symbols that may be used in the text.

[0209] 1) Target sound.

[0210] The target sound is the sound that needs to be masked by the "sound masking system". The target sound is usually the sound made by the mouth. In some scenarios, the target sound can also be the sound emitted by the sound playback device, such as a mobile phone, camera, tablet or laptop.

[0211] 2) Location of the sound source.

[0212] The sound source location can be, for example, the initial location from which the target sound is emitted. For instance, if the target sound is a sound emitted by a 'mouth,' the sound source location is the location of the 'mouth.' Alternatively, if the target sound is emitted by a sound playback device, the sound source location is the location of the sound playback device, or the location of the speaker of the sound playback device. This application does not limit the coordinate system used to represent the sound source location; for example, a vehicle coordinate system can be used. Optionally, the sound source location can refer to one or more spatial points, one or more spatial surfaces, or one or more spatial regions; this application does not limit this. For example, if the target sound is a sound emitted by a 'mouth,' the sound source location can refer to a spatial point, or the sound source location can refer to the spatial region where the 'mouth' is located. Another example is if the target sound is emitted by a sound playback device, the sound source location can refer to a spatial point / region (for a single-speaker sound playback device), or multiple spatial points / regions (including multiple-speaker sound playback devices).

[0213] 3) Sound masking location.

[0214] A sound masking location can be understood as the location where the target sound needs to be masked. For example, the masking area shown in Figure 2 above can be called a sound masking location. It should be noted that a sound masking location can also refer to one or more spatial points, one or more spatial surfaces, or one or more spatial regions; this application does not limit this. For instance, a sound masking location can refer to the area where the head / ears of the person to be masked are located. Referring to the "masking the driver's seat" shown in Figure 3A above, if the person to be masked is in the driver's seat, then the sound masking location is the spatial area where the head / ears of the person in the driver's seat are located.

[0215] Optionally, the sound-masking location is also known as a secure area.

[0216] It should also be noted that the aforementioned sound source location and sound masking location can be determined by collecting information from inside the vehicle using one or more devices such as optical cameras, depth cameras, or millimeter-wave sensors, combined with image recognition, machine vision, and other methods. Optionally, there can be multiple sound source locations and sound masking locations, but for the same target sound, the sound source location and sound masking location cannot overlap or partially overlap.

[0217] 4) Covering sound.

[0218] Masking sound is noise used to mask the sound of a target. It can interfere with the target sound at the masking location, significantly reducing the intelligibility of the target sound and achieving secrecy. For example, masking sound can be a reversed version of the target sound. A reversed sound is the sound obtained by reversing the original sound (target sound) along the time axis.

[0219] 5) Masking sound generation device.

[0220] A masking sound generating device, also known as a masking sound loudspeaker, is used to play masking sounds. A vehicle may have one or more masking sound generating devices. When multiple masking sound generating devices play masking sounds, multiple sound masking positions can be generated. It should be noted that this application does not limit the number of masking sound generating devices inside a vehicle.

[0221] 6) Target sound signal S t This is the electrical signal converted from the target sound by the sound pickup device. It can be understood that generating the target sound signal S... t The sound pickup device is located close to the sound source to facilitate the acquisition of a true and complete target sound signal S. t To distinguish it from the sound pickup device that generates the sound observation signal S0 in the following text, the sound pickup device located near the sound source can be referred to as the first sound pickup device, and the sound pickup device that generates the sound observation signal S0 in the following text can be referred to as the second sound pickup device.

[0222] 7) Masking sound signal S m , is the electrical signal output by the masking signal generator, which enables the masking sound generator to play masking sounds based on the masking signal. When there are multiple masking sound generators, the masking sound signal S... m It is a multi-channel system used to generate multiple sound masking locations.

[0223] 8) The sound observation signal S0 can be an electrical signal converted by the sound pickup device from the sound passing through the sound masking position, or it can be an electrical signal output by one or more target sound pickup devices. The target sound pickup device can be the sound pickup device closest to the sound masking position, or it can be one or more sound pickup devices located on the first side of the sound masking position, where the first side is the side of the sound masking position away from the sound source position.

[0224] 9) The transfer function of the target sound from the sound source location to the sound masking location is H. sd Optionally, when there are multiple sound source locations, there exists a transfer function H between each sound source location and the sound masking location. sd To make it easier to distinguish, H can be used. sd1H sd2 Similarly, when there are multiple sound masking locations, there exists a transfer function H between the sound source location and each sound masking location. sd Similarly, H can be encoded to represent different propagation paths. sd To make distinctions.

[0225] 10) The transfer function of the target sound from the sound source location to the target sound pickup device is H. so Optionally, when there are multiple sound source locations, each sound source location has a transfer function H between it and the target sound pickup device. so To make it easier to distinguish, H can be used. so1 H so2 Similarly, when there are multiple target sound pickup devices, there exists a transfer function H between the sound source location and each target sound pickup device. so Similarly, H can be encoded to represent different propagation paths. so To make distinctions.

[0226] 11) The transfer function of the masking sound from the location of the masking sound generator to the sound masking location is H. md Optionally, when there are multiple masking sound generating devices, there may also be multiple sound masking locations, with each masking sound generating device uniquely corresponding to one sound masking location. In this case, there exists a transfer function H between each masking sound generating device and its corresponding sound masking location. md To make it easier to distinguish, H can be used. md1 H md2 Etc. are used to indicate this.

[0227] 12) The transfer function of the masking sound propagating from the location of the masking sound generator to the target sound pickup device is H. mo Optionally, when there are multiple masking sound generating devices, each masking sound generating device and the target sound picking device have a transfer function H. mo To make it easier to distinguish, H can be used. mo1 H mo2 Similarly, when there are multiple target sound pickup devices, there exists a transfer function H between the masking sound generator and each target sound pickup device. mo Similarly, H can be encoded to represent different propagation paths. mo To make distinctions.

[0228] It should be noted that the above transfer function H sd H so Hmd and H mo The transfer function can be predetermined. For example, it can be determined using theoretical modeling or experimental measurement methods. Theoretical modeling methods include determining the transfer function based on physical equation derivation or vertical simulation calculations, while experimental measurement methods include impulse response measurement or frequency response measurement. It should also be noted that the alerting device can update the transfer function H of the masking sound between the masking sound generation device and the target sound pickup device based on the masking sound signal and the sound observation signal. mo Alternatively, update the transfer function H of the masking sound between the masking sound generating device and the secure area. md For example, the alerting device can use an adaptive filtering algorithm to process the masking sound signal and sound observation signal acquired in real time, and update the transfer function H. mo In order to obtain the transfer function H mo The reminder device is then combined with the transfer function H. mo and transfer function H md And through H md *H mo ' / H mo The updated transfer function H is calculated. md '.

[0229] To further explain the above transfer function H sd H so H md and H mo The following is a further description with reference to the specific accompanying drawings.

[0230] Please refer to Figure 8, which is a schematic diagram of a transfer function provided in this application. For a description of the sound masking position 1, target sound pickup device 530-2, sound pickup device 530-1, masking signal generation device, and masking sound generation device 801 shown in Figure 8, please refer to the descriptions in Figures 1 and 2 above, or other corresponding content; they will not be repeated here. The target sound pickup device 530-2 is used to pick up the sound observation signal S0. Figure 8 includes a sound masking position and a sound source position. The transfer functions related to the target sound and the observed sound are as follows: The transfer function of the target sound between the sound source position and the sound masking position 1 can be represented by H... sd This indicates that the transfer function between the target sound and the sound source location and the target sound pickup device 530-2 can be expressed as H. so This indicates that the transfer function of the masking sound between the masking sound generating device 801 and the sound masking position 1 can be expressed as H. md This indicates that the transfer function of the masking sound between the masking sound generating device 801 and the target sound picking device 530-2 can be expressed as H. moOptionally, the target sound pickup device 530-2 is the sound pickup device in the cockpit that is closest to the sound masking position.

[0231] Please refer to Figure 9, which is a schematic diagram of another transfer function provided in this application. For a description of the sound masking position 1, sound masking position 2, target sound pickup device 530-2, target sound pickup device 530-3, sound pickup device 530-1, masking signal generation device, masking sound generation device 901, and masking sound generation device 902 shown in Figure 9, please refer to the descriptions in Figures 1 and 2 above or other corresponding content; they will not be repeated here. The target sound pickup device 530-2 is used to pick up the sound observation signal S. 01 The target sound pickup device 530-3 is used to pick up the sound observation signal S. 02 Figure 9 includes two sound masking locations and two sound source locations. The transfer functions related to the target sound and the observed sound are shown below: The transfer function of the target sound between the sound source location and sound masking location 1 can be represented by H. sd1 This indicates that the transfer function between the target sound and the sound source location and the target sound pickup device 530-2 can be expressed as H. so1 This indicates that the transfer function of the masking sound between the masking sound generating device 901 and the sound masking position 1 can be expressed as H. md1 This indicates that the transfer function of the masking sound between the masking sound generating device 901 and the target sound picking device 530-2 can be expressed as H. mo1 The transfer function of the target sound between the sound source location and the sound masking location 2 can be represented by H. sd2 This indicates that the transfer function between the target sound and the sound source location and the target sound pickup device 530-3 can be expressed as H. so2 This indicates that the transfer function of the masking sound between the masking sound generating device 902 and the sound masking position 2 can be expressed as H. md2 This indicates that the transfer function of the masking sound between the masking sound generating device 902 and the target sound picking device 530-3 can be expressed as H. mo2 Optionally, target sound pickup device 530-2 is the sound pickup device in the cockpit that is closest to sound masking position 1, and target sound pickup device 530-3 is the sound pickup device in the cockpit that is closest to sound masking position 2. Optionally, target sound pickup device 530-2 is the sound pickup device in the cockpit that is closest to sound masking position 1. Optionally, target sound pickup device 530-3 is the sound pickup device in the cockpit that is closest to sound masking position 2.

[0232] Please refer to Figure 10, which is a schematic diagram of another transfer function provided in this application. For a description of the sound masking position 1, target sound pickup device 530-2, target sound pickup device 530-3, sound pickup device 530-1, masking signal generation device, and masking sound generation device 1001 shown in Figure 9, please refer to the descriptions in Figures 1 and 2 above, or other corresponding content; they will not be repeated here. The target sound pickup device 530-2 is used to pick up the sound observation signal S. 01 The target sound pickup device 530-3 is used to pick up the sound observation signal S. 02 Figure 10 includes one sound masking location and two sound source locations. The transfer functions related to the target sound and the observed sound are shown below: The transfer function of the target sound between the sound source location and sound masking location 1 can be represented by H. sd1 This indicates that the transfer function between the target sound and the sound source location and the target sound pickup device 530-2 can be expressed as H. so1 This indicates that the transfer function between the target sound and the sound source location and the target sound pickup device 530-3 can be expressed as H. so3 This indicates that the transfer function of the masking sound between the masking sound generating device 901 and the sound masking position 1 can be expressed as H. md1 This indicates that the transfer function of the masking sound between the masking sound generating device 901 and the target sound picking device 530-2 can be expressed as H. mo1 This indicates that the transfer function of the masking sound between the masking sound generating device 901 and the target sound picking device 530-3 can be expressed as H. mo3 Optionally, the target sound pickup device 530-2 and the target sound pickup device 530-3 are sound pickup devices located near the sound masking position 1 in the cockpit.

[0233] The scenarios shown in Figures 8 to 10 above sequentially include scenarios corresponding to one target sound pickup device and one sound masking position, two target sound pickup devices and two sound masking positions, and two target sound pickup devices and one sound masking position. It should be noted that the scenarios shown in Figures 8 to 10 are exemplary and not intended to limit the scope of this application. Furthermore, the scenarios shown in Figures 8 to 10 can be combined to determine more scenarios, for example, a scenario including four target sound pickup devices and two sound masking positions. Another example is a scenario including two target sound pickup devices and four sound masking positions.

[0234] The methods of the embodiments of this application will be described below.

[0235] Please refer to Figure 11, which is a flowchart illustrating a reminder method provided in an embodiment of this application. Optionally, this method is applied to a reminder system, such as the reminder system 500 shown in Figure 5, or for example, the method is executed by the control device 510 (also known as the reminder device) shown in Figure 5. For ease of description, the following description uses the reminder device as the executing entity.

[0236] The reminder method shown in Figure 11 may include one or more steps from S1101 to S1103. It should be understood that, for ease of description, the steps S1101 to S1103 are described in this order, but this embodiment does not limit the order of execution, the execution time, or the number of executions of the above one or more steps. Steps S1101 to S1103 are as follows:

[0237] S1101, The alerting device determines the location of the sound source, the sound masking location, and the sound observation signal inside the cockpit.

[0238] For a description of the reminder device, please refer to the description of the control device 510 above, which will not be repeated here.

[0239] The location of the sound source can be referred to the relevant description in "2) Sound Source Location" above. There can be one or more sound source locations. For example, the sound source location could be the rear right seat 405 in Figure 4 above, or the sound source location could be the rear left seat 403 and the rear right seat 405 in Figure 4 above. As another example, the sound source location could be the mouth of a passenger in the rear right seat 405 in Figure 4 above, or the sound source location could be the mouth of a passenger in the rear left seat 403 and the rear right seat 405 in Figure 4 above. For ease of description, the following example uses a scenario where there is one sound source location (as shown in Figures 8 to 10 above) for illustrative explanation.

[0240] The location of the sound source is related to the target object. The sound emitted by the target object includes the target sound itself. The target object includes the user inside the cabin and / or the space area inside the cabin. Next, the location of the sound source will be described in different cases depending on the target object.

[0241] Scenario 1: The target audience includes users inside the cockpit.

[0242] For example, the cabin includes Xiaoming, Xiaozhang, Xiaowang, and Xiaoli. The target object can be one or more of them. For instance, the target objects include Xiaoming and Xiaozhang. The reminder device can identify Xiaoming and Xiaozhang in the cabin, as well as their seats, through the sensor device 520. Even if Xiaoming and Xiaozhang's seats change (for example, Xiaozhang moves from the right rear seat to the left rear seat), the target objects remain Xiaoming and Xiaozhang. When the target object is a user, the reminder device will detect the user in the cabin in real time through the sensor device to determine the location of the sound source. Therefore, the location of the sound source will continuously move as the user moves.

[0243] Optionally, when the target is a user, the spatial area where the user's mouth is located can be determined as the sound source location. In this case, the alerting device needs to use a sensor device to detect the spatial area where the user's mouth is located in real time to determine the sound source location.

[0244] Scenario 2: The target area includes the space inside the cockpit.

[0245] For example, the cabin layout is as shown in Figure 4 above, including a driver's seat 401, a front passenger seat 402, a rear left seat 403, a rear middle seat 404, and a rear right seat 405. The space corresponding to each seat in the cabin is, for example, the space above the seat.

[0246] When the target object is a space area inside the cockpit, the sound source location is determined by a part or all of the space area of ​​the target object.

[0247] For example, the target object is the space area corresponding to the driver's seat, and the sound source location includes the entire space area above the driver's seat, or the sound source location includes a portion of the space area above the driver's seat (e.g., the space area near the headrest above the driver's seat). It is understood that when the target object includes the space area within the cabin, the sound source location does not need to continuously move with the user's movement.

[0248] Scenario 3: The target objects include the users inside the cabin and the space area inside the cabin.

[0249] For example, the cabin layout is shown in Figure 4 above, including a driver's seat 401, a front passenger seat 402, a rear left seat 403, a rear middle seat 404, and a rear right seat 405. Each seat in the cabin corresponds to a specific spatial area, such as the space above the seat. It should be noted that the target object includes both the user in the cabin and the spatial area within the cabin; that is, the target object includes the user within the spatial area of ​​the cabin. For example, if the target object includes the spatial area corresponding to the driver's seat, then the sound source location is the user within the spatial area corresponding to the driver's seat.

[0250] The location of the sound source will move as the user moves; for example, the location of the sound source will move as the user moves within the space corresponding to the driver's seat.

[0251] Alternatively, the sound source location can also be the spatial area where the user's mouth is located within the spatial area included in the target object. In this case, the alerting device needs to use a sensor device to detect the spatial area where the user's mouth is located within the spatial area included in the target object in real time, thereby determining the sound source location.

[0252] The sound masking location can be referred to the relevant description in "3) Sound Masking Location" above. There can be one or more sound masking locations. For example, the sound masking location can be the driver's seat 401 in Figure 4, or the sound masking location can be both the driver's seat 401 and the passenger seat 402 in Figure 4. As another example, the sound masking location can be the mouth of a passenger in the driver's seat 401 in Figure 4, or the sound masking location can be the mouth of a passenger in both the driver's seat 401 and the passenger seat 402 in Figure 4. For ease of description, the following example uses a scenario with one sound masking location (as shown in Figure 8 above) for illustrative explanation.

[0253] Sound masking locations are related to the masking target, which is the object whose sound is to be masked. Masking targets include users inside the cockpit and / or the space area inside the cockpit. Next, we will describe sound masking locations based on the different masking targets.

[0254] Scenario 1: The target of the masking includes users inside the cockpit. When the target of the masking is a user inside the cockpit, the spatial area where the user's head is located is determined as the sound masking location. For details, please refer to the relevant description above for "Scenario 1: The target includes users inside the cockpit".

[0255] Scenario 2: The target area includes the space inside the cockpit. When the target area is the space inside the cockpit, a portion or all of that space is designated as the sound masking location. For details, please refer to the description above for "Scenario 2: The target area includes the space inside the cockpit".

[0256] Scenario 3: The target objects include users inside the cockpit and the space within the cockpit. For details, please refer to the description above for "Scenario 3: The target objects include users inside the cockpit and the space within the cockpit".

[0257] The acoustic observation signal inside the cockpit can be referred to the relevant description in "8) Acoustic Observation Signal S0" above. The acoustic observation signal, for example, is an electrical signal converted from the received acoustic signal (sound wave) by the acoustic pickup device. There can be one or more acoustic observation signals inside the cockpit. For example, the acoustic observation signal includes the acoustic observation signal picked up by acoustic pickup device 530-1 in Figure 5, or the acoustic observation signals picked up by acoustic pickup devices 530-1 and 530-2. The following will describe different scenarios depending on whether there are one or more acoustic observation signals inside the cockpit.

[0258] It should be noted that, for ease of understanding, the sound observation signal inside the cockpit is presented as an example using the sound masking location.

[0259] In one possible implementation, the cockpit includes N sound pickup devices. Determining the sound observation signal within the cockpit includes: picking up the sound observation signal through the N sound pickup devices. The sound observation signal includes M sub-sound observation signals picked up by M of the N sound pickup devices, where M is a positive integer less than or equal to N. Specifically, N is a positive integer, for example, N equals 1, 2, or 3, etc. M is a positive integer less than or equal to N, for example, N = 2, M equals 1 or 2.

[0260] Scenario 1: The sound observation signal in the cockpit includes a signal picked up by a sound pickup device, i.e., M=1.

[0261] The sound pickup device that picks up the sound signal can be the sound pickup device closest to the sound masking position among multiple sound pickup devices in the cockpit. For example, as shown in FIG8, the target sound pickup device 530-2 is, for example, the sound pickup device closest to the sound masking position 1 among multiple sound pickup devices in the cockpit.

[0262] Scenario 2: The sound observation signals in the cockpit include signals picked up by multiple sound pickup devices, i.e., M > 1, for example, M = 2 or 3, etc.

[0263] In this case, the signals picked up by multiple sound pickup devices in the cockpit can become observation sub-signals. For example, if there are 10 sound pickup devices in the cockpit, sound observation signals can be picked up by 6 of them. These 6 sound pickup devices will pick up a total of 6 observation sub-signals, with each sound pickup device picking up one observation sub-signal.

[0264] In one possible implementation, one or more of the M sound pickup devices are located within a first range of the sound masking location.

[0265] Optionally, when the sound masking location is a spatial point, the first range can be a spatial range centered on the sound masking location with a radius of a first length, wherein the first length is, for example, 10cm, 20cm, or 30cm. When the sound masking location is a spatial region, the spatial region represented by the first range includes the spatial region represented by the sound masking location. Alternatively, the first range can be used to describe a region whose distance from the boundary of the sound masking location is less than a second length, wherein the second length is, for example, 10cm, 20cm, or 30cm.

[0266] S1102. The reminder device determines the masking result based on the location of the sound source, the location of the sound masking, and the sound observation signal.

[0267] The masking result includes one or more of the following: effective state, ineffective state, or warning state. An effective state means that the masking sound can effectively mask the target sound at the masking location, preventing the masked object from hearing / understanding the target sound. An ineffective state means that the masking sound cannot effectively mask the target sound at the masking location, and the masked object can hear / understand the target sound. A warning state means that the masking sound's masking of the target sound at the masking location may pose a risk of information leakage. Of course, the masking result can include more or fewer types, and this application does not limit this; for example, the masking result can also include effective or ineffective states.

[0268] In one possible implementation, the alerting device determines a first signal based on the sound source location, the sound masking location, and M sub-sound observation signals. This first signal describes the signal related to the target sound at the sound masking location. Based on the first signal, the alerting device determines the masking result.

[0269] In this context, both the sound source location and the sound masking location can be spatial points or spatial regions. When both are spatial points, the sound source location can be represented by P1(X1, Y1, Z1), and the target masking location can be represented by P2(X2, Y2, Z2). When both are spatial regions, the sound source location can be represented by A1X1+B1Y1+C1Z1+D1≥0, and the sound masking location can be represented by A2X2+B2Y2+C2Z2+D2≥0. Here, A1X1+B1Y1+C1Z1+D1=0 represents the plane at the bottom of the sound source location, for example, the plane shown in the image of the left rear seat. A2X2+B2Y2+C2Z2+D2=0 represents the plane at the bottom of the target masking location, for example, the plane shown in the image of the driver's seat. For ease of description, in the following text, P1(X1, Y1, Z1) represents the location of the sound source, and P2(X2, Y2, Z2) represents the location of the target cover.

[0270] When M = 1, the M sub-sound observation signals can be represented by S0. When M > 1, the M sub-sound observation signals can be represented by {S1, S2, ..., S...} M}express.

[0271] For example, the reminder device is based on the sound observation signal S o Masking acoustic signal S m First transfer function H mo The second transfer function H sd and the third transfer function H so Determine the first signal S sd Sound observation signal S o There can be one or more, which will be discussed in detail below.

[0272] Case 1: Sound observation signal S o Includes an acoustic observation signal.

[0273] Please refer to Figure 8, the sound observation signal S o For example, the sound observation signal picked up by the target sound pickup device 530-2 in Figure 8, the masking sound signal S. m For example, the masking sound played by the masking sound generator 801, the first transfer function H mo The second transfer function H is used to represent the transfer function between the masking sound generating device 801 and the sound pickup device 530-2. sd The third transfer function H is used to represent the transfer function between the sound source location and the sound masking location 1. so This is a transfer function used to represent the location of the sound source and the sound pickup device 530-2.

[0274] The reminder device can be accessed via (S) o -S m *H mo )H sd / H so The first signal S was calculated. sd Among them, the first signal S sd S is used to describe the signal related to the target sound at the sound masking location. m *H mo The relevant signal used to represent the masking sound at the target sound pickup device 530-2, (S) o -S m *H mo ) is used to represent the sound observation signal S o The residual sound signal in the sound, or, used to represent the sound observation signal S o Signals other than masking acoustic signals.

[0275] Case 2: Sound observation signal S o It includes multiple sound pickup devices that pick up multiple sound observation signals, with each sound pickup device referred to as an observation channel.

[0276] Please refer to Figure 10, the sound observation signal S o Including the sound observation signals picked up by the target sound pickup device 530-2 and the target sound pickup device 530-3 in Figure 8, and the masking sound signal S m For example, the masking sound played by the masking sound generator 1001, the first transfer function H mo Including the transfer function H between the masking sound generating device 1001 and the sound pickup device 530-2 mo1 The transfer function H between the masking sound generating device 1001 and the sound pickup device 530-3 mo2 The second transfer function H sd Including the transfer function H between the sound source location and the sound masking location 1 sd1 The third transfer function H so Including the transfer function H between the sound source location and the sound pickup device 530-2 so1 The transfer function H between the sound source location and the sound pickup device 530-3 so2 Therefore, S can be represented using matrices. o S m H mo and H so Each column in the matrix corresponds to an observation channel. For example, S can be represented by [S1 S2]. o Then S1 corresponds to the first observation channel, and S2 corresponds to the second observation channel. m H mo and Hso It's similar, so I won't go into details here.

[0277] The reminder device can also be accessed via (S) o -S m *H mo )H sd / H so The first signal S was calculated. sd Because of S o S m H mo and H so Both are in matrix form, therefore, S m *H mo The asterisk (*) in the H column indicates point-by-point operation. sd / H so The ' / ' operator in the diagram also represents a point-by-point operation. It's understandable that each observation channel can generate a first signal S. sd The sub-signals. In one possible implementation, multiple first signals S can be used. sd The sub-signals are weighted and averaged to generate the first signal S. sd .

[0278] Optionally, each first signal S sd The weighted value corresponding to the sub-signal is inversely correlated with the distance between the corresponding sound pickup device and the sound masking position. In other words, the farther the observation channel is from the sound masking position, the smaller its weight; the closer the observation channel is to the sound masking position, the larger its weight. Taking Figure 10 as an example, the first signal S corresponding to the target sound pickup device 530-2... sd The sub-signal of sd1 This indicates that the first signal S corresponding to the standard sound pickup device 530-3 sd The sub-signal of sd2 This indicates that the first signal S sd S can be calculated using the following formula. sd =W1S sd1 +W2S sd2 Where W1 can be the reciprocal of the distance between the target sound pickup device 530-2 and the sound masking position 1, and W2 can be the reciprocal of the distance between the target sound pickup device 530-3 and the sound masking position 1.

[0279] It is understandable that the above-mentioned "Situation 2, Sound Observation Signal S" o The scheme shown, which includes multiple sound pickup devices, is different from "Scenario 1, sound observation signal S". o The scheme, which includes a sound pickup device, has multiple observation channels, which can more accurately and comprehensively acquire signals inside the cockpit, so that the generated first signal can more accurately describe the relevant signals of the target sound in the sound masking position.

[0280] The alerting device determines the masking result based on the first signal, including the following two cases.

[0281] Scenario 1: The alert device determines the masking result based on the signal strength of the first signal.

[0282] Since the first signal is used to describe the relevant signal of the target sound at the sound masking location, the higher the signal strength of the first signal, the worse the masking effect; conversely, the lower the signal strength of the first signal, the better the masking effect.

[0283] For example, if the signal strength of the first signal is greater than or equal to a third threshold, the alerting device determines the masking result to be invalid. Alternatively, if the signal strength of the first signal is less than or equal to a fourth threshold, the masking result is determined to be valid. Or, if the signal strength of the first signal is between the third and fourth thresholds, the masking result is determined to be a warning state. Typically, the third threshold is greater than the fourth threshold; for example, the third threshold is 20 dB and the fourth threshold is 10 dB. In some implementations, the third threshold is equal to the fourth threshold; for example, both the third and fourth thresholds are equal to 15 dB. In this case, the masking result does not include a warning state. The third and fourth thresholds can be predetermined or customizable, and this application does not limit their specificity.

[0284] Scenario 2: The alerting device determines the masking result based on the signal strength of the first signal and the signal strength of the second signal.

[0285] The second signal is used to describe the signal related to the masking sound at the masking location.

[0286] For example, the alerting device can use a masked sound signal and a fourth transfer function H md The second signal is calculated; for example, the second signal can be obtained through the masking sound signal and the fourth transfer function, which is the transfer function between the masking sound generating device and the sound masking position. As shown in Figure 8, H md This is used to represent the transfer function between the masking sound generation device 801 and the sound masking position 1.

[0287] Understandably, the larger the ratio of the masking sound signal strength to the target sound signal strength at masking location 1, the stronger the masking sound signal strength compared to the target sound, and the better the masking effect. Conversely, the smaller the ratio, the weaker the masking sound signal strength compared to the target sound, and the worse the masking effect. Therefore, the alerting device can first calculate the ratio R between the signal strength of the second signal and the signal strength of the first signal, and then combine this with a preset threshold to determine the masking result.

[0288] For example, the ratio R of the signal strength of the second signal to the signal strength of the first signal is less than R. L In the case of R, the masking result is determined to be in a failed state. Alternatively, if the ratio R is greater than R... H In the case where the ratio R is greater than or equal to R, the masking result is determined to be valid. L And the ratio R is less than or equal to R H In this case, the masking result is determined to be a warning state. Among them, R L ≤R H Among them, R L and R H It can be preset, or measured in advance. For example, R L The value range of R includes [5dB, 18dB]. H The value range includes [10dB, 30dB].

[0289] Optionally, the ratio R can be the short-time energy ratio of the signal strength of the second signal to the signal strength of the first signal. This short-time energy can be calculated by accumulating across the entire frequency band. For example, the ratio R can be calculated using the following formula:

[0290] Optionally, to highlight the importance of certain sub-bands, weighting can be applied during the calculation of the short-time energy of the signal strength of the second signal and the signal strength of the first signal. For example, the ratio R can be calculated using the following formula:

[0291] Among them, G subband represents the weighting coefficient for each spectral energy.

[0292] The sub-band division described above can be customized, and this application does not impose any restrictions on it. For example, the sub-band division can be set according to octaves, 1 / 2 octaves, 1 / 3 octaves, etc.

[0293] Scenario 3: The alerting device can also determine the masking result based on the signal strength of the second signal.

[0294] Based on the above three points, we can see that the second signal describes the signal related to the masking sound at the masking location. The signal strength of the masking sound at the masking location can also indirectly reflect the masking result. Therefore, the higher the signal strength of the second signal, the better the masking effect; conversely, the lower the signal strength of the second signal, the worse the masking effect.

[0295] For example, if the signal strength of the second signal is less than or equal to a fifth threshold, the masking result is determined to be invalid. Alternatively, if the signal strength of the second signal is greater than or equal to a sixth threshold, the masking result is determined to be valid. Or, if the signal strength of the second signal is between the fifth and sixth thresholds, the masking result is determined to be a warning state. Typically, the fifth threshold is less than the sixth threshold; for example, the fifth threshold is 40 dB and the sixth threshold is 60 dB. In some implementations, the fifth threshold is equal to the sixth threshold; for example, both the fifth and sixth thresholds are equal to 50 dB. In this case, the masking result does not include a warning state. The fifth and sixth thresholds can be predetermined or customizable, and this application does not limit their specificity.

[0296] S1103. The reminder device determines the first reminder information based on the masking result.

[0297] The masking result includes one or more of the following: effective status, ineffective status, or warning status. The first reminder information is used to remind the user of the masking result.

[0298] In one possible implementation, the alerting device can alert the user to the masking result through one or more of the following: images, text, animation, light, voice announcement, or haptic stimulation.

[0299] For example, the reminder device displays the first reminder information to the user through a display device such as a vehicle infotainment screen, HUD, or smart headlights.

[0300] For example, the reminder device can display to the user, through the aforementioned device with display function, the words "sound masking function is not working", "sound masking function is working", or "sound masking function is in warning state".

[0301] Furthermore, the reminder device can also display relevant prompts based on the masking result. For example, when the masking result is effective, the displayed prompt is "Voice privacy protection is normal, please use with confidence!", as shown in Figure 12A. When the masking result is ineffective, the displayed prompt is "Privacy protection failed, please speak softly and check the driver's headrest volume!", as shown in Figure 12B. When the masking result is in a warning state, the displayed prompt is "Privacy protection weakened, please speak softly!", as shown in Figure 12C. By setting corresponding prompts, the reminder device can inform the user of the working status of the sound masking system and guide the user to perform relevant operations, thereby ensuring the user's privacy and security as much as possible.

[0302] Optionally, when the reminder device displays the first reminder information to the user through the aforementioned display-enabled device, it may display images, text, animations, etc., through scrolling, flashing, or fixing. For example, when the user is using the sound masking system, the screen icon shown in Figure 13 may be displayed on the display device in the cockpit, and / or the words "Rear seat voice privacy is being protected" may be displayed / scrolled in the status bar of the screen.

[0303] For example, the reminder device plays a first reminder message to the user through a speaker.

[0304] For example, if the masking result is invalid, the alerting device plays a message saying "Security Failed" through the speaker. If the masking result is valid, the alerting device plays a message saying "Security Valid" through the speaker. If the masking result is a warning, the alerting device plays a message saying "Security is Weak" through the speaker.

[0305] Optionally, if the masking result is valid, the reminder device can play a voice message saying "Security valid" through a speaker at a first frequency, where the first frequency can be once per minute, or once every two minutes, etc.

[0306] Optionally, if the masking result is a warning state, the alerting device can play a voice message saying "Security effective" through a speaker at a second frequency, where the second frequency can be twice per minute, or four times per minute, etc.

[0307] Optionally, if the masking result is invalid, the reminder device can play a voice message saying "Security valid" through a speaker at a third frequency, where the third frequency can be five times per minute, or ten times per minute, etc.

[0308] Optionally, the first frequency is lower than the second frequency, and the second frequency is lower than the third frequency. By setting the first frequency to its minimum, the user is alerted that the sound masking system is in an effective state, while also reducing interference from voice announcements. Setting the second frequency to be higher than the first frequency serves two purposes: it alerts the user that the sound masking system is in an early warning state, and it also ensures a certain frequency of alerts, keeping the user constantly aware of the system's status.

[0309] Optionally, when the sound masking system changes from a first state to a second state, the alerting device can play the second state message through a speaker. The first and second states differ; both can be valid, invalid, or warning states.

[0310] For example, the reminder device displays a first reminder message to the user via an ambient light.

[0311] For example, the reminder device can display a first reminder message to the user via the ambient light shown in Figure 8. As an example, when the masking result is valid, the reminder device can control the ambient light to display a continuous or breathing effect in a cool color such as green or blue. When the masking result is a warning, the reminder device can control the ambient light to display a continuous or breathing yellow. When the masking result is invalid, the reminder device can control the ambient light to flash red.

[0312] For example, the reminder device uses haptic stimulation to remind the user of the first reminder information.

[0313] For example, the reminder device can control the vibration / massage frequency of the seat to be in three states: soothing, gentle, and rapid, respectively, and correspondingly remind the user of the masking result as effective, warning, and ineffective.

[0314] Optionally, the reminder device can alert the user to the first reminder by controlling the vibration of the steering wheel.

[0315] Optionally, the reminder device can also send an initial reminder to the user by controlling the ventilation speed of the seat. For example, the seat's gentle breeze, medium breeze, and strong breeze represent the masking result as effective, warning, and ineffective, respectively.

[0316] For example, the reminder device can also send a first reminder message to the user through associated devices of the vehicle. These associated devices include, for example, mobile phones, smartwatches, smart bracelets, and smart glasses.

[0317] For example, the reminder device can push the first reminder information to the mobile phone associated with the vehicle, and push vibration warnings, alarm sound warnings, information and notification bar prompts through the mobile phone.

[0318] For example, the reminder device can push the first reminder information to the smart bracelet associated with the vehicle and notify the user of the first reminder information through vibration.

[0319] For example, the reminder device can push the first reminder information to the smartwatch associated with the vehicle and notify the user of the first reminder information by flashing a light.

[0320] For example, the reminder device can remind the user of the first reminder information in one or more combinations.

[0321] For example, when the privacy protection is valid, the alert device displays "Privacy protection is normal, please rest assured!" on the in-cabin display, while simultaneously controlling the ambient lighting to display a calm blue breathing effect, as shown in Figure 14A. When the privacy protection is in a warning state, the alert device displays "Privacy protection weakened, please speak softly!" on the in-cabin display, while simultaneously controlling the ambient lighting to display a faster yellow breathing effect, as shown in Figure 14B. When the privacy protection is invalid, the alert device displays "Privacy invalid, please speak softly and check the driver's headrest volume!" on the in-cabin display, while simultaneously controlling the ambient lighting to display a flashing red effect, as shown in Figure 14C.

[0322] It should be noted that the various reminder methods shown above can be combined to obtain richer reminder methods or functions. For the sake of brevity, they will not be listed one by one here.

[0323] In one possible implementation, the reminder method shown in FIG11 above further includes: when the volume of the target sound is greater than a first threshold, and / or when the distance between the sound source location and the sound masking location is less than a second threshold, the reminder device provides a second reminder message.

[0324] The volume of the target sound can refer to the volume of the target sound at any location within the cockpit. For example, the volume of the target sound can be the volume at the sound source location, or the volume of the target sound at the sound masking location. The first threshold can be preset. For example, the first threshold can be preset according to the performance of the sound shielding system, and the first threshold can be 60dB, 70dB, or 80dB, etc. This application does not limit this. Similarly, the second threshold can also be preset. For example, the second threshold is 30cm, 40cm, or 50cm, etc. This application does not limit this.

[0325] For example, the scenario corresponding to "the volume of the target sound is greater than the first threshold" could be that the user emitting the target sound is the first user, who is speaking loudly with other passengers or during a video call due to emotions or personal characteristics. As shown in Figure 15A, the first user is the user emitting the target sound, and the driver is the object whose target sound needs to be masked. If the first user's volume is within an appropriate range, the driver cannot hear / understand the first user's voice during the video call. The first user may speak too loudly during the video call, as shown in Figure 15B. In this case, the driver may hear / understand the first user's voice during the video call. Therefore, when the volume of the target sound is detected to be greater than the first threshold, the reminder device can determine the second reminder information. The way the reminder device reminds the second information can refer to the way the reminder device reminds the first information described above, and will not be repeated here. Optionally, when the volume of the target sound is greater than the first threshold, the second reminder information is used to remind the user that the volume of the target sound is too loud, and / or, the second reminder information is used to remind the user to lower the volume of the target sound. For example, the second reminder message is used to indicate that the first user's volume is too high, and / or the second reminder message is used to remind the first user to lower the volume of their speech.

[0326] For example, the scenario corresponding to "the distance between the sound source location and the target shielding location is less than a second threshold" is as follows: the user emitting the target sound is the first user (located in the second row), and the passenger in the target shielding location is the second user (located in the front row). The first user leans forward while speaking, or the second user leans back. As shown in Figure 16A, the first user is the user emitting the target sound, and the second user is the object whose target sound needs to be masked. When the distance between the first user and the second user is within a suitable range (the first distance shown in Figure 16A), the driver cannot hear / understand the first user's voice during the video call. The first user may lean forward during the video call, as shown in Figure 16B. In this case, the distance between the first user and the second user (the second distance shown in Figure 16B) may be too close, and the second user may hear / understand the first user's voice during the video call.

[0327] Optionally, if the distance between the sound source location and the sound masking location is less than a second threshold, the second reminder information indicates that the distance between the sound source location and the sound masking location is too close, and / or, the second reminder information is used to remind the user to increase the distance between the sound source location and the sound masking location. For example, the second reminder information is used to indicate that the distance between the first user and the second user is too close, and / or, the second reminder information is used to instruct the first user to increase the distance between the first user and the second user.

[0328] The alert device, by providing a secondary reminder, enables users to promptly identify the risk of potential information leakage, thereby preventing such leaks. This implementation not only informs users of existing risks but also allows them to use the sound masking system with greater peace of mind, thus improving its practicality.

[0329] In another possible implementation, after the reminder device reminds the user of the second reminder information, the reminder method shown in FIG11 above further includes: when the volume of the target sound is less than or equal to a first threshold, and / or when the distance between the sound source location and the sound masking location is greater than or equal to a second threshold, the reminder device determines the third reminder information.

[0330] For example, in conjunction with the scenarios shown in Figures 15A and 15B above, the alerting device will provide a second alert to the first user. After receiving the first alert, the first user may lower their speaking volume. If the first user's volume is less than or equal to a first threshold, the alerting device will also provide a third alert, which indicates that the target sound volume is within an appropriate range. The method by which the alerting device provides the third alert to the first user can be referred to the relevant description of the alerting device providing the first alert, and will not be repeated here. It is understandable that by providing the third alert, the alerting device can inform the user that their current volume is within an appropriate range, and also allow the user to use the sound masking system with greater confidence.

[0331] For example, in conjunction with the scenarios shown in Figures 16A and 16B above, the alerting device will provide a second alert to the first user. After the first user receives the first alert, the distance between the first user and the second user may increase. If the distance between the first user and the second user is greater than or equal to a second threshold, the alerting device will also provide a third alert to the first user. The third alert indicates that the distance between the sound source location and the sound masking location is within an appropriate range. The method by which the alerting device provides the third alert to the first user can be referred to the relevant description of the alerting device providing the first alert to the user, and will not be repeated here. It is understandable that by providing the third alert, the alerting device can, on the one hand, let the user know that their distance from the sound masking location is within an appropriate range, and on the other hand, allow the user to use the sound masking system with greater peace of mind.

[0332] In another possible implementation, the reminder method shown in Figure 11 further includes: the reminder device receiving a first instruction from the user. Based on the first instruction, the reminder device identifies at least one target object within the cabin, the sound emitted by the target object being the target sound. And / or, based on the first instruction, the reminder device identifies at least one masking object within the cabin, the masking object being the object whose target sound is to be masked. Both the target object and the masking object include the user within the cabin, and / or, the spatial area within the cabin. The first instruction is used to indicate at least one target object within the cabin, and / or at least one masking object within the cabin. For a detailed description of the implementation of this method, please refer to the description of step S1701 of the reminder method shown in Figure 17 below; it will not be detailed here.

[0333] In another possible implementation, the reminder method shown in FIG11 further includes: the reminder device displaying at least one target object determined based on a first instruction, and / or at least one masking object. For a detailed description of the implementation of this method, please refer to the description of step S1702 of the reminder method shown in FIG17 below, which will not be detailed here.

[0334] In another possible implementation, the reminder method shown in FIG11 further includes: the reminder device receiving a second instruction from the user. Based on the second instruction, the reminder device adjusts at least one target object, wherein the at least one target object before adjustment is different from the at least one target object after adjustment. And / or, based on the second instruction, the reminder device adjusts at least one masking object, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment. For a detailed description of the implementation of this embodiment, please refer to the description of step S3204 of the reminder method shown in FIG32 below, which will not be detailed here.

[0335] In another possible implementation, both the first and second instructions described above are user instructions, which include one or more of touch instructions, voice instructions, or gesture instructions. Touch instructions are, for example, instructions generated by a user through touching a human-computer interaction device; voice instructions are, for example, instructions generated by a user through voice control; and gesture instructions are, for example, instructions generated by a user through gesture control.

[0336] In another possible implementation, determining the sound source location and the sound masking location includes: when the target is a user inside the cabin, the alerting device determines the space area where the target's mouth is located as the sound source location; or, when the target is a space area inside the cabin, the alerting device determines part or all of the target's space area as the sound source location. And / or, when the masking target is a user inside the cabin, the alerting device determines the space area where the masking target's head is located as the sound masking location; or, when the masking target is a space area inside the cabin, the alerting device determines part or all of the masking target's space area as the sound masking location.

[0337] In another possible implementation, determining the spatial region where the head of the object being masked is located as the sound masking location includes: the alerting device determining the first spatial region where the head of the object being masked is located based on perception technology, and determining the first spatial region as the sound masking location. The perception technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0338] In another possible implementation, after determining the first spatial region as the sound masking location, the method further includes: if the head of the masked object moves to the second spatial region, the alerting device determines the second spatial region as the sound masking location, the first spatial region being different from the second spatial region.

[0339] In another possible implementation, determining the spatial region where the target object's mouth is located as the sound source location includes: the alerting device determining a third spatial region where the target object's mouth is located based on sensing technology, and the alerting device determining the third spatial region as the sound source location. The sensing technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0340] In another possible implementation, after determining the third spatial region as the sound source location, the method further includes: if the target's mouth moves to the fourth spatial region, the alerting device determines the fourth spatial region as the sound source location, which is different from the third spatial region.

[0341] In another possible implementation, the method further includes: the alerting device determining the sound masking location, and / or the alerting device determining the sound source location.

[0342] In another possible implementation, the method further includes: the alerting device sending a masking sound based on the target sound, the masking sound being used to mask the target sound at a sound masking location.

[0343] The reminder method shown in Figure 11 above can determine the masking result of the sound masking system on the target sound at the masking location, and can also remind the user of the masking result in various ways, so that the user can know the working status of the sound masking system during the use of the sound masking system, thereby increasing the user's confidence in using the sound masking system and thus improving the practicality of the sound masking system.

[0344] Please refer to Figure 17, which is a flowchart illustrating another reminder method provided in this application. It is understood that the steps in the embodiments of this application can be considered reasonable variations or supplements to the embodiments in Figure 11 above; or, it is understood that the reminder method in the embodiments of this application can also be considered an embodiment that can be executed independently, and this application does not limit this. The reminder method shown in Figure 17 may include one or more steps S1701 to S1702. For example, some solutions may only include step S1701 or step S1702. It should be understood that, for ease of description, the description is based on the order of steps S1701 to S1702, and is not intended to limit the execution to the above order. This application does not limit the order of execution, the execution time, or the number of executions of the above one or more steps. Steps S1701 to S1702 are as follows:

[0345] S1701, The reminder device receives the first instruction from the user.

[0346] The first instruction is used to instruct at least one target object within the cockpit, the sound emitted by the target object being the target sound. And / or, the first instruction is used to instruct at least one masking object within the cockpit, the masking object being the object whose target sound is to be masked. Both the target object and the masking object include a user within the cockpit or a spatial area within the cockpit. The first instruction can be one or more of a touch instruction, a voice instruction, or a gesture instruction. Next, the reception of the first instruction from the user by the alerting device will be described in detail according to the type of the first instruction.

[0347] Case 1: The first instruction is a touch instruction.

[0348] In one possible implementation, the alerting device can acquire a first command from the user via a touch sensor, such as a touchscreen in the cockpit or a physical key. The following example, using a touchscreen in the cockpit as the touch sensor, will illustrate how the alerting device receives the first command from the user.

[0349] For example, in the cockpit shown in Figure 7 above, the alerting device can obtain a first command from the first user through screen A, screen B, or screen C within the cockpit. The cockpit shown in Figure 7 is exemplary; the cockpit may include more or fewer screens, for example, screen A may be included within the cockpit.

[0350] For example, a user can issue the first command by clicking on a control on the touchscreen.

[0351] For example, the first instruction is used to designate at least one target object within the cockpit. Referring to Figure 18A, the user can tap the "Please Select Target Object" control on the touchscreen to enter the target object selection process. The user selects "Right Rear Seat" and "Left Rear Seat" as target objects on the touchscreen, as shown in Figure 18B. The user taps the control corresponding to "Right Rear Seat," and a new "Right Rear Seat" control is added below the "Please Select Target Object" control, while the original "Right Rear Seat" control becomes unselectable. The user can directly tap the "OK" control in the middle of the screen to indicate that the selection is complete. The user can continue to select new target objects, as shown in Figure 18C. The user taps the control corresponding to "Left Rear Seat," and a new "Left Rear Seat" control is added below the "Please Select Target Object" control, while the original "Left Rear Seat" control becomes unselectable. After the user completes the selection, they can tap the "OK" control, as shown in Figure 18D. It should be noted that after the user taps the "OK" control, the relevant processing unit / program will determine the user's selection as the first instruction and send it to the notification device.

[0352] Optionally, if the user selects the wrong target object, the user can also deselect it by clicking the wrong target object below the "Please select target object" control before clicking the "Confirm" control.

[0353] For example, the first command is used to instruct at least one shielded object within the cockpit. Specific operational methods can be found in one or more of the figures shown in Figures 18A to 18D above, and will not be repeated here.

[0354] For example, the first instruction is used to designate at least one target object and at least one shielding object within the cockpit. Exemplarily, after the user completes the selection of a target object (as shown in Figure 18C), they can click the "Please select a shielding object" control and enter the process of selecting a shielding object, as shown in Figure 18E. The user selects the "driver's seat" on the touchscreen as the shielding object, as shown in Figure 18F. The user clicks the control corresponding to the "driver's seat," and a new "driver's seat" control is added below the "Please select a target object" control, while the original "driver's seat" control becomes unselectable. After the user completes the selection, they can click the "OK" control, as shown in Figure 18G. It should be noted that after the user clicks the "OK" control, the relevant processing unit / program will determine the user's selection as the first instruction and send it to the alerting device.

[0355] For example, more controls can be added to the screen to provide more convenient functions.

[0356] The interface shown in Figure 19 includes a "one-click masking" control. The "one-click masking" control is used to set the target user in the cockpit as the "target object" and set all objects in the cockpit other than the target user as "masked objects". In this case, the sound masking system masks the voice of the target user so that other users in the cockpit cannot hear / understand the voice of the target user. This is used for the target user to participate in telephone conferences or video calls, or for conversations between multiple target users.

[0357] Understandably, activating the sound masking system by sending touch commands offers high privacy, convenience, and minimizes disturbance to other users in the cockpit. Alternatively, the "one-click masking" control shown in Figure 19 can be configured as a physical button for even faster access to the corresponding sound masking mode.

[0358] Scenario 2: The first instruction is a voice instruction.

[0359] For example, a user can select a target object and / or a masking object according to preset language rules. For instance, a user can set the target object and / or the masking object by saying, "I want to mask, the target object is XXX, the masking object is XXX". Example: A user saying, "I want to mask, the target object is the right rear seat, the masking object is the driver's seat," means setting the target object to "the right rear seat" and the masking object to "the driver's seat". As another example, a user can set the target object by saying, "I want to mask, the target object is XXX," and set the masking object to themselves. Example: A user saying, "I want to mask, the target object is the driver's seat," means setting the target object to "the driver's seat" and the masking object to themselves. For the sake of clarity, no further examples are provided here.

[0360] For example, the user's voice output does not directly indicate the target object and / or the masked object. Instead, it is associated with the controlled object through conjunctions. Furthermore, determining the specific implementation of the controlled object based on conjunction association can be done by pre-defining the association between words, or by obtaining the user's semantically associated controlled object through semantic analysis (e.g., through artificial intelligence analysis). For instance, when the target user utters the voice "I want to make a phone call," the reminder device may need to mask the target user's voice through semantic analysis. In this case, the target object is the target user, and the masked object is all other objects in the cabin besides the target user, as shown in Figure 20.

[0361] It is understandable that more forms or methods of disseminating voice messages can be set, and this application does not limit this.

[0362] Scenario 3: The first instruction is a gesture instruction.

[0363] Gesture commands can include air gestures and gestures applied to a specific device. Gestures applied to a specific device include swiping or pressing gestures on the display screen, swiping or pinching gestures on the steering wheel, or swiping or pinching gestures on the seat armrest. Gesture commands can be obtained by processing data collected by the in-cabin sensor device 520.

[0364] For example, as shown in Figure 21, passenger A can first point in the direction of the driver and display the gesture shown in Figure 22A, indicating that the target object is "passenger A" and the masking object is "driver". Furthermore, if passenger A wants to end the masking of "driver", they can point in the direction of the driver again (as shown in Figure 21) and display the gesture shown in Figure 22B. The gesture shown in Figure 22A is a pinching gesture, and the gesture shown in Figure 22B is an opening gesture, representing the process of two fingers moving from touching to separating.

[0365] S1702, the alerting device responds to the first instruction by displaying at least one target object, and / or displaying at least one concealed object.

[0366] It should be noted that the reminder methods shown in Figure 17 all involve the reminder device displaying relevant reminder information or instructions to the user. In some implementations, the reminder device can also remind the user of relevant reminder information or instructions through text, sound, light, haptic stimulation, or linked devices. For details, please refer to the relevant description of S1103 above, which will not be repeated here.

[0367] For example, the alerting device may display at least one target object to the user through a display-enabled device in the cockpit, and / or display at least one concealed object.

[0368] For example, as shown in Figure 18D above, if the first instruction is used to set the target objects as "right rear seat" and "left rear seat", the reminder device can display the content shown in Figure 23A on the screen, that is, the target objects include the rear seats of the vehicle.

[0369] For example, as shown in Figure 18G above, the first instruction is used to set the target objects as "right rear seat" and "left rear seat", and to set the masking objects as "driver's seat" and "passenger's seat". Then the reminder device can display the content shown in Figure 23B on the screen, that is, the target objects include the rear seats of the vehicle, and the masking objects include the front seats.

[0370] For example, when a user clicks the "One-Click Masking" control as shown in Figure 19, the alert device can display the content shown in Figure 23C on the screen. The target objects include the driver's seat, the front passenger seat, and the left rear seat, while the masked object includes the right rear seat. Optionally, the user can choose to mask the driver, the front passenger, or the passenger in the left rear seat.

[0371] For example, if the user says "I want to make a phone call" (as shown in Figure 20), the reminder device can display the content shown in Figure 23D on the screen.

[0372] For example, if the user performs the gestures shown in Figures 21 and 22A in sequence, the reminder device can display the content shown in Figure 24 on the screen, that is, the target object includes the right rear seat and the obscured object includes the driver's seat.

[0373] Understandably, the alerting device, in response to the first instruction, displays at least one target object and / or at least one masked object. This allows the user to clearly identify the currently selected target object and / or masked object, thereby increasing the user's confidence in using the sound masking system and thus enhancing its practicality. Furthermore, the alerting device's message also helps the user confirm whether their selection of the target object and / or masked object is correct, allowing them to reselect if an incorrect selection is made, further improving the system's usability.

[0374] In one possible implementation, the reminder method shown in FIG17 further includes: the reminder device receiving a second instruction from a user, the second instruction indicating adjustment of at least one target object, wherein the at least one target object before adjustment is different from the at least one target object after adjustment. And / or, the second instruction indicating adjustment of at least one masking object, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment.

[0375] The description of the second instruction can be found in the description of the first instruction above, and will not be repeated here. For example, the second instruction may include one or more of the following: touch instruction, voice instruction, or gesture instruction. For the sake of brevity, the second instruction will be described exemplarily using touch instruction as an example.

[0376] For example, a user can clear the current configuration of the target object and / or the masking object by clicking the "×" control shown in Figure 25A. The cleared interface is shown in Figure 25B. The user can reselect at least one target object and / or at least one masking object by following the steps shown in Figures 18A to 18G. The selected interface is shown in Figure 25C. The interface shown in Figure 25C indicates that the target objects selected by the user include the right rear seat and the front passenger seat, and the masking objects include the driver's seat and the left rear seat. Further, the user can click the "√" control shown in Figure 25C to complete the selection.

[0377] Optionally, in response to the second instruction, the notification device displays at least one adjusted target object, and / or displays at least one adjusted masked object. For example, if the second instruction is the instruction corresponding to the "√" control shown in Figure 25C, the display device can display the interface shown in Figure 26.

[0378] The alerting device, in response to the second instruction, displays at least one adjusted target object and / or at least one masked object, enabling the user to clearly understand whether the adjustment is complete or whether the adjustment was correct. When the user is informed of the adjusted target object and / or at least one masked object through the alerting device, it helps to increase the user's confidence in using the sound masking system, thereby improving the practicality of the sound masking system.

[0379] In another possible implementation, the reminder method shown in Figure 17 further includes: the reminder device displaying volume information of the target sound. Alternatively, if the volume of the target sound is less than or equal to a first threshold, the reminder device displays information indicating that the volume of the target sound is within a suitable range; or, if the volume of the target sound is greater than the first threshold, the reminder device displays information indicating that the volume of the target sound is too high. The information indicating that the volume of the target sound is too high includes information reminding the user to lower the volume of the target sound.

[0380] For example, when there are multiple target objects, the alerting device can obtain the volume information of the target sound through the sound pickup device in the cockpit, and display the volume information of the target sound to the target objects through the screen in the cockpit.

[0381] For example, when passenger A, as shown in Figure 27, is the target, the reminder device can display passenger A's volume on screen A in front of passenger A while passenger A is speaking. Screen A can display content as shown in Figure 28A or Figure 28B. The content shown in Figure 28A includes the words "Your volume is within the appropriate range." and the content shown in Figure 28B includes the words "Your volume is too loud, please speak softly!"

[0382] For example, when there are multiple target objects, the alerting device acquires the volume information of each target object through the sound pickup device in the cockpit and displays the volume information of each target object to the target objects through the screen in the cockpit.

[0383] For example, when passengers A and B, as shown in Figure 27, are the target subjects, the reminder device can display the volume of passengers A and B on screens A and B while they are speaking. The content displayed on screens A and B can be as shown in Figure 29, which includes the message: "Passenger A's volume is within a suitable range. Passenger B's volume is too loud, please speak softly!"

[0384] In another possible implementation, the at least one target object includes a first object, the location from which the first object emits sound is the sound source location, and the at least one masking object includes a second object, the location from which the second object receives sound is the sound masking location. The alerting method shown in Figure 17 further includes: the alerting device displaying information about the distance between the sound source location and the sound masking location. And / or, if the distance between the sound source location and the sound masking location is greater than or equal to a second threshold, the alerting device displays information indicating that the distance between the sound source location and the sound masking location is within an appropriate range; or, if the distance between the sound source location and the sound masking location is less than the second threshold, the alerting device displays information indicating that the distance between the sound source location and the sound masking location is too small. The information indicating that the distance between the sound source location and the sound masking location is too small includes information reminding the user to increase the distance between the sound source location and the sound masking location.

[0385] The first object is, for example, the first user shown in Figure 16A above, and the second object is, for example, the second user shown in Figure 16A above. For a description of the above implementation, please refer to the relevant descriptions in Figures 16A and 16B above, which will not be repeated here. Regarding how the reminder device displays "spacing information between the sound source location and the sound masking location," "information that the spacing between the sound source location and the sound masking location is within a suitable range," or "information that the spacing between the sound source location and the sound masking location is too small," please refer to the relevant description of the reminder device reminding the first reminder information in S1103 above, which will not be repeated here.

[0386] In another possible implementation, when the second object is a user inside the cabin, the alert method shown in Figure 17 further includes: displaying the spatial area where the second object's head is located after the head has moved, in the event that the second object's head has moved. Here, "the second object's head moving" includes the head moving a range greater than a second range.

[0387] It is understandable that the spatial area where the head of the second object is located is the sound masking position. When the head of the second object moves, the sound masking position needs to move synchronously to effectively mask the second object. However, if the movement range of the head of the second object exceeds a certain second range, users, including those in the target audience, may worry whether the sound masking system can still effectively mask the second object. In this case, displaying the spatial area where the head of the second object is located after movement helps users, including those in the target audience, to use the sound masking system with confidence, thereby improving the practicality of the sound masking system. The setting of the second range is not limited in this application; for example, the second range can be a radius of 10cm or 15cm centered on the sound masking position.

[0388] For example, in Figure 30A, the driver is the second object, and the space where the second object's head is located is the first space region. The movement of the second object's head is shown in Figure 30B, where the space where the second object's head is located is the second space region. When the alerting device detects that the second object's head has moved from the first space region to the second space region, it can display the new space region where the second object's head is located on the in-cabin display screen. As shown in Figure 31A, the alerting device can display text such as "The passenger in the driver's seat has moved 20cm to the right" to inform the user that the movement of the second object has been detected. Alternatively, as shown in Figure 31B, the alerting device can display text such as "Please do not lean forward, maintain a normal sitting posture" to remind the passenger in the driver's seat to maintain a normal sitting posture.

[0389] Optionally, the alerting device may determine the spatial region where the head of the second object is located using one or more of machine vision technology, positioning technology, or sensor technology.

[0390] Optionally, the alerting device may also identify the second spatial area as the sound masking location, and the alerting device may also control the sound masking system to send masking sounds to the second spatial area to mask the target sound.

[0391] Optionally, the reminder device may also display the location of the sound masking. For details on the display method, please refer to the relevant description of the reminder device displaying the first reminder information in S1103 above.

[0392] Optionally, the alerting device may also simultaneously display one or more of the following: the target object, the masked object, the first source location, and the sound masking location. When one or more of these changes, the alerting device will also display the changed content. The method by which the alerting device displays the above content can also refer to the relevant description of the alerting device displaying the first alert information in S1103 above.

[0393] Optionally, when the alerting device displays the first source location and / or the sound masking location, the first source location and / or the sound masking location may be obtained from the sound masking system.

[0394] Optionally, the alerting device can also obtain the sound masking position before and after the second object's head moves from the sound masking system to determine whether the sound masking system updates the sound masking position in a timely manner.

[0395] It should be noted that the "user" mentioned above can be any driver in the cockpit, and this application does not limit this.

[0396] In one possible implementation, when the object being masked moves away from the masking sound generating device, the object can continue to be masked by increasing the volume of the masking sound or adjusting its phase. As shown in Figure 30A, the masking sound generating device is located inside the headrest of the second object. When the head of the second object moves (as shown in Figure 30B), the masking sound generating device can continue to mask the second object by increasing the volume of the masking sound or adjusting its phase.

[0397] Please refer to Figure 32, which is a flowchart illustrating another reminder method provided in this application. It is understood that the steps in the embodiments of this application can be considered reasonable variations or supplements to the embodiments in Figure 11 or Figure 17 above; or, it is understood that the information processing method in the embodiments of this application can also be considered an embodiment that can be executed independently, and this application does not limit this. The information processing method shown in Figure 32 may include one or more steps S3201 to S3213. For example, some solutions may only include steps S3201 and S3213. It should be understood that, for ease of description, the description is based on the order of steps S3201 to S3213, and is not intended to limit the execution to the above order. This application does not limit the order of execution, the execution time, or the number of executions of the above one or more steps. Steps S3201 to S3213 are as follows:

[0398] S3201. The user activates the sound masking system. For example, the user activates the sound masking system through touch commands, voice commands, or gesture commands, etc. The specific implementation is not limited in this application.

[0399] S3202, The user sends a first command to the alerting device, and the alerting device receives the first command from the user accordingly. Please refer to the description of step S1701. Optionally, the user sends a first command to the sound masking system, and the sound masking system receives the first command from the user accordingly.

[0400] S3203, The alerting device identifies the target object and / or, masks the object. Please refer to the description of step S1702.

[0401] S3204. The user sends a second command to the alerting device, and the alerting device receives the second command from the user accordingly. Please refer to the relevant description of the second command in step S1702. See Figures 25A to 25C for details. Optionally, the user sends a second command to the sound masking system, and the sound masking system receives the second command from the user accordingly.

[0402] S3205, The reminder device determines the modified target object and / or the masked object. Please refer to the relevant description of the second instruction in step S1702. See Figure 26 for details.

[0403] S3206. The reminder device determines the first reminder information. Please refer to the description of step S1103.

[0404] S3207. The user lowers the volume or adjusts their posture. If the first warning message indicates that the target sound volume is too loud, or that the distance between the first source location and the sound masking location is too close, the user can perform the steps shown in S3207.

[0405] S3208. The reminder device determines that the volume is appropriate and / or the sitting posture is appropriate. When the reminder device detects that the user's volume is within an appropriate range and / or the first source position and the sound masking position are within an appropriate range, the reminder device determines the content shown in S3208, as specifically shown in Figure 28A.

[0406] S3209. The user checks the masking speaker. If the first warning message indicates that the masking volume is too low, the user can perform the steps shown in S3209.

[0407] S3210, The alerting device determines that the masking sound level is appropriate. When the alerting device detects that the masking sound level is within an appropriate range, the alerting device determines the content shown in S3210, as shown in Figure 28A.

[0408] S3211. The reminder device determines the second reminder information. Refer to the description of the second reminder information in step S1103, as shown in Figures 14A to 14B.

[0409] S3212, The reminder device determines the third reminder information. Refer to the description of the third reminder information in step S1103.

[0410] S3213. The user disables the sound masking system. For example, the user disables the sound masking system through touch commands, voice commands, or gesture commands. The specific implementation is not limited in this application.

[0411] In summary, the reminder method provided in this application, by determining the masking result of the sound masking system on the target sound at the masking location and displaying this result to the user, can enhance the user's confidence in using the sound masking system, thereby improving the practicality of the sound masking system. Furthermore, the aforementioned reminder method can also display the target object, the masked object, the first source location, the sound masking location, and their updated content in real time, allowing the user to clearly understand the working status of the sound masking system, thus increasing the user's confidence in using the sound masking system. Further, the aforementioned reminder method can also send relevant reminders to the user in cases of improper user operation (e.g., excessive volume, obstruction of the headrest), enabling the user to operate correctly based on the reminders, thereby avoiding information leakage and further increasing the user's confidence in using the sound masking system. In conclusion, the reminder method provided in this application offers multiple ways to receive user instructions and multiple ways to send reminder information to the user, enabling timely and effective feedback to the user on various possible situations, thereby enhancing the user's confidence in using the sound masking system.

[0412] The foregoing has described the application scenarios and methods provided by the embodiments of this application. The apparatus of the embodiments of this application is provided below. It is understood that the various apparatuses provided in the embodiments of this application, such as interactive devices, computing devices, chips, etc., include hardware structures, software units, or combinations of hardware and software structures to perform the functions described in the above method embodiments. Those skilled in the art should readily recognize that the apparatus and modules within it can be implemented in hardware or a combination of hardware and computer software in conjunction with the various functions described in the embodiments disclosed herein. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different apparatus implementations in different application scenarios to implement the aforementioned method embodiments, and different implementations of the apparatus should not be considered beyond the scope of the embodiments of this application.

[0413] Several possible devices are listed below.

[0414] Please refer to Figure 33, which is a schematic diagram of the structure of a reminder device provided in an embodiment of this application. The reminder device 330 may include a processing unit 3301 and an interaction unit 3302. The reminder device 330 may be an independent device, such as a processor, computing device, or vehicle. Alternatively, the reminder device 330 may also be a software module and / or hardware module in an independent device. The reminder device 330 is used to implement the aforementioned reminder method, such as the reminder method in the embodiments shown in Figures 11, 17, or 32. The processing unit 3301 is used to implement one or more of the aforementioned data processing, instruction execution, and other related operations such as determining, adjusting, generating, deciding, and judging. The interaction unit 3302 is used to implement one or more of the following operations: displaying, sending, receiving, or deleting.

[0415] In one possible design, the reminder device 330 is used to implement the reminder method shown in FIG11 or FIG32.

[0416] In one possible implementation, if the volume of the target sound is greater than a first threshold, and / or if the distance between the sound source location and the sound masking location is less than a second threshold, the interaction unit 3302 is used to determine the second reminder information.

[0417] In another possible implementation, if the volume of the target sound exceeds a first threshold, the second reminder information is used to alert the user that the target sound volume is too high, and / or, the second reminder information is used to remind the user to lower the volume of the target sound. And / or, if the distance between the sound source location and the sound masking location is less than a second threshold, the second reminder information indicates that the distance between the sound source location and the sound masking location is too close, and / or, the second reminder information is used to remind the user to increase the distance between the sound source location and the sound masking location.

[0418] In another possible implementation, the interaction unit 3302 is further configured to determine a third reminder message when the volume of the target sound is less than or equal to a first threshold, and / or when the distance between the sound source location and the sound masking location is greater than or equal to a second threshold.

[0419] In another possible implementation, the interaction unit 3302 is further configured to receive a first instruction from the user. The processing unit 3301 is configured to, based on the first instruction, determine at least one target object within the cockpit, wherein the sound emitted by the target object is the target sound. And / or, the processing unit 3301 is configured to, based on the first instruction, determine at least one masking object within the cockpit, wherein the masking object is the object whose target sound is to be masked. Both the target object and the masking object include the user within the cockpit, and / or, a spatial area within the cockpit. The first instruction is used to indicate at least one target object within the cockpit, and / or at least one masking object within the cockpit.

[0420] In another possible implementation, the interaction unit 3302 is also used to display at least one target object determined based on the first instruction, and / or at least one masking object.

[0421] In another possible implementation, the interaction unit 3302 is further configured to receive a second instruction from the user. The processing unit 3301 is configured to adjust at least one target object based on the second instruction, wherein the at least one target object before adjustment is different from the at least one target object after adjustment. And / or, the processing unit 3301 is configured to adjust at least one masking object based on the second instruction, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment.

[0422] In another possible implementation, both the first and second instructions described above are user instructions, which include one or more of touch instructions, voice instructions, or gesture instructions. Touch instructions are, for example, instructions generated by a user through touching a human-computer interaction device; voice instructions are, for example, instructions generated by a user through voice control; and gesture instructions are, for example, instructions generated by a user through gesture control.

[0423] In another possible implementation, when the target is a user inside the cabin, the processing unit 3301 is specifically configured to determine the spatial area where the target's mouth is located as the sound source location; or, when the target is a spatial area inside the cabin, the processing unit 3301 is specifically configured to determine part or all of the spatial area of ​​the target as the sound source location. And / or, when the masking target is a user inside the cabin, the processing unit 3301 is specifically configured to determine the spatial area where the masking target's head is located as the sound masking location; or, when the masking target is a spatial area inside the cabin, the processing unit 3301 is specifically configured to determine part or all of the spatial area of ​​the masking target as the sound masking location.

[0424] In another possible implementation, the processing unit 3301 is specifically used to determine the first spatial region where the head of the object to be masked is located based on perception technology, and to determine the first spatial region as the sound masking location. The perception technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0425] In another possible implementation, when the head of the object being masked moves to the second spatial region, the processing unit 3301 is specifically used to determine the second spatial region as the sound masking location, which is different from the first spatial region.

[0426] In another possible implementation, the third spatial region where the mouth of the target object is located is determined based on perception technology, and the processing unit 3301 is specifically used to determine the third spatial region as the location of the sound source. The perception technology includes one or more of machine vision technology, positioning technology, or sensor technology.

[0427] In another possible implementation, when the mouth of the target object moves to the fourth spatial region, the processing unit 3301 is specifically used to determine the fourth spatial region as the sound source location, which is different from the third spatial region.

[0428] In another possible implementation, the interaction unit 3302 is also used to determine the sound masking location and / or the sound source location.

[0429] In another possible implementation, the interactive device further includes a masking unit for transmitting a masking sound based on the target sound, the masking sound being used to mask the target sound at a sound masking location.

[0430] In another possible implementation, the cockpit includes N sound pickup devices, where N is a positive integer. The processing unit 3301 is specifically used to pick up sound observation signals through the N sound pickup devices. The sound observation signals include M sub-sound observation signals picked up by M of the N sound pickup devices, where M is a positive integer less than or equal to N. The sound pickup device is a device that has the function of converting sound (sound waves) into electrical signals; for example, the sound pickup device is a microphone or a pickup unit.

[0431] In another possible implementation, one or more of the M sound pickup devices are located within a first range of the sound masking location.

[0432] In another possible implementation, M=1, and the M sound pickup devices are the first sound pickup device, which is the sound pickup device closest to the sound masking position among the N sound pickup devices.

[0433] In another possible implementation, processing unit 3301 is specifically used to determine a first signal based on the sound source location, the sound masking location, and M sub-sound observation signals. The first signal describes the signal related to the target sound at the sound masking location. Processing unit 3301 is specifically used to determine a masking result based on the first signal. The first signal describes the signal related to the target sound at the sound masking location; for example, it describes the signal strength of the target sound at the sound masking location. It is understood that the first signal is calculated, not actually measured; therefore, there may be errors between the first signal and the actually measured signal strength of the target sound.

[0434] In another possible implementation, the cockpit includes a masking sound generating device for playing a masking sound to mask target sound at a sound masking location. The processing unit 3301 is specifically configured to determine a second signal describing a signal related to the masking sound at the sound masking location. The processing unit 3301 is further configured to determine a masking result based on the first and second signals.

[0435] In another possible implementation, the processing unit 3301 is specifically configured to base its processing on the acoustic observation signal S. o Masking acoustic signal S m First transfer function H mo The second transfer function H sd and the third transfer function H so Determine the first signal S sd Sound observation signal S o Masking acoustic signal S m First transfer function H mo The second transfer function H sd The third transfer function H so and the first signal S sd S satisfies the following relationship: sd =(S o -S m *H mo )H sd / H so Among them, the masking acoustic signal S m Used to generate masking sound, the first transfer function H mo The second transfer function H is used to represent the transfer function between the masking sound generation device and the sound pickup device. sd The third transfer function H is used to represent the transfer function between the sound source location and the sound masking location. so This is used to represent the transfer function between the location of the sound source and the sound pickup device.

[0436] In another possible implementation, the processing unit 3301 is specifically configured such that the ratio R of the signal strength of the second signal to the signal strength of the first signal is less than R0. L In the case of R, the masking result is determined to be in a failed state. Alternatively, if the ratio R is greater than R... H In the case where the ratio R is greater than or equal to R, the masking result is determined to be valid. L And the ratio R is less than or equal to R H In this case, the masking result is determined to be a warning state. Among them, R L ≤R H .

[0437] In another possible implementation, the aforementioned ratio R is, for example, the first signal S. sd With the second signal S md The short-time energy ratio between them. For example, R, the first signal S sd With the second signal S md The following relationship exists between them:

[0438] In another possible implementation, R L The value range of R includes [5dB, 18dB]. H The value range includes [10dB, 30dB].

[0439] In another possible implementation, the cockpit includes a sound masking system for masking target sounds at a masking location. The processing unit 3301 is specifically configured to: determine that the masking result is invalid if the signal strength of the first signal is greater than or equal to a third threshold; and / or determine that the masking result is valid if the signal strength of the first signal is less than or equal to a fourth threshold; and / or determine that the masking result is a warning if the signal strength of the first signal is between the third and fourth thresholds; and / or determine that the masking result is invalid if the signal strength of the second signal is less than or equal to a fifth threshold; and / or determine that the masking result is valid if the signal strength of the second signal is greater than or equal to a sixth threshold; and / or determine that the masking result is a warning if the signal strength of the second signal is between the fifth and sixth thresholds.

[0440] In another possible implementation, the aforementioned first reminder information is used to remind the user of the masking result.

[0441] The specific operations performed by the aforementioned reminder device 330 can also be found in the embodiments shown in Figure 11 or Figure 32.

[0442] In another possible design, the reminder device 330 is used to implement the reminder method shown in FIG17 or FIG32. The reminder device 330 also includes a receiving unit for receiving instructions from the user, including a first instruction, a second instruction, etc. The processing unit 3301 can be used to process the user instructions received by the receiving unit.

[0443] In one possible implementation, the receiving unit is configured to receive a second instruction from a user, the second instruction indicating adjustment of at least one target object, wherein the at least one target object before adjustment is different from the at least one target object after adjustment. And / or, the second instruction indicates adjustment of at least one masking object, wherein the at least one masking object before adjustment is different from the at least one masking object after adjustment. The interaction unit 3302 is configured to, in response to the second instruction, display the at least one target object after adjustment, and / or display the at least one masking object after adjustment.

[0444] In another possible implementation, the first instruction is a user instruction, which includes one or more of touch instructions, voice instructions, or gesture instructions.

[0445] In another possible implementation, the interaction unit 3302 is also used to display volume information of the target sound. Alternatively, if the volume of the target sound is less than or equal to a first threshold, it displays information indicating that the volume of the target sound is within a suitable range; or if the volume of the target sound is greater than the first threshold, it displays information indicating that the volume of the target sound is too high. The information indicating that the volume of the target sound is too high includes information reminding the user to lower the volume of the target sound.

[0446] In another possible implementation, the at least one target object includes a first object, the location where the first object emits sound is the sound source location, and the at least one masking object includes a second object, the location where the second object receives sound is the sound masking location. The interaction unit 3302 is also configured to display information about the distance between the sound source location and the sound masking location. And / or, if the distance between the sound source location and the sound masking location is greater than or equal to a second threshold, information indicating that the distance between the sound source location and the sound masking location is within a suitable range is displayed; or, if the distance between the sound source location and the sound masking location is less than the second threshold, information indicating that the distance between the sound source location and the sound masking location is too small is displayed. The information indicating that the distance between the sound source location and the sound masking location is too small includes information reminding the user to increase the distance between the sound source location and the sound masking location.

[0447] In another possible implementation, when the second object is a user inside the cabin, the interaction unit 3302 is further configured to display the spatial area where the head is located after the head movement, in the event that the head of the second object moves. Here, "movement of the head of the second object" includes the movement of the head within a range greater than a second range.

[0448] In another possible implementation, the interaction unit 3302 is also used to display the masking result of masking the target sound for the masking object, the masking result including one or more of the following: masking success, masking failure, or masking warning.

[0449] The specific operations performed by the aforementioned reminder device 330 can also be found in the embodiments shown in Figure 17 or Figure 32.

[0450] Figure 34 is a schematic block diagram of another reminder device provided in an embodiment of this application. The reminder device 3400 shown in Figure 34 can be a hardware circuit implementation of the device shown in Figure 33. The reminder device 3400 is used to implement the reminder method shown in Figures 11, 17, or 32. For ease of explanation, Figure 34 only shows the main components of the reminder device.

[0451] The reminder device 3400 shown in Figure 34 includes at least one processor 3401. The reminder device 3400 may also include at least one memory 3402 for storing program instructions and / or data. The memory 3402 and the processor 3401 are coupled. The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. The processor 3401 can operate collaboratively with the memory 3402, and the processor 3401 can execute program instructions stored in the memory 3402. At least one of the at least one memory 3402 may be included in the processor 3401.

[0452] The reminder device 3400 may further include a communication interface 3403 for communicating with other devices via a transmission medium, thereby enabling the reminder device 3400 to communicate with other devices. In this embodiment, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface. In this embodiment, when the communication interface is a transceiver, the transceiver may include an independent receiver, an independent transmitter, or a transceiver with integrated transceiver functions, or an interface circuit.

[0453] It should be understood that the connection medium between the processor 3401, memory 3402, and communication interface 3403 is not limited in the embodiments of this application. In the embodiments of this application, the memory 3402, processor 3401, and communication interface 3403 are connected by a communication bus 3404 in FIG34. The bus is represented by a thick line in FIG34. The connection methods between other components are only illustrative and are not intended to be limiting. The bus may include an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in FIG34, but this does not mean that there is only one bus or one type of bus. For the operations performed by the processor 3401, please refer to the operations performed by the processing unit 3301 in FIG33, which will not be repeated here. For the operations performed by the communication interface 3403, please refer to the operations performed by the interaction unit 3302 in FIG33, which will not be repeated here.

[0454] A chip includes: a processor coupled to a memory for storing programs or instructions, wherein when the program or instructions are executed by the processor, a method is provided to cause the device to perform the flow shown in any one of Figures 11, 17 or 32 above.

[0455] In this application embodiment, the processor can be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in this application embodiment. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0456] In the embodiments of this application, the memory can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions, used to store program instructions and / or data.

[0457] This application also provides a chip, including a processor and a communication interface. The communication interface is used for outputting and / or outputting data (including instructions), and / or for receiving and / or sending data. When the processor executes program instructions in memory, the aforementioned interaction methods, such as those shown in the embodiments of FIG11, FIG17, or FIG32, are used.

[0458] As one possible example, the communication interface is used to input voice and gesture information, and the processor is used to determine the controlled object based on the voice information and adjust the state of the controlled object based on the gesture information. Optionally, the communication interface is also used to output at least some information related to various prompts.

[0459] This application provides a computer-readable storage medium storing instructions that, when executed by at least one processor, implement the aforementioned reminder method, such as the reminder method in the embodiments shown in FIG11, FIG17, or FIG32.

[0460] The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that contains one or more available media. Computer-readable storage media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0461] This application provides a computer program product including computer instructions that, when executed on at least one processor, implement the aforementioned reminder method, such as the reminder method in the embodiments shown in FIG11, FIG17, or FIG32.

[0462] Optionally, the computer program product can be a software installation package or an image package. If the aforementioned method is required, the computer program product can be downloaded and executed on a computing device.

[0463] This application provides a vehicle that includes the aforementioned reminder device 330, or the vehicle includes the aforementioned chip, or the vehicle includes the aforementioned computer storage medium, or the vehicle is equipped with the aforementioned computer program product.

[0464] In this application, the terms "exemplarily" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplarily" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0465] In this application, "at least one" in the embodiments refers to one or more items, and "more than one" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, and c can be single or multiple. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0466] Furthermore, unless otherwise stated, the use of ordinal numbers such as "first" and "second" in the embodiments of this application is for distinguishing multiple objects and is not for limiting the order, sequence, priority or importance of multiple objects.

Claims

1. A reminding method characterized by, The method comprises: determining a sound source position, a sound masking position, and a sound observation signal, the sound source position being a position from which a target sound is emitted, and the sound masking position being a position at which the target sound needs to be masked; based on the sound source position, the sound masking position, and the sound observation signal, determining a masking result; based on the masking result, determining first reminding information.

2. The method of claim 1, wherein, The method further comprises: in a case where a volume of the target sound is greater than a first threshold value, and / or in a case where a distance between the sound source position and the sound masking position is less than a second threshold value, prompting second reminding information to a user.

3. The method of claim 2, wherein, in the case where the volume of the target sound is greater than the first threshold value, the second reminding information is used to remind that the volume of the target sound is too large, and / or the second reminding information is used to remind the user to reduce the volume of the target sound; and / or in the case where the distance between the sound source position and the sound masking position is less than the second threshold value, the second reminding information indicates that the distance between the sound source position and the sound masking position is too close, and / or the second reminding information is used to remind the user to increase the distance between the sound source position and the sound masking position. After the prompting of the second reminding information to the user, the method further comprises:

4. The method according to claim 2 or 3, characterized in that, in a case where the volume is less than or equal to the first threshold value, and in a case where the distance between the sound source position and the sound masking position is greater than or equal to the second threshold value, determining third reminding information.

5. The method of claim 4, wherein, in the case where the volume is less than or equal to the first threshold value, the third reminding information is used to indicate that the volume of the target sound is in a suitable range; and in the case where the distance between the sound source position and the sound masking position is greater than or equal to the second threshold value, the third reminding information is used to indicate that the distance between the sound source position and the sound masking position is in a suitable range. The method further comprises:

6. The method according to any one of claims 1 to 5, characterized in that, receiving a first instruction from a user; based on the first instruction, determining at least one target object in the cabin, the target object emitting the target sound; and / or, based on the first instruction, determining at least one masking object in the cabin, the masking object being an object to be masked from the target sound; the target object and the masking object both including a user in the cabin and / or a space region in the cabin. The method further comprises:

7. The method of claim 6, wherein, receiving a second instruction from a user; based on the second instruction, adjusting the at least one target object, the at least one target object before adjustment being different from the at least one target object after adjustment; and / or, based on the second instruction, adjusting the at least one masking object, the at least one masking object before adjustment being different from the at least one masking object after adjustment. The determination of the sound source position and the sound masking position comprises:

8. The method according to claim 6 or 7, characterized in that, ​ In a case where the target object is a user in the cabin, a space region where a mouth of the target object is located is determined as the sound source position, or in a case where the target object is a space region in the cabin, a partial space region or a whole space region of the target object is determined as the sound source position; and / or, In a case where the masking object is a user in the cabin, a space region where a head of the masking object is located is determined as the sound masking position, or in a case where the masking object is a space region in the cabin, a partial space region or a whole space region of the masking object is determined as the sound masking position.

9. The method of claim 8, wherein, The determination of the space region where the head of the masking object is located as the sound masking position comprises: determining a first space region where the head of the masking object is located based on a perception technology, and determining the first space region as the sound masking position. The perception technology comprises one or more of a machine vision technology, a positioning technology or a sensor technology.

10. The method of claim 9, wherein, After the determination of the first space region as the sound masking position, the method further comprises: In a case where the head of the masking object moves to a second space region, the second space region is determined as the sound masking position, the first space region being different from the second space region.

11. The method according to any one of claims 8-10, characterized in that, The method further comprises: determining the sound masking position.

12. The method according to any one of claims 1 to 11, characterized in that, The method further comprises: Based on the target sound, a masking sound is sent, the masking sound being used to mask the target sound at the sound masking position.

13. The method according to any one of claims 1 to 12, characterized in that, The cabin comprises N sound pickup devices, N being a positive integer; the determination of the sound observation signal in the cabin comprises: picking up the sound observation signal by the N sound pickup devices, the sound observation signal comprising M sub-sound observation signals picked up by M sound pickup devices of the N sound pickup devices, M being a positive integer less than or equal to N.

14. The method of claim 13, wherein, One or more of the M sound pickup devices is located within a first range of the sound masking position.

15. The method according to claim 13 or 14, characterized in that, M = 1, the M sound pickup devices being a first sound pickup device, the first sound pickup device being a sound pickup device closest to the sound masking position among the N sound pickup devices.

16. The method according to any one of claims 13-15, characterized in that, The determination of the masking result based on the sound source position, the sound masking position and the sound observation signal comprises: Based on the sound source position, the sound masking position and the M sub-sound observation signals, a first signal is determined, the first signal being used to describe a signal related to the target sound at the sound masking position; The determination of the masking result based on the first signal comprises:

17. The method of claim 16, wherein, The cabin comprises a masking sound generation device, the masking sound generation device being used to play a masking sound, the masking sound being used to mask the target sound at the sound masking position; The determination of the masking result based on the first signal comprises: A second signal is determined, the second signal being used to describe a signal related to the masking sound at the sound masking position; determine the masking result based on the first signal and the second signal.

18. The method of claim 16 or 17, wherein, The determining the masking result based on the sound source position, the sound masking position and the sound observation signal comprises: based on the sound observation signal S o , a masking sound signal S m , a first transfer function H mo , a second transfer function H sd , and a third transfer function H so , determining the first signal S sd ; the sound observation signal S o , the masking sound signal S m , the first transfer function H mo , the second transfer function H sd , the third transfer function H so and the first signal S sd , satisfy the following relation: S sd = (S o - S m * H mo ) H sd / H so ; wherein the masking sound signal S m for generating the masking sound, the first transfer function H mo for representing a transfer function between the masking sound generating device and the sound picking-up device, the second transfer function H sd for representing a transfer function between the sound source position and the sound masking position, the third transfer function H so for representing a transfer function between the sound source position and the sound picking-up device.

19. The method of claim 16 or 17, wherein, The method further comprises: based on the sound observation signal S o and the masking sound signal S m determining an updated value H mo of the first transfer function H mo ’; The first transfer function H mo The first transfer function H mo Updated value H mo ', Fourth transfer function H md and the fourth transfer function H md 'Satisfies the following relationship: H md '=H md *H mo ' / H mo ; wherein the masking sound signal S m for generating the masking sound, the first transfer function H mo for representing a transfer function between the masking sound generating device and a sound pickup device, the fourth transfer function H md for representing a transfer function between the masking sound generating device and the sound masking location.

20. The method of claim 18, wherein, The determining the masking result based on the first signal and the second signal comprises: The ratio R of the signal strength of the second signal to the signal strength of the first signal is less than R L , the masking result is determined as a failure state; or the ratio R is greater than R H , the masking result is determined as a valid state; or the ratio R is greater than or equal to R L , and the ratio R is less than or equal to R H , the masking result is determined as a pre-warning state; wherein R L ≤ R H .

21. The method of claim 20, wherein, R L The value range of R includes [5dB, 18dB]. H The value range includes [10dB, 30dB].

22. The method of claim 20 or 21, wherein, The cabin comprises a sound masking system configured to mask the target sound at the sound masking position; and the determining the masking result based on the first signal and the second signal comprises: In a case where the signal strength of the first signal is greater than or equal to a third threshold value, the masking result is determined as an invalid state; or in a case where the signal strength of the first signal is less than or equal to a fourth threshold value, the masking result is determined as an effective state; or in a case where the signal strength of the first signal is between the third threshold value and the fourth threshold value, the masking result is determined as a pre-warning state. And / or, In a case where the signal strength of the second signal is less than or equal to a fifth threshold value, the masking result is determined as an invalid state; or in a case where the signal strength of the second signal is greater than or equal to a sixth threshold value, the masking result is determined as an effective state; or in a case where the signal strength of the second signal is between the fifth threshold value and the sixth threshold value, the masking result is determined as a pre-warning state.

23. The method of any one of claims 20-22, wherein, The first reminder information is used to remind the user of the masking result.

24. The method of any one of claims 1-23, wherein, The method further comprises prompting the user with the first reminder information.

25. A reminding method characterized by comprising: The method comprises: receiving a first instruction from a user, the first instruction being used to indicate at least one target object in a cabin, the sound emitted by the target object being a target sound; and / or, the first instruction being used to indicate at least one masking object in the cabin, the masking object being an object to be masked from the target sound; the target object and the masking object each comprising a user in the cabin or a space region in the cabin; in response to the first instruction, displaying the at least one target object and / or displaying the at least one masking object.

26. The method of claim 25, wherein, The method further comprises: receiving a second instruction from a user, the second instruction being used to indicate adjustment of the at least one target object, the at least one target object before adjustment being different from the at least one target object after adjustment; and / or, the second instruction being used to indicate adjustment of the at least one masking object, the at least one masking object before adjustment being different from the at least one masking object after adjustment; in response to the second instruction, displaying the at least one target object after adjustment and / or displaying the at least one masking object after adjustment.

27. The method of claim 25 or 26, wherein, The method further comprises: displaying volume information of the target sound; or in a case where the volume of the target sound is less than or equal to a first threshold value, displaying information that the volume of the target sound is in an appropriate range, or in a case where the volume of the target sound is greater than the first threshold value, displaying relevant information that the volume of the target sound is too large; The information related to the volume of the target sound being too large includes information reminding the user to reduce the volume of the target sound.

28. The method as claimed in any one of claims 25 to 27, wherein, The at least one target object includes a first object, a position at which the first object emits sound being a sound source position, and the at least one masking object includes a second object, a position at which the second object receives sound being a sound masking position; the method further includes: displaying distance information between the sound source position and the sound masking position; and / or in a case where the distance between the sound source position and the sound masking position is greater than or equal to a second threshold, displaying information that the distance between the sound source position and the sound masking position is within a suitable range, or in a case where the distance between the sound source position and the sound masking position is less than the second threshold, displaying information related to the distance between the sound source position and the sound masking position being too small; The information related to the distance between the sound source position and the sound masking position being too small includes information reminding the user to increase the distance between the sound source position and the sound masking position.

29. The method of claim 28, wherein, In a case where the second object is a user in a cabin, the method further includes: in a case where the head of the second object moves, displaying a spatial region in which the head is located after the head moves; wherein the head of the second object moves includes the head moving a range greater than a second range.

30. The method of any one of claims 25-29, wherein, The method further includes: displaying a masking result of the target sound being masked by the masking object, the masking result including one or more of masking success, masking failure, or masking warning.

31. A reminding device, comprising a module or unit for performing the method of any one of claims 1-23, or a module or unit for performing the method of any one of claims 24-30.

32. A reminder device, comprising: comprising: a processor; when the processor invokes a computer program or instructions in a memory, causing the method of any one of claims 1-23 to be performed, or causing the method of any one of claims 24-30 to be performed.

33. A reminder system, comprising: The reminding system comprises the reminding device of claim 31 or 32.

34. The reminder system of claim 33, wherein, The reminding system further comprises a sound masking system configured to mask a target sound at a target position.

35. A vehicle characterized by The vehicle comprises the reminding device of claim 31 or 32, or the reminding system of claim 33 or 34.

36. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store a computer program, which, when executed, causes the method of any one of claims 1-23 to be performed, or the method of any one of claims 24-30 to be performed.

37. A computer program product, characterised in that, The computer program product comprises instructions, which, when executed by a processor, cause the method of any one of claims 1-23 to be implemented, or the method of any one of claims 24-30 to be implemented.

Images