Vehicle exterior integrated sound production device based on multi-module configuration and sound production feedback system

Abstract

The invention relates to the technical field of loudspeakers, in particular to an external integrated sound production device based on multi-module configuration and a sound production feedback system. Comprising a sound production control unit and a signal cooperation unit, data feedback sound production signals of a visual acquisition instrument and a turning prompt sensor are received through the signal cooperation unit, and the sound production control unit receives control signals to drive a whistling horn to produce sound. The signal cooperation unit at least comprises a pedestrian sensing module, a behavior recognition module, a distance prompting module and an out-vehicle calling module. On one hand, the pedestrian approaching unintentionally is reminded, the closer the distance is, the higher the voice volume is, the pedestrian warning probability is increased, and the vehicle driving safety is improved, on the other hand, the pedestrian and a driver can know the distance condition in time, avoidance planning is facilitated, risks are reduced, and the driving safety is improved. And the driver can communicate with external pedestrians in the vehicle, so that risk avoidance in the vehicle driving process is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of loudspeakers, and in particular to an off-vehicle integrated sound device and a sound feedback system based on a multi-module configuration. Background Art

[0002] The car speaker is an indispensable and important device in a sound system. The magnet and coil inside produce sound through vibration. When the car is driving, the user presses the switch and the car speaker emits sound outside the car to remind pedestrians. Current cars lack sound-generating functions. When pedestrians approach the vehicle, the driver must press a switch to make a sound to alert the pedestrians. When the driver fails to react or pedestrians suddenly approach the vehicle, the vehicle cannot issue an autonomous warning sound in time, which can easily lead to dangerous situations. In addition, when someone outside the car needs to communicate with the inside of the car, communication is inconvenient due to the sound insulation inside and outside the car. In view of this, we propose an integrated sound device and sound feedback system outside the car based on a multi-module configuration. Summary of the invention

[0003] The purpose of the present invention is to provide an off-vehicle integrated sound device and a sound feedback system based on a multi-module configuration to solve the problems raised in the above-mentioned background technology.

[0004] In order to solve the above technical problems, one of the objectives of the present invention is to provide an integrated sound-generating device outside the vehicle based on a multi-module configuration, including a sound-generating body for installation outside the vehicle, the sound-generating body at least including an outer shell, a horn, a turn warning sensor and a visual collector are embedded in the outer shell, the horn is used to issue a reminder sound, the turn warning sensor is used to sense the vehicle turning signal and then transmit a control signal to drive the horn to issue a sound, and the visual collector is used to collect image data outside the vehicle.

[0005] Preferably, the rear surface of the outer shell is connected with a heat dissipation shell, which is used for heat exchange with the interior of the outer shell, and the front surface of the outer shell is connected with a dustproof shell, which is used to reduce the entry of dust into the interior of the outer shell, and the visual collector passes through the center of the dustproof shell.

[0006] The second object of the present invention is to provide a sound feedback system based on a multi-module configuration of an off-vehicle integrated sound device, which is applied to any of the above-mentioned off-vehicle integrated sound devices based on a multi-module configuration, including a sound control unit and a signal coordination unit, which receives data feedback sound signals from a visual collector and a turn prompt sensor through the signal coordination unit, and converts the sound signals into control signals, so that the sound control unit receives the control signals to drive the horn to sound; The signal coordination unit at least includes a pedestrian perception module, a behavior recognition module, a distance prompt module and an external vehicle shouting module; The pedestrian sensing module is used to receive the image data outside the vehicle output by the visual acquisition device, identify pedestrians through target detection and tracking algorithms, extract pedestrian features to estimate the distance between the pedestrian and the vehicle, and output a pedestrian signal when the distance value is less than the safety threshold; The behavior recognition module is used to train the pedestrian intention model by annotating the images of pedestrian postures and actions of two types of pedestrians, unintentional approach and intentional approach, as training sets and inputting them into the convolutional neural network architecture, learning the features of different postures and actions to predict the behavior intention type, and receiving the pedestrian signal of the pedestrian perception module, extracting the pedestrian posture and action from the features of the external image data and inputting them into the pedestrian intention model; When the unintentional approach signal is output, the distance value received by the pedestrian sensing module is substituted into the voice output rule through the distance prompt module, so that the sound control unit drives the horn to output the voice, and establishes an inverse proportional relationship between the distance and the volume; When the intentional approach signal is output, the vehicle interior and the horn are connected through the external voice module, the sound signal inside the vehicle is converted into an electrical signal, and the horn is controlled to convert the electrical signal into a sound signal to push the surrounding air molecules to generate sound waves.

[0007] Preferably, the pedestrian perception module identifies pedestrians through a target detection and tracking algorithm, comprising the following steps: The vehicle exterior image data is divided into multiple grids, each of which is responsible for predicting a fixed number of bounding boxes and the probability and category of the target contained in the bounding box. A convolutional neural network is used to extract the features of the target in the bounding box and output the feature vector , and pre-transfer the pedestrian feature template in the database , by calculating the cosine distance between eigenvectors: , if the cosine distance is less than the probability threshold, the match is successful and the output is that there is a pedestrian outside the car.

[0008] Preferably, extracting pedestrian features in the pedestrian perception module to estimate the distance value from the pedestrian to the vehicle includes the following steps: When a pedestrian is detected outside the car, the feature of the proportion of the pedestrian in the entire image is extracted, and the pixel height occupied by the pedestrian in the image is output. , and the actual height of the pedestrian is , based on the principle of similar triangles, estimate the distance from the pedestrian to the vehicle: ; Wherein, D is the distance from the pedestrian to the vehicle, and F is the focal length of the camera. The distance from the pedestrian to the vehicle is compared with the safety threshold through a numerical comparison algorithm. When D is less than the safety threshold, a pedestrian signal is output.

[0009] Preferably, the safety threshold is determined using a dynamic threshold adjustment algorithm, which is used to sense the braking performance to obtain the maximum braking deceleration, calculate the braking distance in combination with the current driving speed, and integrate the factor distance to output the safety threshold = braking distance + factor distance.

[0010] Preferably, the pedestrian intention model adopts a convolutional neural network, comprising the following steps: Collect a large amount of image data containing pedestrian postures and movements, extract image features for annotation, classify pedestrian postures and movements into two categories: unintentional approach and intentional approach, and pre-process the annotated image data; The number of convolutional layers, filter size, and pooling layer parameters are adjusted to adapt to the pedestrian intention model. The preprocessed image data is used as input and the labeled categories are used as output. They are input into the convolutional neural network for training, and the optimization algorithm and loss function are used to optimize the network parameters so that the network can learn the characteristics of different postures and movements and predict the pedestrian's behavioral intention type.

[0011] Preferably, when extracting image features for annotation and extracting pedestrian postures and movements from the features of the external image data, a human key point detection algorithm is used, including the following steps: Collect pedestrian image data covering various poses and actions to ensure that the model can learn diverse features; Use OpenPose to process pedestrian image data and output the coordinate information of key nodes converted into feature vectors. The key points include head, shoulders, elbows, hands, hips, knees and feet. Calculate the position changes of key points between adjacent frames and obtain motion information.

[0012] Preferably, the voice output rule is to set a voice output template and annotate key positions, receive the distance value of the pedestrian perception module to replace the key positions to form new voice content, convert the voice content into voice through voice synthesis technology, and establish a mapping rule between the distance value and the volume of the horn sound. The sound control unit synchronously receives the voice and distance information, calculates the corresponding volume value according to the pre-set mapping relationship, and drives the horn to output voice with the corresponding volume value.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The pedestrian perception module recognizes pedestrians based on the external image data, and the behavior recognition module extracts the pedestrian posture and action from the external image data features and inputs them into the pedestrian intention model; when outputting the unintentional approach signal, the distance value of the pedestrian perception module is received by the distance prompt module and substituted into the voice output rule, so that the sound control unit drives the horn to output voice and establishes an inverse proportional relationship between distance and volume. On the one hand, it reminds pedestrians who are approaching unintentionally to avoid traffic accidents caused by pedestrians not noticing the vehicle. At the same time, according to the inverse proportional relationship between distance and volume, as the distance gets closer, the voice volume gets louder, which increases the probability of pedestrians being alert and improves vehicle driving safety. On the other hand, by sending out the distance value between pedestrians and vehicles through voice, both pedestrians and drivers can understand the distance situation in time, which is conducive to making avoidance plans and reducing risks; when outputting the intentional approach signal, the external voice module is connected to the car and the horn, and the sound signal in the car is converted into an electrical signal. The horn is controlled to convert the electrical signal into a sound signal to push the surrounding air molecules to form sound waves, so that the driver can communicate with pedestrians outside in the car, avoiding unclear communication due to long distance or vehicle sound insulation problems. Especially when facing traffic police command communication or pedestrian reminders, the intentions of both parties can be understood in time, which is conducive to avoiding risks during vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is the overall principle block diagram of the present invention; Figure 2 It is a structural diagram of the sound-generating body of the present invention; Figure 3 It is a principle block diagram of the signal coordination unit of the present invention.

[0015] The meaning of each number in the figure is: 100. Sound-generating body; 110. Outer shell; 111. Heat dissipation shell; 112. Dust-proof shell; 120. Horn; 130. Turn warning sensor; 140. Visual acquisition device; 200. Sound-generating control unit; 300. Signal coordination unit. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] like Figure 1-3As shown, one of the purposes of the present invention is to provide an integrated sound device outside the vehicle based on a multi-module configuration, including a sound-generating body 100 for installation outside the vehicle, the sound-generating body 100 at least including an outer shell 110, a horn 120, a turn prompt sensor 130 and a visual collector 140 are embedded in the outer shell 110, the horn 120 is used to make a reminder sound, the turn prompt sensor 130 is used to sense the vehicle turning signal and then transmit a control signal to drive the horn 120 to make a sound, and the visual collector 140 is used to collect image data outside the vehicle, wherein: The working principle of the whistle speaker 120 is based on the principle of electromagnetic induction. When the audio current passes through the voice coil of the speaker, the voice coil generates a changing magnetic field. This magnetic field interacts with the magnetic field generated by the permanent magnet inside the speaker, so that the voice coil is acted upon by the force in the magnetic field and moves. The movement of the voice coil drives the diaphragm cone of the speaker to vibrate, thereby pushing the air to form sound waves. The vibration frequency and amplitude of the diaphragm correspond to the frequency and amplitude of the input audio signal, so that the electrical signal can be converted into a sound signal, allowing us to hear various sounds, such as music, voice, etc. The working principle of the turn signal sensor 130 is to detect angle changes using technologies such as Hall effect sensors, potentiometers or optical encoders. When the vehicle turns, the steering angle sensor sends the steering angle information to the vehicle's electronic control unit ECU, and the ECU controls the lighting of the turn signal lamp or other related functions based on the information, thereby transmitting the turn signal to the horn 120, converting the turn signal into voice text and sending it out from the horn 120; The visual acquisition device 140 usually adopts a camera, which converts the optical signal into an electrical signal, and an image sensor such as CMOS or CCD converts the electrical signal into a digital image signal for capturing the scene around the vehicle; Therefore, the present invention places the horn 120, the turn warning sensor 130 and the visual collector 140 in the outer shell 110 to form an integrated structure, which can be installed on the floor or anti-collision beam position in the front cabin, which can meet customers' requirements for cost, installation space and sound customization, and greatly improve the usage needs of drivers and passengers in multiple scenarios and multiple applications.

[0018] Specifically, the rear surface of the outer shell 110 is connected and snapped with a heat dissipation shell 111, and the heat dissipation shell 111 is used for heat exchange with the interior of the outer shell 110. The front surface of the outer shell 110 is connected and snapped with a dustproof shell 112, and the dustproof shell 112 is used to reduce the dust from entering the interior of the outer shell 110. The visual collector 140 runs through the center of the dustproof shell 112. While the visual collector 140 collects the scene outside the vehicle, the dust is reduced from contacting the horn 120, thereby increasing the service life. At the same time, the heat dissipation shell 111 performs heat exchange and dissipation to avoid high temperature generated by the internal equipment of the outer shell 110 when in use, thereby further increasing the service life.

[0019] The second object of the present invention is to provide a sound feedback system based on a multi-module configuration of an off-vehicle integrated sound device, including any one of the above-mentioned off-vehicle integrated sound devices based on a multi-module configuration, including a sound control unit 200 and a signal coordination unit 300, receiving a data feedback sound signal from a visual acquisition device 140 and a turn prompt sensor 130 through the signal coordination unit 300, and converting the sound signal into a control signal, so that the sound control unit 200 receives the control signal to drive the horn 120 to sound, wherein the sound control unit 200 converts the electrical signal into a sound signal by controlling the horn 120, so that the horn 120 can generate different vibrations according to the electrical signal and output different sound waves, which is conducive to subsequent honking and emitting voice; The signal coordination unit 300 includes at least a pedestrian perception module, a behavior recognition module, a distance prompt module and an external vehicle shouting module; The pedestrian sensing module is used to receive the vehicle exterior image data output by the visual acquisition device 140, identify pedestrians through target detection and tracking algorithms, extract pedestrian features to estimate the distance between pedestrians and vehicles, and output pedestrian signals when the distance value is less than the safety threshold; The behavior recognition module is used to train the pedestrian intention model by annotating the images of pedestrian postures and actions of two types of pedestrians, unintentional approach and intentional approach, as training sets and inputting them into the convolutional neural network architecture. It learns the features of different postures and actions to predict the behavior intention type, receives the pedestrian signal from the pedestrian perception module, extracts the pedestrian posture and action from the features of the external image data and inputs them into the pedestrian intention model. Furthermore, the pedestrian perception module identifies pedestrians through target detection and tracking algorithms, including the following steps: The vehicle exterior image data is divided into multiple grids, each of which is responsible for predicting a fixed number of bounding boxes and the probability and category of the target contained in the bounding box. A convolutional neural network is used to extract the features of the target in the bounding box and output the feature vector , whose dimensions are , and pre-set pedestrian feature templates in the database , whose dimension is , by calculating the cosine distance between eigenvectors: If the cosine distance is less than the probability threshold, the match is successful and the output is that there is a pedestrian outside the car. As for the determination of the probability threshold, it is necessary to strike a balance between false positives that misjudge non-pedestrians as pedestrians and false negatives that miss pedestrians. A lower threshold may increase the false alarm rate but reduce the false alarm rate; a higher threshold may reduce the false alarm rate but increase the false alarm rate. According to the needs of the specific application, determine the tolerance for false alarms and false negatives, and then select a suitable threshold. You can start with an initial threshold, such as 0.8, and then test it on the validation set. If the false alarm rate is too high, you can appropriately increase the threshold, such as adjusting it to 0.9; if the false alarm rate is too high, you can appropriately lower the threshold. By constantly adjusting and comparing the detection effects under different thresholds, you can eventually determine a threshold that achieves a good balance between false alarms and false negatives, thereby increasing the probability of pedestrian recognition and improving the accuracy of the system.

[0020] It is worth noting that the pedestrian perception module extracts pedestrian features to estimate the distance value from the pedestrian to the vehicle, including the following steps: When a pedestrian is detected outside the car, the feature of the proportion of the pedestrian in the entire image is extracted, and the pixel height occupied by the pedestrian in the image is output. , and the actual height of the pedestrian is , based on the principle of similar triangles, estimate the distance from the pedestrian to the vehicle: ; Among them, D is the distance from the pedestrian to the vehicle, and F is the focal length of the camera. The distance from the pedestrian to the vehicle is compared with the safety threshold through a numerical comparison algorithm. When D is less than the safety threshold, a pedestrian signal is output. Otherwise, normal driving is performed, which is conducive to judging whether to remind the pedestrian based on the distance. If the pedestrian is far away from the vehicle, there is no need to remind the driver, and the driver has enough time to respond. If the pedestrian is close to the vehicle, the pedestrian is reminded, which is conducive to the driver and the pedestrian to respond at the same time and increase the probability of avoiding danger.

[0021] Specifically, due to the different speeds of vehicles, the braking ranges required by drivers are different. Therefore, if a single safety threshold is used, the error is large. Therefore, a dynamic threshold adjustment algorithm is used when determining the safety threshold. The dynamic threshold adjustment algorithm is used to sense the braking performance to obtain the maximum braking deceleration, calculate the braking distance in combination with the current driving speed, and integrate the factor distance to output the safety threshold = braking distance + factor distance, where the factor distance at least includes the reaction distance. The driver needs a certain reaction time between discovering the obstacle and starting to brake. Generally speaking, the human reaction time is between 0.75 seconds and 1.5 seconds. To be conservative, a longer reaction time can be taken, such as 1.5 seconds. Taking the reaction distance as an example: the factor distance = speed * reaction time, and the braking distance is the distance the vehicle travels from the start of braking to complete stop, the braking distance = speed squared / 2*maximum braking deceleration, and the safety distance is the sum of the factor distance and the braking distance. When the distance between the vehicle and the obstacle in front is greater than the safety distance, the vehicle can theoretically brake and stop before reaching the obstacle, thereby improving driving safety.

[0022] Furthermore, the pedestrian intention model adopts a convolutional neural network, which includes the following steps: Collect a large amount of image data containing pedestrian postures and movements, extract image features for annotation, classify pedestrian postures and movements into two categories: unintentional approach and intentional approach, and preprocess the annotated image data, including image cropping, resizing, normalization, etc., so as to facilitate input into the convolutional neural network for training; The number of convolutional layers, filter size, and pooling layer parameters are adjusted to adapt to the pedestrian intention model. The preprocessed image data is used as input and the labeled categories are used as output. They are input into the convolutional neural network for training, and optimization algorithms such as stochastic gradient descent and loss functions such as cross entropy loss function are used to optimize the network parameters so that the network can learn the characteristics of different postures and movements and predict the behavior intention types of pedestrians. They are applied to actual pedestrian perception scenarios to predict the behavior intention types of pedestrians and provide auxiliary information for safe driving of vehicles.

[0023] Among them, when extracting image features for annotation and extracting pedestrian postures and movements from the features of the external image data, a human key point detection algorithm is used, including the following steps: Collect pedestrian image data covering various poses and actions to ensure that the model can learn diverse features; OpenPose is used to process pedestrian image data and output the coordinate information of key nodes converted into feature vectors. The key points include the head, shoulders, elbows, hands, hips, knees and feet. The position changes of key points between adjacent frames are calculated to obtain motion information. For example, when a pedestrian raises his hand and swings it to indicate parking, when OpenPose detects a specific change in the position of the key points of the pedestrian's hand, such as raising it upward and swinging it, the relevant features can be extracted and input into the pedestrian intention model. The model can recognize that the pedestrian's behavioral intention is to approach intentionally by learning the association between these features and the intention to park.

[0024] When outputting the unintentional approach signal, the distance value received by the pedestrian perception module through the distance prompt module is substituted into the voice output rule, so that the sound control unit 200 drives the horn 120 to output the voice, and establishes an inverse proportional relationship between the distance and the volume. On the one hand, it reminds the pedestrians who are approaching unintentionally to avoid the pedestrians not noticing the vehicle and causing traffic accidents. At the same time, according to the inverse proportional relationship between the distance and the volume, as the distance is closer, the voice volume is louder, which increases the probability of pedestrians being alert and improves the driving safety of the vehicle. On the other hand, by sending the distance value between the pedestrian and the vehicle through voice, both the pedestrian and the driver can understand the distance situation in time, which is conducive to making avoidance plans and reducing risks. When the intentional approach signal is output, the vehicle interior and the horn 120 are connected through the external voice module, the sound signal in the vehicle is converted into an electrical signal, and the horn 120 is controlled to convert the electrical signal into a sound signal to drive the surrounding air molecules to generate sound waves, so that the driver can communicate with pedestrians outside in the car, avoiding unclear communication due to long distance or vehicle sound insulation problems, especially when facing traffic police command communication or pedestrian reminders, the intentions of both parties can be understood in time, which is conducive to avoiding risks during vehicle driving; Specifically, the conversion of sound signals involves the acquisition, encoding and decoding technology of audio signals. After the sound inside the car is collected by a microphone, the sound signal is converted into a digital signal through analog-to-digital conversion. Then, the digital signal is transmitted to the horn 120 outside the car through a communication link. The digital signal is then restored to an analog electrical signal through digital-to-analog conversion to drive the horn 120 to make a sound. The working principle of the horn 120 is to convert the electrical signal into a sound signal. When the electrical signal passes through the electromagnet, a changing magnetic field is generated, which interacts with the magnetic field of the permanent magnet to vibrate the diaphragm and push the air molecules to generate sound waves.

[0025] The voice output rule is to set a voice output template and annotate key positions, receive the distance value of the pedestrian perception module to replace the key positions to form new voice content, convert the voice content into voice through voice synthesis technology, and establish a mapping rule between the distance value and the sound volume of the horn 120. The sound control unit 200 synchronously receives the voice and distance information, calculates the corresponding volume value according to the preset mapping relationship, and drives the horn 120 to output the voice of the corresponding volume value; Among them, speech synthesis technology analyzes the input text content, including lexical analysis, syntactic analysis and semantic understanding, to better understand the meaning and structure of the text, and uses speech acoustic models to simulate the acoustic features of speech, such as spectrum, fundamental frequency, duration, etc. Common acoustic models include statistical-based models such as hidden Markov model, HMM and deep learning models such as convolutional neural network, CNN; recurrent neural network, RNN; long short-term memory network, LSTM, etc. According to the pronunciation rules of the text, the text is converted into the corresponding phoneme sequence, considering the prosodic features of the speech, such as intonation, stress, rhythm, etc., to make the synthesized speech more natural and fluent, and synthesize the speech signal through digital signal processing technology according to the parameters generated by the acoustic model and prosodic model.

[0026] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An integrated sound device outside the vehicle is configured based on multiple modules, characterized in that: The invention comprises a sound-generating body (100) for installation outside a vehicle, wherein the sound-generating body (100) comprises at least an outer shell (110), wherein a whistle horn (120), a turn prompting sensor (130) and a visual acquisition device (140) are embedded inside the outer shell (110), wherein the whistle horn (120) is used to emit a warning sound, the turn prompting sensor (130) is used to sense a vehicle turn signal and then transmit a control signal to drive the whistle horn (120) to emit a sound, and the visual acquisition device (140) is used to collect image data outside the vehicle.

2. The multi-module configuration vehicle-mounted integrated sound device according to claim 1, characterized in that: The rear surface of the outer shell (110) is connected to a heat dissipation shell (111), and the heat dissipation shell (111) is used to perform heat exchange with the interior of the outer shell (110). The front surface of the outer shell (110) is connected to a dustproof shell (112), and the dustproof shell (112) is used to reduce dust from entering the interior of the outer shell (110). The visual acquisition instrument (140) passes through the center of the dustproof shell (112).

3. A sound feedback system based on a multi-module configuration of an off-vehicle integrated sound device, applied to the off-vehicle integrated sound device based on a multi-module configuration as claimed in any one of claims 1-2, characterized in that: It comprises a sound control unit (200) and a signal coordination unit (300), wherein the signal coordination unit (300) receives a sound signal fed back by a visual acquisition device (140) and a turn prompt sensor (130), and converts the sound signal into a control signal, so that the sound control unit (200) receives the control signal and drives the horn (120) to produce a sound; The signal coordination unit (300) comprises at least a pedestrian perception module, a behavior recognition module, a distance prompt module and an external vehicle shouting module; The pedestrian sensing module is used to receive the image data outside the vehicle output by the visual acquisition device (140), identify pedestrians through target detection and tracking algorithms, extract pedestrian features to estimate the distance between the pedestrian and the vehicle, and output a pedestrian signal when the distance value is less than a safety threshold; The behavior recognition module is used to train the pedestrian intention model by annotating the images of pedestrian postures and actions of two types of pedestrians, unintentional approach and intentional approach, as training sets and inputting them into the convolutional neural network architecture, learning the features of different postures and actions to predict the behavior intention type, and receiving the pedestrian signal of the pedestrian perception module, extracting the pedestrian posture and action from the features of the external image data and inputting them into the pedestrian intention model; When an unintentional approach signal is output, the distance value received from the pedestrian sensing module is substituted into a voice output rule through a distance prompt module, so that the sound control unit (200) drives the horn (120) to output the voice, and an inverse proportional relationship between the distance and the volume is established; When the intentional approach signal is output, the vehicle interior and the horn (120) are connected via the external voice module, the sound signal inside the vehicle is converted into an electrical signal, and the horn (120) is controlled to convert the electrical signal into a sound signal to push the surrounding air molecules to generate sound waves.

4. The sound feedback system based on the multi-module configuration of the off-vehicle integrated sound device according to claim 3 is characterized in that: The pedestrian perception module identifies pedestrians through target detection and tracking algorithms, including the following steps: The vehicle exterior image data is divided into multiple grids, each of which is responsible for predicting a fixed number of bounding boxes and the probability and category of the target contained in the bounding box. A convolutional neural network is used to extract the features of the target in the bounding box and output the feature vector , and pre-transfer the pedestrian feature template in the database , by calculating the cosine distance between eigenvectors: , if the cosine distance is less than the probability threshold, the match is successful and the output is that there is a pedestrian outside the car.

5. The sound feedback system based on the multi-module configuration of the vehicle external integrated sound device according to claim 4 is characterized in that: Extracting pedestrian features in the pedestrian perception module to estimate the distance value from the pedestrian to the vehicle includes the following steps: When a pedestrian is detected outside the car, the feature of the proportion of the pedestrian in the entire image is extracted, and the pixel height occupied by the pedestrian in the image is output. , and the actual height of the pedestrian is , based on the principle of similar triangles, estimate the distance from the pedestrian to the vehicle: ; Wherein, D is the distance from the pedestrian to the vehicle, and F is the focal length of the camera. The distance from the pedestrian to the vehicle is compared with the safety threshold through a numerical comparison algorithm. When D is less than the safety threshold, a pedestrian signal is output.

6. The sound feedback system based on the multi-module configuration of the off-vehicle integrated sound device according to claim 5, characterized in that: The safety threshold is determined using a dynamic threshold adjustment algorithm, which is used to sense the braking performance to obtain the maximum braking deceleration, calculate the braking distance in combination with the current driving speed, and integrate the factor distance to output the safety threshold = braking distance + factor distance.

7. The sound feedback system based on the multi-module configuration of the off-vehicle integrated sound device according to claim 6, characterized in that: The pedestrian intention model adopts a convolutional neural network and includes the following steps: Collect a large amount of image data containing pedestrian postures and movements, extract image features for annotation, classify pedestrian postures and movements into two categories: unintentional approach and intentional approach, and pre-process the annotated image data; The number of convolutional layers, filter size, and pooling layer parameters are adjusted to adapt to the pedestrian intention model. The preprocessed image data is used as input and the labeled categories are used as output. They are input into the convolutional neural network for training, and the optimization algorithm and loss function are used to optimize the network parameters so that the network can learn the characteristics of different postures and movements and predict the pedestrian's behavioral intention type.

8. The sound feedback system based on the multi-module configuration of the off-vehicle integrated sound device according to claim 7, characterized in that: When extracting image features for annotation and extracting pedestrian postures and movements from the features of the external image data, a human key point detection algorithm is used, including the following steps: Collect pedestrian image data covering various poses and actions to ensure that the model can learn diverse features; Use OpenPose to process pedestrian image data and output the coordinate information of key nodes converted into feature vectors. The key points include head, shoulders, elbows, hands, hips, knees and feet. Calculate the position changes of key points between adjacent frames and obtain motion information.

9. The sound feedback system based on the multi-module configuration of the off-vehicle integrated sound device according to claim 8, characterized in that: The speech output rule is to set a speech output template and annotate key positions, receive the distance value of the pedestrian perception module to replace the key positions to form new speech content, convert the speech content into speech through speech synthesis technology, and simultaneously establish a mapping rule between the distance value and the sound volume of the horn (120), the sound control unit (200) synchronously receives the speech and distance information, calculates the corresponding volume value according to a pre-set mapping relationship, and drives the horn (120) to output speech with the corresponding volume value.

Images