Traffic jam state determination method and device, vehicle and storage medium

Abstract

The invention discloses a traffic jam state determination method and device, a vehicle and a storage medium, and the method comprises the steps: obtaining continuous frame information shot by a camera device of the vehicle and an environment audio signal collected by an audio device of the vehicle when it is monitored that the driving speed of the vehicle meets a jam early-warning vehicle speed; a first motion state and a second motion state of a front vehicle are determined according to the continuous frame information, sound information of the front vehicle is determined according to the environment audio signal, and the front vehicle comprises at least one vehicle in front of the vehicle; determining a front vehicle motion state of the front vehicle according to the first motion state and the second motion state, and determining a front vehicle sound state of the front vehicle according to the front vehicle sound information; and determining and displaying a traffic jam state according to the front vehicle motion state and the front vehicle sound state. According to the method, the problem that a large judgment error is generated due to visual errors or perception differences of drivers is solved, the current traffic jam state is rapidly and accurately provided for the drivers, and the safety and experience feeling of the drivers in driving the vehicle are improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a method, device, vehicle and storage medium for determining a traffic jam state. Background Art

[0002] In today's driving environment, drivers often encounter traffic jams, such as highway congestion during holidays. In these situations, drivers typically rely on visual observation or subjective judgment to determine whether the traffic jam is still ongoing. If the traffic jam is over, they control the vehicle to start and accelerate to leave.

[0003] However, if the above-mentioned method of judging traffic congestion only relies on the driver's observation or subjective judgment of the traffic congestion state, it will produce a large judgment error due to the driver's visual error or perception difference. Summary of the Invention

[0004] The present invention provides a method, device, vehicle and storage medium for determining traffic jam status, which solves the problem of large judgment errors caused by the driver's visual errors or perception differences, provides the driver with the current traffic jam status quickly and accurately, and improves the driver's driving safety and experience.

[0005] According to one aspect of the present invention, a method for determining a traffic jam state is provided, which is applied to a vehicle. The method comprises:

[0006] When it is monitored that the driving speed of the vehicle meets the congestion warning speed, continuous frame information captured by the vehicle's camera equipment and environmental audio signals collected by the vehicle's audio equipment are obtained.

[0007] A first motion state and a second motion state of a preceding vehicle are determined based on continuous frame information, and preceding vehicle sound information is determined based on an ambient audio signal, wherein the preceding vehicle includes at least one vehicle in front of the vehicle.

[0008] The preceding vehicle motion state of the preceding vehicle is determined based on the first motion state and the second motion state, and the preceding vehicle sound state of the preceding vehicle is determined based on the preceding vehicle sound information.

[0009] The traffic jam status is determined and displayed based on the movement status and sound status of the preceding vehicle.

[0010] The traffic jam state determination method provided by an embodiment of the present invention obtains continuous frame information captured by the vehicle's camera device and ambient audio signals collected by the vehicle's audio device when the vehicle's driving speed is detected to meet the congestion warning speed; determines the first and second motion states of the preceding vehicle based on the continuous frame information, and determines the preceding vehicle sound information based on the ambient audio signals; determines the preceding vehicle's motion state based on the first and second motion states, and determines the preceding vehicle sound state based on the preceding vehicle sound information; and determines and displays the traffic jam state based on the preceding vehicle's motion state and sound state. The above technical solution, on the one hand, controls the camera device and audio device to start collecting data only when the vehicle's driving speed meets the congestion warning speed, thereby reducing the vehicle's energy consumption and providing accurate data support for subsequent determination of the traffic jam state. On the other hand, the first and second motion states are visual representations of the preceding vehicle's state, while the preceding vehicle sound information is audio representations of the preceding vehicle's state. Therefore, the preceding vehicle's motion state and sound state can provide a variety of common judgment indicators for subsequent determination of the traffic jam state from both a visual and audio perspective. Finally, the traffic jam status is jointly determined based on the movement status of the preceding vehicle in the visual mode and the sound status of the preceding vehicle in the auditory mode. This solves the problem that the existing technology will produce large judgment errors due to the driver's visual errors or perception differences. It provides the driver with the current traffic jam status quickly and accurately, improving the driver's driving safety and experience.

[0011] According to another aspect of the present invention, a traffic jam state determination device is provided, which is applied to a vehicle; the device comprises:

[0012] The acquisition module is used to obtain continuous frame information captured by the vehicle's camera equipment and environmental audio signals collected by the vehicle's audio equipment when it is monitored that the vehicle's driving speed meets the congestion warning speed.

[0013] The processing module is used to determine the first motion state and the second motion state of the preceding vehicle according to the continuous frame information, and to determine the preceding vehicle sound information according to the ambient audio signal, wherein the preceding vehicle includes at least one vehicle in front of the vehicle.

[0014] The determining module is configured to determine a preceding vehicle motion state of the preceding vehicle according to the first motion state and the second motion state, and to determine a preceding vehicle sound state of the preceding vehicle according to the preceding vehicle sound information.

[0015] The display module is used to determine and display the traffic jam status according to the movement status and sound status of the preceding vehicle.

[0016] According to another aspect of the present invention, there is provided a vehicle, the vehicle comprising:

[0017] electronic devices;

[0018] The electronic device includes at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the traffic jam state determination method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining a traffic jam state according to any embodiment of the present invention is implemented.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the traffic jam state determination method according to any embodiment of the present invention when executed.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flow chart of a method for determining a traffic jam state provided by an embodiment of the present invention;

[0026] Figure 2 A flow chart of another method for determining a traffic jam state provided by an embodiment of the present invention;

[0027] Figure 3 A schematic structural diagram of a traffic jam status determination device provided by an embodiment of the present invention;

[0028] Figure 4 A schematic diagram of the structure of an electronic device in a vehicle provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Figure 1 This is a flow chart of a method for determining a traffic jam state provided by an embodiment of the present invention. This embodiment is applicable to situations where a driver encounters road congestion while driving a vehicle. The method can be executed by a traffic jam state determination device, which can be implemented in the form of hardware and / or software. The traffic jam state determination device can be configured in an electronic device in the vehicle. In this embodiment, the electronic device can be a vehicle controller, an engine control unit, or a hybrid power controller, which is determined specifically according to the vehicle type. Figure 1 As shown, the method includes:

[0032] S101: When it is monitored that the driving speed of a vehicle meets the congestion warning speed, continuous frame information captured by a camera device of the vehicle and an ambient audio signal collected by an audio device of the vehicle are obtained.

[0033] The congestion warning speed is the vehicle speed threshold when congestion occurs. In this embodiment, the congestion warning speed can be obtained through experiments or tests, or can be determined based on the vehicle's current city, road type, or speed limit. Continuous frame information refers to image frames captured by a camera and arranged in chronological order. Ambient audio signals refer to the ambient sound signals around the vehicle collected by an audio device.

[0034] Specifically, the vehicle can determine the congestion warning speed based on the road type and speed limit of the current road. Furthermore, when the vehicle's current speed is detected to meet the congestion warning speed, the camera installed in front of the vehicle and the audio equipment installed around the vehicle are activated. The camera captures images in front of the vehicle to obtain continuous frame information, and the audio equipment collects sounds around the vehicle to obtain ambient audio signals.

[0035] For example, an adaptive system for an elevated road ramp can determine that when the speed on the bridge is less than 20 kilometers per hour (km / h), it is considered severe congestion; when the speed is between 20-35 km / h, it may be moderate congestion; and when the speed is between 35-50 km / h, it may be light congestion. In this case, the congestion warning speed can be determined to be 0-50 km / h. When the vehicle's speed is between 0-50 km / h, it meets the congestion warning speed. At this time, the camera and audio equipment will be turned on.

[0036] In this embodiment, the camera and audio equipment will be controlled to start collecting data only when the vehicle's driving speed meets the congestion warning speed, which can reduce the vehicle's energy consumption and provide accurate data support for subsequent determination of the traffic jam status.

[0037] S102 : Determine a first motion state and a second motion state of a preceding vehicle according to the continuous frame information, and determine sound information of the preceding vehicle according to the ambient audio signal.

[0038] The first motion state and the second motion state are motion states of the preceding vehicle obtained by observing the preceding vehicle's movements from different angles. In this embodiment, the first motion state is the change in the preceding vehicle's position in consecutive image frames, and the second motion state is the change in the preceding vehicle's indicator lights in consecutive image frames. The preceding vehicle includes at least one vehicle in front of the preceding vehicle.

[0039] Specifically, by analyzing the video captured by the camera in consecutive frames, the displacement changes of the preceding vehicle in the consecutive frames, i.e., the first motion state, and the changes in the indicator lights of the preceding vehicle, i.e., the second motion state, can be obtained. Furthermore, based on the ambient audio signal, the sound emitted by the preceding vehicle or vehicles in the environment can be determined.

[0040] S103 : determining a preceding vehicle motion state of the preceding vehicle according to the first motion state and the second motion state, and determining a preceding vehicle sound state of the preceding vehicle according to the preceding vehicle sound information.

[0041] Specifically, the first motion state can be used to determine whether the preceding vehicle has displaced, and the second motion state can be used to predict the motion that the preceding vehicle may trigger. Therefore, the preceding vehicle's motion state can be determined based on the first and second motion states. In this embodiment, there may be multiple preceding vehicles, and therefore, the preceding vehicle motion states of multiple preceding vehicles can be determined. Furthermore, the presence of sounds that occur when the preceding vehicle starts or accelerates can be determined from the preceding vehicle's sound information to determine the preceding vehicle's state.

[0042] In this embodiment, the first motion state and the second motion state present the state of the preceding vehicle in a visual form, while the preceding vehicle sound information presents the state of the preceding vehicle in the form of sound. Therefore, obtaining the preceding vehicle motion state and the preceding vehicle sound state can provide a variety of common judgment indicators for subsequent determination of the traffic jam state from the perspective of visualization and sound.

[0043] S104: Determine and display a traffic jam status based on the movement status and sound status of the preceding vehicle.

[0044] Specifically, the preceding vehicle's motion state reflects its visual modality, while its sound state reflects its auditory modality. By combining the preceding vehicle's motion state (visual modality) and the preceding vehicle's sound state (auditory modality), a joint judgment based on these different modalities can be made to determine the current traffic jam status. Furthermore, the traffic jam status can be displayed to the driver.

[0045] In this embodiment, the traffic jam status is jointly determined based on the movement status of the preceding vehicle in the visual mode and the sound status of the preceding vehicle in the auditory mode, which solves the problem of large judgment errors caused by the driver's visual errors or perception differences in the existing technology. The current traffic jam status is provided to the driver quickly and accurately, thereby improving the driver's driving safety and experience.

[0046] The traffic jam state determination method provided by an embodiment of the present invention obtains continuous frame information captured by the vehicle's camera device and ambient audio signals collected by the vehicle's audio device when the vehicle's driving speed is detected to meet the congestion warning speed; determines the first and second motion states of the preceding vehicle based on the continuous frame information, and determines the preceding vehicle sound information based on the ambient audio signals; determines the preceding vehicle's motion state based on the first and second motion states, and determines the preceding vehicle sound state based on the preceding vehicle sound information; and determines and displays the traffic jam state based on the preceding vehicle's motion state and sound state. The above technical solution, on the one hand, controls the camera device and audio device to start collecting data only when the vehicle's driving speed meets the congestion warning speed, thereby reducing the vehicle's energy consumption and providing accurate data support for subsequent determination of the traffic jam state. On the other hand, the first and second motion states are visual representations of the preceding vehicle's state, while the preceding vehicle sound information is audio representations of the preceding vehicle's state. Therefore, the preceding vehicle's motion state and sound state can provide a variety of common judgment indicators for subsequent determination of the traffic jam state from both a visual and audio perspective. Finally, the traffic jam status is jointly determined based on the movement status of the preceding vehicle in the visual mode and the sound status of the preceding vehicle in the auditory mode. This solves the problem that the existing technology will produce large judgment errors due to the driver's visual errors or perception differences. It provides the driver with the current traffic jam status quickly and accurately, improving the driver's driving safety and experience.

[0047] Figure 2 This is a flow chart of another method for determining a traffic jam state provided by an embodiment of the present invention. Based on the above embodiment, this embodiment describes in detail the steps of determining the first motion state and the second motion state of the preceding vehicle, determining the preceding vehicle motion state according to the first motion state and the second motion state, and determining and displaying the traffic jam state. Figure 2 As shown, the method includes:

[0048] S201: When it is monitored that the driving speed of a vehicle meets the congestion warning speed, continuous frame information captured by a camera device of the vehicle and an ambient audio signal collected by an audio device of the vehicle are obtained.

[0049] For example, if the vehicle is traveling on a highway with a known speed limit of 60-120 km / h, 30 km / h can be used as the upper limit for the congestion warning speed, i.e., the congestion warning speed is 0-30 km / h. If the vehicle's speed is within the range of 0-30 km / h, the congestion warning speed is met, and the camera can be turned on to capture continuous frames and the audio device can be turned on to collect ambient audio signals.

[0050] S202: Calculate the instantaneous velocity vector of the pixel point according to the degree of change of pixel intensity between every two adjacent frames in the continuous frame information.

[0051] Specifically, because the pixel brightness changes for the same object in two adjacent frames are essentially the same, the pixel intensity changes between the two adjacent frames can be calculated to determine whether the object has changed. Therefore, the instantaneous velocity vector of each pixel can be obtained by taking the difference between the pixel intensities of the previous and next frames.

[0052] S203: Determine traffic flow information based on the vector direction and vector components of the instantaneous velocity vector.

[0053] Specifically, the instantaneous velocity vector consists of two parameters: horizontal velocity and vertical velocity. These two parameters are known as vector components. The direction of the vector can be determined based on the composite direction of these two parameters. Therefore, the average speed and direction can be calculated based on the vector direction and components. This average speed and direction reflects the dominant direction and speed of the overall flow, and can be used to determine traffic flow information.

[0054] Optionally, S202 and S203 may be replaced by a deep learning optical flow method, a Horn-Schunck algorithm, or a Lucas-Kanade algorithm.

[0055] It is worth noting that after S202-S203 are executed, S209 can be executed together with S204-S208.

[0056] In this embodiment, by comparing the pixel intensities of each two adjacent frames in the continuous frame information, information that can reflect the overall situation of traffic flow is determined, which can accurately determine the current overall traffic flow situation and provide a basis for subsequently determining the traffic congestion status.

[0057] S204: Preprocess the continuous frame information and the obtained ambient audio signal.

[0058] Specifically, preprocessing methods for continuous frame information include image denoising, dynamic exposure correction, and light balancing to enhance image quality in harsh environments such as low light, strong light, rain, and snow. These preprocessing methods can also be applied to the continuous frames in S202-S203. Furthermore, preprocessing methods for ambient audio signals include denoising, echo suppression, and bandpass filtering (e.g., 300 Hz-3000 Hz) to improve audio signal clarity and filter out irrelevant background noise.

[0059] In this embodiment, preprocessing of continuous frame information, particularly processing the image's light intensity, not only denoises the image and ensures the accuracy of continuous frame information, but also improves accuracy in poor weather conditions. Furthermore, preprocessing of ambient audio signals not only denoises the audio signal and improves its clarity, but also retains audio in required frequency bands and filters out audio in unneeded frequency bands, ensuring audio signal accuracy.

[0060] S205 , performing target recognition on the pre-processed continuous frame information to identify at least one preceding vehicle, and performing feature extraction on the pre-processed ambient audio signal to obtain a preceding vehicle status sound.

[0061] The preceding vehicle status sound is the sound characteristic of the preceding vehicle status that can be initially reflected after feature extraction of the ambient audio signal.

[0062] Specifically, target recognition is used to identify at least one preceding vehicle from pre-processed continuous frames. For example, the first vehicle in front of the vehicle, or even the second vehicle in front of the vehicle, etc. Furthermore, feature extraction is performed on the ambient audio signal to extract audio sounds with the desired characteristics as the preceding vehicle status sounds.

[0063] Optionally, target detection and target tracking methods can be used for target recognition, for example, a target detection model based on the YOLO detection model, a single-shot multi-frame detector (SSD), or a self-attention mechanism deep learning structure. Furthermore, the ambient audio signal can be framed first, and each frame of audio can be windowed using the short-time Fourier transform (STFT) and then subjected to a fast Fourier transform to convert it from the time domain to the frequency domain. Mel-frequency cepstral coefficients (MFCCs) are then used to extract features to obtain the preceding vehicle status sound. Finally, the obtained preceding vehicle status sound can be normalized, differentially characterized, and fused.

[0064] Optionally, in this embodiment, target tracking can be performed using a Kalman filter or a sorting SORT algorithm. Furthermore, semantic segmentation can be performed on continuous frame information to identify elements such as lane lines, vehicles, pedestrians, and traffic lights, and further determine whether the lane ahead is clear.

[0065] S206 . For each preceding vehicle, determine a first motion state based on pixel information of the preceding vehicle in continuous frame information; and determine a second motion state based on the indicator light of the preceding vehicle.

[0066] Specifically, for each preceding vehicle, the pixel information in consecutive frames can be used to determine whether the preceding vehicle has moved, thereby obtaining the preceding vehicle's first motion state. Furthermore, the preceding vehicle's indicator lights can be used to predict the preceding vehicle's likely next move, i.e., its second motion state.

[0067] Exemplarily, the pixel information includes a pixel range and a pixel position; and determining the first motion state includes:

[0068] (1) For each frame of continuous frame information, determine the pixel range and pixel position of the preceding vehicle in the frame of continuous frame information.

[0069] Specifically, after selecting a preceding vehicle as a target, the preceding vehicle can be determined based on information from successive frames to determine whether it has moved, and the distance it has moved if it has. For example, the pixel range occupied by the preceding vehicle in the first frame can be obtained, and the pixel positions of that pixel range within the entire first frame can be determined. Furthermore, the pixel ranges and pixel positions of the preceding vehicle in the second, third, and subsequent frames can be obtained.

[0070] (2) Determine the first motion state based on the pixel range and pixel position of the preceding vehicle in each frame of information.

[0071] Specifically, it is possible to determine whether the range of the leading vehicle has changed based on the pixel range of the leading vehicle in the previous frame and the pixel range in the next frame, and to determine whether the position of the leading vehicle has changed based on the pixel position of the leading vehicle in the previous frame and the pixel position in the next frame.

[0072] For example, the range of the leading vehicle in each frame of pixels is first frame-selected to obtain a pixel range. Based on the pixel range, the center coordinates of the vehicle can be determined. At this time, based on the center coordinates in the two frames of information of the leading vehicle, the lateral displacement vector and longitudinal displacement vector of the leading vehicle can be calculated. In addition, some feature points of the leading vehicle, such as the edges, corners, and locations with obvious texture changes of the leading vehicle, can be used as feature points, and the pixel positions of the feature points can be determined. Based on the pixel positions of the feature points, the average displacement is determined. Finally, the degree of change in the pixel range is determined based on the pixel range of the leading vehicle in the two adjacent frames, and the average displacement is determined based on the pixel positions of the feature points of the leading vehicle in the two adjacent frames. When the average displacement exceeds a certain threshold, it is determined that the leading vehicle has moved. Based on the degree of change in the pixel range and the calculated displacement of the leading vehicle, it can be determined between which frames the leading vehicle has moved, and based on the obtained average displacement, the actual object distance, and the focal length of the camera device, the first motion state of the leading vehicle can be calculated.

[0073] Specifically, the second motion state is determined according to the indicator light of the preceding vehicle, which may be:

[0074] Generally, vehicles have indicator lights such as left turn signals, right turn signals, brake lights, and start lights. Therefore, the motion state of the preceding vehicle can be predicted based on the indicator lights of the preceding vehicle captured in each frame and the changes in the indicator lights between the preceding and following frames.

[0075] For example, if the preceding vehicle activates its left or right turn signal in a particular frame, it can be predicted that the preceding vehicle is preparing to change lanes, and thus, it can be inferred that congestion in the lane the preceding vehicle is changing to has ended. Furthermore, if the preceding vehicle's brake lights go from on to off in multiple adjacent frames, it can be determined that the preceding vehicle is preparing to start, and thus, it can be inferred that lane congestion is about to end. Based on this, the preceding vehicle's indicator lights can provide the vehicle with motion prediction information, namely the second motion state. Therefore, detecting changes in the preceding vehicle's indicator lights can serve as an auxiliary judgment of the preceding vehicle's intention to start.

[0076] In this embodiment, the first motion state can be determined based on the position change of the front vehicle in the continuous frame information, and the second motion state can be determined based on the indicator signal light of the front vehicle. This realizes the ability to accurately and diversely determine the traffic jam state based on the situation of the front vehicle from the visualized information based on the continuous frame information of the front vehicle.

[0077] S207 , input the preceding vehicle status sound into a pre-trained preceding vehicle sound event classifier, determine the preceding vehicle sound event, and determine the preceding vehicle sound information based on the time domain characteristics of the preceding vehicle sound event.

[0078] Among them, the original model of the sound event classifier is an end-to-end deep neural network model. The sound event classifier is obtained after training using labeled audio samples. After inputting audio sounds, it can identify key sounds of ending traffic jams, such as: the engine starting sound of the vehicle in front (a low-frequency vibration sound), the friction sound of the tires when starting, the sound of airflow enhancement during vehicle acceleration (white noise will be amplified after the car starts), and continuous honking sounds, prompt sounds, and other sounds that remind you to start.

[0079] Specifically, by inputting the status sound of the preceding vehicle into the preceding vehicle sound event classifier, one or more preceding vehicle sound events can be obtained. At this time, the preceding vehicle sound information can be determined based on the time domain characteristics of the preceding vehicle sound event, such as the duration of the sound, frequency changes, sound signal strength and other characteristics.

[0080] Exemplarily, determining the preceding vehicle sound information includes:

[0081] (1) Obtaining a first weight index corresponding to the preceding vehicle sound event.

[0082] The first weight index is the weight value corresponding to different time domain features of different preceding vehicle sound events.

[0083] Specifically, a corresponding weight index may be set in advance for each preceding vehicle sound event, or a first weight index may be assigned to each sound event based on its distance, sound intensity, number of occurrences, and credibility.

[0084] (2) Determine the preceding vehicle sound information based on the sound duration event, frequency change, amplitude change, and the first weight index corresponding to the preceding vehicle sound event.

[0085] Among them, time domain features include sound duration, frequency changes, and amplitude changes.

[0086] Specifically, each sound event, the duration of the event, the frequency change of the event throughout the acquisition process, the amplitude change, and its corresponding first weight index can be determined as the preceding vehicle sound information. Therefore, when multiple sound events are acquired, multiple preceding vehicle sound information can be determined.

[0087] In this embodiment, based on the sound events of the preceding vehicle and their corresponding time domain characteristics, such as frequency changes, amplitude changes, etc., and their corresponding first weight indicators, each sound event can be aggregated into a preceding vehicle sound information to achieve information integration of each sound event, which is convenient for subsequent determination of the preceding vehicle sound status.

[0088] S208. Determine the motion state of the preceding vehicle based on the first motion state, the second motion state, and the traffic flow information, and determine the preceding vehicle sound state of the preceding vehicle based on the preceding vehicle sound information.

[0089] Specifically, determining the motion state of the preceding vehicle according to the first motion state, the second motion state, and traffic flow information includes:

[0090] (1) Determine the vehicle motion state for the first motion state, the second motion state, and the traffic flow information respectively, and obtain a first state corresponding to the first motion state, a second state corresponding to the second motion state, and a third state corresponding to the traffic flow information.

[0091] The first state, the second state, and the third state can each be one of the following three states: the vehicle is in motion, the vehicle is in a congested state, and the vehicle is in a semi-congested state. The semi-congested state means that the vehicle stops and moves, that is, although it is in a traffic jam, it can move forward bit by bit.

[0092] Specifically, the first motion state is the displacement information of the preceding vehicle. Therefore, when the displacement information of the preceding vehicle is obtained, the first state corresponding to the first motion state can be determined. For example, when the displacement information of the preceding vehicle is greater than a preset displacement, the first state is determined to be that the vehicle is in motion; if the displacement information is less than or equal to the preset displacement, or the displacement information is less than the minimum displacement threshold, the first state is determined to be that the vehicle is in a congested state. Similarly, the second motion state is the motion prediction information. If the motion prediction information is that the brake lights of the preceding vehicle are off, the second state can be determined to be that the vehicle is in motion; if the motion prediction information is that the brake lights of the preceding vehicle are on, the second state can be determined to be that the vehicle is in a congested state. Traffic flow information can reflect the overall state of traffic. Therefore, if the traffic flow information is that the traffic is in a flowing state, the third state can be determined to be that the vehicle is in motion; if the traffic flow information is that the traffic is in a non-flowing state, the third state can be determined to be that the vehicle is in a congested state.

[0093] (2) Determine whether the first state, the second state, and the third state are all the same; if so, execute (3); if not, execute (4).

[0094] Specifically, it is necessary to determine whether the three states are the same state. If the three states are the same state, for example, the vehicle is in motion, then (3) can be executed. If any one of the three states is different from the others, then (4) needs to be executed.

[0095] (3) If the first state, the second state, and the third state are all the same, the consistent state is directly determined as the movement state of the preceding vehicle.

[0096] Specifically, if the three states are all the same, then the state can be directly used as the preceding vehicle motion state. For example, if the three states are all that the vehicle is in motion, then the preceding vehicle motion state is the vehicle in motion state.

[0097] (4) If at least one of the first state, the second state, and the third state is different, the dynamic weight of the first motion state, the dynamic weight of the second motion state, and the dynamic weight of the traffic flow information are determined based on the number of identical states, the current time, the time for meeting the congestion warning speed, and the second weight index; based on the dynamic weight of the first motion state, the dynamic weight of the second motion state, and the dynamic weight of the traffic flow information, the motion state of the preceding vehicle is determined from the first state, the second state, and the third state.

[0098] The second weight index includes a preset basic weight of the first motion state, a preset basic weight of the second motion state, and a preset basic weight of traffic flow information. The current time is the time when each state is obtained.

[0099] Specifically, if at least one of the first state, the second state and the third state is different, that is, the two states are consistent, and the other state is different from these two states; or if the three states are all different, then the dynamic weight of the first motion state, the dynamic weight of the second motion state and the dynamic weight of the traffic flow information can be determined based on the number of identical states, the current time, the time to meet the congestion warning speed and the second weight index.

[0100] For example, if the first state is that the vehicle is in a semi-congested state, the second state is that the vehicle is in motion, and the third state is that the vehicle is in a congested state, then the three states are all different. And if the first state is that the vehicle is in motion, the second state is that the vehicle is in a semi-congested state, and the third state is that the vehicle is in motion, then two of the states are the same. At this time, the congestion time can be determined based on the current time of each state and the time when the congestion warning speed is met; and the pre-set weights of the first motion state, the second motion state, and the traffic flow information are first determined based on the second weight index. The weight of each state is adjusted using the number of identical states and the congestion time to determine the dynamic weight of the first motion state, the dynamic weight of the second motion state, and the dynamic weight of the traffic flow information.

[0101] For example, if the first state is semi-congested, the second state is moving, and the third state is moving; the number of vehicles in motion is 2, and the number of vehicles in semi-congested is 1; the weight of the first moving state is a, the current time is time t1 corresponding to the last frame in the continuous frame information, the weight of the second moving state is b, the current time is time t2 corresponding to the eighth frame in the continuous frame information, the weight of the traffic flow information is c, the current time is time t1 corresponding to the last frame in the continuous frame information, and the time when the congestion warning speed is met is t3. Among them, time t3 is earlier than time t2, and time t2 is earlier than time t1. In this case, the dynamic weight of the first moving state can be calculated as a*(t3-t1)*1, the dynamic weight of the second moving state is b*(t3-t2)*2, and the dynamic weight of the third moving state is c*(t3-t1)*2. Since the second moving state is determined based on the indicator light of the preceding vehicle, it may only appear in certain frames in the continuous frame. In actual situations, for example, the second motion state corresponds to the second state, and the second state can also be comprehensively judged based on the indicator light conditions of multiple preceding vehicles determined from continuous frames, or the second state can be judged based on the indicator light conditions in the frame closest to the current real-time time.

[0102] Furthermore, based on the dynamic weight of the first motion state, the dynamic weight of the second motion state, and the dynamic weight of the traffic flow information, the motion state of the preceding vehicle is determined from the first state, the second state, and the third state. Specifically, based on these three dynamic weights, the state corresponding to the largest dynamic weight is selected as the motion state of the preceding vehicle. For example, if the dynamic weight of the traffic flow information is the largest among the three dynamic weights, then the third state can be determined to be the motion state of the preceding vehicle.

[0103] In addition, the sound status of the preceding vehicle is determined based on the sound information of the preceding vehicle, that is, the obtained sound event, the duration of the event, the frequency change of the event during the entire acquisition process, the amplitude change and the corresponding first weight index are determined as the sound information of the preceding vehicle. The weight value corresponding to each sound can be determined based on each sound event and its corresponding first weight index, and the movement status of the preceding vehicle can be determined based on the size of the weight value and the current traffic jam status indicated by each preceding vehicle sound information.

[0104] For example, the preceding vehicle sound information includes the number of occurrences of the sound event and the first weight index. The same principle can also be used in the actual weight calculation, which will not be repeated here. The preceding vehicle sound information includes the first information, the second information and the third information. Among them, the sound event of the first information is the engine start-up sound, the corresponding first weight index is d, and the number of sound event occurrences is 6 times; the second information is the airflow enhancement sound of vehicle acceleration, the corresponding first weight index is e, and the number of sound occurrences is 3 times; the third information is the vehicle prompt sound, the corresponding first weight index is f, and the number of sound occurrences is 10 times; among them, d is greater than e and greater than f. At this time, the total weight sum corresponding to the preceding vehicle sound information can be obtained: 6d+3e+10f. If the total weight sum corresponding to the preceding vehicle sound information exceeds the preset total weight, it can be determined that the preceding vehicle sound state is the congestion end state.

[0105] In this embodiment, the corresponding traffic jam states are determined according to the first motion state, the second motion state, and the traffic flow information, and it is determined whether the three traffic jam states are the same. If they are the same, it can be determined that the traffic jam state is the motion state of the preceding vehicle, thereby achieving rapid and accurate determination of the motion state of the preceding vehicle; or, if at least one state is different from the other states, the dynamic weights corresponding to the first motion state, the second motion state, and the traffic flow information are determined according to different factors, and the final motion state of the preceding vehicle is determined according to the dynamic weights. While achieving diversified determination of the motion state of the preceding vehicle, it can also improve the adaptive ability of judging the motion state of the preceding vehicle in different scenarios according to the dynamic weights determined by different factors, improve the decision-making quality in different scenarios, and reduce the influence of human subjective factors on the results.

[0106] S209: Determine and display a traffic jam state based on the movement state and sound state of the preceding vehicle.

[0107] Specifically, a multimodal fusion of the preceding vehicle's motion state and the preceding vehicle's sound state can be directly performed to determine the traffic jam state. For example, a decision-making layer fusion or weighted voting method can be used to determine the preceding vehicle's motion state and the preceding vehicle's sound state to determine whether the traffic jam state exists.

[0108] Optionally, the first motion state, the second motion state, the traffic flow information and each sound event obtained above can be directly judged using a decision-making layer fusion method to determine the current traffic jam state.

[0109] And, after the traffic jam status is determined, the traffic jam status can be displayed on a visualization screen of the vehicle.

[0110] In this embodiment, the traffic jam state is determined based on the movement state and sound state of the preceding vehicle, which can achieve a comprehensive judgment of the traffic jam state from both visual and auditory aspects, reducing the problem of abnormal judgment caused by determining the traffic jam state only by vision or only by hearing, and achieving a comprehensive and accurate judgment of the traffic jam state.

[0111] Optionally, after determining and displaying the traffic jam status, the method further includes:

[0112] (1) If the traffic jam status is that the traffic jam is ended, a traffic jam end reminder signal is sent to the central control screen, so that the central control screen broadcasts the status of the traffic jam ending.

[0113] Specifically, if the traffic jam status indicates that the vehicle has ended, a traffic jam reminder signal is sent to the central control screen, which then directly announces the end of the traffic jam to the driver. This allows the driver to quickly be notified of the end of the traffic jam and prompt them to start as soon as possible, effectively improving driving response speed and traffic efficiency. Optionally, when the traffic jam status is determined to be ended, the camera and audio equipment functions activated in response to the traffic jam status can be disabled to reduce vehicle energy consumption.

[0114] (2) If the traffic jam status is traffic jam, return to the step of obtaining continuous frame information captured by the vehicle's camera device and the ambient audio signal collected by the vehicle's audio device.

[0115] Specifically, if the traffic jam state is still a traffic jam, the process may return to executing the step of obtaining continuous frame information and the ambient audio signal in S201 to achieve continuous monitoring before the traffic jam ends.

[0116] In this embodiment, if the traffic jam status is "End of Traffic Jam," the central control screen proactively displays the end of the traffic jam to the user. This prevents drivers from missing the end of the traffic jam due to distraction during long traffic jams, leading to problems such as slow starts and improving the driver's experience. Alternatively, if the traffic jam status is "Congestion," the camera and audio equipment continue to collect relevant information to provide the driver with continuous traffic jam status monitoring.

[0117] Figure 3 This is a schematic diagram of the structure of a traffic jam status determination device provided by an embodiment of the present invention. Figure 3 As shown, the device is applied to a vehicle and includes:

[0118] An acquisition module 301 is configured to acquire continuous frame information captured by a camera device of the vehicle and an ambient audio signal collected by an audio device of the vehicle when the vehicle's driving speed meets a congestion warning speed;

[0119] a processing module 302 for determining a first motion state and a second motion state of a preceding vehicle based on the continuous frame information, and determining preceding vehicle sound information based on the ambient audio signal, wherein the preceding vehicle includes at least one vehicle in front of the vehicle;

[0120] a determination module 303 for determining a preceding vehicle motion state of the preceding vehicle based on the first motion state and the second motion state, and determining a preceding vehicle sound state of the preceding vehicle based on the preceding vehicle sound information;

[0121] The display module 304 is used to determine and display the traffic jam status according to the movement status and sound status of the preceding vehicle.

[0122] Optionally, the determining module 303 is specifically configured to:

[0123] Preprocess the continuous frame information and the obtained environmental audio signal; perform target recognition on the preprocessed continuous frame information to identify at least one leading vehicle, and perform feature extraction on the preprocessed environmental audio signal to obtain the leading vehicle status sound; for each leading vehicle, determine the first motion state based on the pixel information of the leading vehicle in the continuous frame information; and determine the second motion state based on the indicator light of the leading vehicle; input the leading vehicle status sound into a pre-trained leading vehicle sound event classifier to determine the leading vehicle sound event, and determine the leading vehicle sound information based on the time domain characteristics of the leading vehicle sound event.

[0124] Optionally, the pixel information includes a pixel range and a pixel position; and the first motion state is determined based on the pixel information of the preceding vehicle in the continuous frame information. The determining module 303 is specifically configured to:

[0125] For each frame of continuous frame information, the pixel range and pixel position of the preceding vehicle in the frame are determined; and the first motion state is determined according to the pixel range and pixel position of the preceding vehicle in each frame of continuous frame information.

[0126] Optionally, the time domain features include sound duration, frequency change, and amplitude change; based on the time domain features of the preceding vehicle sound event, the preceding vehicle sound information is determined. The determination module 303 is specifically configured to:

[0127] Obtain the first weight index corresponding to the preceding vehicle sound event, wherein the first weight index is the weight value corresponding to different time domain characteristics of different preceding vehicle sound events; determine the preceding vehicle sound information based on the sound duration event, frequency change, amplitude change, and the first weight index corresponding to the preceding vehicle sound event.

[0128] Optionally, after acquiring the continuous frame information captured by the vehicle's camera device, the processing module 302 is further configured to:

[0129] Calculate the instantaneous velocity vector of the pixel point based on the degree of pixel intensity change between two adjacent frames in the continuous frame information; determine the traffic flow information based on the vector direction and vector components of the instantaneous velocity vector;

[0130] The determination module 303 is specifically configured to:

[0131] The motion state of the preceding vehicle is determined according to the first motion state, the second motion state, and traffic flow information.

[0132] Optionally, the determining module 303 is specifically configured to:

[0133] The vehicle motion state is determined for the first motion state, the second motion state, and the traffic flow information respectively to obtain a first state corresponding to the first motion state, a second state corresponding to the second motion state, and a third state corresponding to the traffic flow information; it is determined whether the first state, the second state, and the third state are all the same; if the first state, the second state, and the third state are all the same, the consistent state is directly determined as the motion state of the preceding vehicle; if at least one of the first state, the second state, and the third state is different, the dynamic weight of the first motion state, the dynamic weight of the second motion state, and the dynamic weight of the traffic flow information are determined according to the number of identical states, the current time, the time for meeting the congestion warning speed, and the second weight index, wherein the second weight index includes a pre-set basic weight of the first motion state, a basic weight of the second motion state, and a basic weight of the traffic flow information; according to the dynamic weight of the first motion state, the dynamic weight of the second motion state, and the dynamic weight of the traffic flow information, the motion state of the preceding vehicle is determined from the first state, the second state, and the third state.

[0134] Optionally, after determining and displaying the traffic jam state based on the movement state and the sound state of the preceding vehicle, the display module 304 is further configured to:

[0135] If the traffic jam status is that the traffic jam is over, a traffic jam end reminder signal is sent to the central control screen so that the central control screen broadcasts the status of the traffic jam end; if the traffic jam status is that the traffic jam is over, the process returns to execute the steps of obtaining the continuous frame information captured by the vehicle's camera equipment and the ambient audio signal collected by the vehicle's audio equipment.

[0136] The traffic jam status determination device provided in the embodiment of the present invention can execute the traffic jam status determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0137] Figure 4 A schematic diagram of the structure of an electronic device in a vehicle provided for an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0138] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0139] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0140] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the traffic jam status method.

[0141] In some embodiments, the traffic jam state method can be implemented as a computer program that is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the traffic jam state method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the traffic jam state method in any other appropriate manner (e.g., by means of firmware).

[0142] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0144] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0147] A computing system may include a client and a server. The client and server are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0148] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for determining a traffic jam state as provided in any embodiment of the present invention.

[0149] The computer program product may be implemented by writing computer program code for performing the operations of the present invention in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0150] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0151] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for determining a traffic jam state, characterized in that: Applied to a vehicle; the method comprises: When it is monitored that the driving speed of the vehicle meets the congestion warning speed, obtaining continuous frame information captured by the vehicle's camera device and environmental audio signals collected by the vehicle's audio device; Determining a first motion state and a second motion state of a preceding vehicle according to the continuous frame information, and determining preceding vehicle sound information according to the ambient audio signal, wherein the preceding vehicle includes at least one vehicle in front of the vehicle; determining a preceding vehicle motion state of the preceding vehicle according to the first motion state and the second motion state, and determining a preceding vehicle sound state of the preceding vehicle according to the preceding vehicle sound information; The traffic jam state is determined and displayed according to the movement state of the preceding vehicle and the sound state of the preceding vehicle.

2. The method for determining a traffic jam state according to claim 1, wherein: The determining the first motion state and the second motion state of the preceding vehicle according to the continuous frame information, and determining the preceding vehicle sound information according to the ambient audio signal, includes: Preprocessing the continuous frame information and the obtained ambient audio signal; Performing target recognition on the pre-processed continuous frame information to identify at least one preceding vehicle, and performing feature extraction on the pre-processed ambient audio signal to obtain a preceding vehicle status sound; For each of the preceding vehicles, determining the first motion state based on pixel information of the preceding vehicle in the continuous frame information; and determining the second motion state based on the indicator light of the preceding vehicle; The preceding vehicle state sound is input into a pre-trained preceding vehicle sound event classifier to determine a preceding vehicle sound event, and the preceding vehicle sound information is determined based on the time domain features of the preceding vehicle sound event.

3. The method for determining a traffic jam state according to claim 2, wherein: The pixel information includes pixel range and pixel position; The determining the first motion state according to pixel information of the preceding vehicle in the continuous frame information includes: For each frame of the continuous frame information, determining the pixel range and the pixel position of the preceding vehicle in the frame; The first motion state is determined according to the pixel range and the pixel position of the preceding vehicle in each frame of information.

4. The method for determining a traffic jam state according to claim 2, wherein: The time domain features include sound duration, frequency change, and amplitude change; and determining the preceding vehicle sound information based on the time domain features of the preceding vehicle sound event includes: Obtaining a first weight index corresponding to the preceding vehicle sound event, wherein the first weight index is a weight value corresponding to different time domain features of different preceding vehicle sound events; The preceding vehicle sound information is determined according to the sound duration event, the frequency change, the amplitude change, and a first weight index corresponding to the preceding vehicle sound event.

5. The method for determining a traffic jam state according to claim 1, wherein: After acquiring the continuous frame information captured by the camera device of the vehicle, the method further includes: Calculating the instantaneous velocity vector of the pixel point according to the degree of change of pixel intensity between each two adjacent frames in the continuous frame information; determining traffic flow information based on the vector direction and vector components of the instantaneous velocity vector; The determining the preceding vehicle motion state of the preceding vehicle according to the first motion state and the second motion state includes: The motion state of the preceding vehicle is determined according to the first motion state, the second motion state, and the traffic flow information.

6. The method for determining a traffic jam state according to claim 5, characterized in that: The determining the motion state of the preceding vehicle according to the first motion state, the second motion state, and the traffic flow information includes: Performing vehicle motion state determination on the first motion state, the second motion state, and the traffic flow information respectively to obtain a first state corresponding to the first motion state, a second state corresponding to the second motion state, and a third state corresponding to the traffic flow information; determining whether the first state, the second state, and the third state are all the same; If the first state, the second state, and the third state are all the same, directly determining the consistent state as the preceding vehicle motion state; If at least one of the first state, the second state and the third state is different, the dynamic weight of the first motion state, the dynamic weight of the second motion state and the dynamic weight of the traffic flow information are determined according to the number of identical states, the current time, the time to meet the congestion warning speed and the second weight index, wherein the second weight index includes the pre-set basic weight of the first motion state, the basic weight of the second motion state and the basic weight of the traffic flow information; according to the dynamic weight of the first motion state, the dynamic weight of the second motion state and the dynamic weight of the traffic flow information, the motion state of the preceding vehicle is determined from the first state, the second state and the third state.

7. The method for determining a traffic jam state according to claim 1, wherein: After determining and displaying the traffic jam state according to the movement state of the preceding vehicle and the sound state of the preceding vehicle, the method further includes: If the traffic jam status is that the traffic jam is over, a traffic jam end reminder signal is sent to the central control screen so that the central control screen announces the traffic jam end status; If the traffic jam state is a traffic jam, the process returns to executing the step of acquiring continuous frame information captured by the camera device of the vehicle and the ambient audio signal collected by the audio device of the vehicle.

8. A traffic jam status determination device, characterized in that: Applicable to a vehicle; the device comprises: an acquisition module, configured to acquire continuous frame information captured by a camera device of the vehicle and an ambient audio signal collected by an audio device of the vehicle when the vehicle's driving speed is detected to meet a congestion warning speed; a processing module, configured to determine a first motion state and a second motion state of a preceding vehicle based on the continuous frame information, and determine preceding vehicle sound information based on the ambient audio signal, wherein the preceding vehicle includes at least one vehicle in front of the vehicle; a determining module, configured to determine a preceding vehicle motion state of the preceding vehicle according to the first motion state and the second motion state, and to determine a preceding vehicle sound state of the preceding vehicle according to the preceding vehicle sound information; The display module is used to determine and display the traffic jam status according to the movement status of the preceding vehicle and the sound status of the preceding vehicle.

9. A vehicle, characterized in that: include: electronic devices; The electronic device includes one or more processors; The memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the traffic jam state determination method according to any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the traffic jam state determination method according to any one of claims 1 to 7 is implemented.

Images