Suspension device control method, device, equipment and storage medium

Abstract

The present invention provides a suspension device control method, device, equipment and storage medium, including: obtaining environmental image data and audio data; performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result; based on the image recognition result and the audio recognition result, controlling the suspension device of the target vehicle to open. The present invention accurately judges the environment of the vehicle by combining image recognition and audio recognition, thereby automatically triggering the suspension device of the target vehicle and reducing the safety risks of the driver and passengers.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a suspension device control method, device, equipment and storage medium. Background Art

[0002] Vehicles have become one of the important means of transportation for people to travel. In the process of daily driving, you may encounter mountain torrents, rainstorms, or traffic accidents that wash down rivers and lakes, causing your vehicle to be submerged. When the vehicle is submerged, due to the high external water pressure, the doors and windows are often difficult to open normally. The current coping methods mainly include: window smashing method. However, the window smashing method has high requirements on the quality of the driver and passengers, which poses a safety hazard to the driver and passengers. Summary of the invention

[0003] The present invention provides a suspension device control method, device, equipment and storage medium, aiming to solve the technical problem of potential safety hazards to drivers and passengers when a vehicle is submerged in water.

[0004] The present invention provides a suspension device control method, comprising:

[0005] Acquire environmental image data and audio data;

[0006] Performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result;

[0007] Based on the image recognition result and the audio recognition result, a suspension device of the target vehicle is controlled to be opened.

[0008] According to a suspension device control method provided by the present invention, the method of controlling the suspension device of a target vehicle to be opened based on the image recognition result and the audio recognition result includes:

[0009] Determining vehicle environment information corresponding to the target vehicle based on the image recognition result and the audio recognition result;

[0010] If the vehicle environment information is water-wading environment information, the suspension device is triggered to open.

[0011] According to a suspension device control method provided by the present invention, after controlling to open the suspension device of the target vehicle based on the image recognition result and the audio recognition result, the method further comprises:

[0012] Acquiring in-vehicle image data of the target vehicle within a preset time;

[0013] Performing image recognition on each of the in-vehicle image data to obtain a state recognition result, wherein the state recognition result is used to characterize the state of the driver and passenger in the target vehicle;

[0014] Determine that the status recognition result belongs to a preset recognition result, and generate rescue prompt information.

[0015] According to a suspension device control method provided by the present invention, the image recognition is performed on the in-vehicle image data to obtain a state recognition result, including:

[0016] Based on the posture detection model, performing posture detection on each of the in-vehicle image data to obtain a posture detection result;

[0017] Performing color recognition on the eye image area in each of the in-vehicle image data to obtain an eye recognition result;

[0018] The state recognition result is determined based on the posture detection result and the eye recognition result.

[0019] According to a suspension device control method provided by the present invention, determining the vehicle environment information corresponding to the target vehicle based on the image recognition result and the audio recognition result includes:

[0020] If the image recognition result is a wading recognition result, and the audio recognition result is a wading recognition result, determining that the vehicle environment information is wading environment information;

[0021] If the image recognition result is not a wading recognition result, or the audio recognition result is not a wading recognition result, it is determined that the vehicle environment information is non-wading environment information.

[0022] According to a suspension device control method provided by the present invention, the image recognition is performed on the environmental image data to obtain an image recognition result; and the audio recognition is performed on the audio data to obtain an audio recognition result, including:

[0023] Inputting the environmental image data into an image recognition model to obtain an image recognition result output by the image recognition model;

[0024] Inputting the audio data into an audio recognition model to obtain an audio recognition result output by the audio recognition model;

[0025] The image recognition model is trained based on a number of image samples and sample labels corresponding to each of the image samples;

[0026] The audio recognition model is trained based on each audio data sample and the audio category label corresponding to each audio data sample.

[0027] According to a suspension device control method provided by the present invention, the step of acquiring environmental image data and audio data comprises:

[0028] In response to a wading signal from the wading sensor, controlling the image acquisition device and the audio acquisition device to be in an on state;

[0029] Acquire the environmental image data captured by the image acquisition device and the audio data collected by the audio acquisition device.

[0030] The present invention also provides a suspension device control device, comprising:

[0031] An acquisition module, used to acquire environmental image data and audio data;

[0032] A recognition module, configured to perform image recognition on the environmental image data to obtain an image recognition result; and perform audio recognition on the audio data to obtain an audio recognition result;

[0033] A control module is used to control the opening of a suspension device of a target vehicle based on the image recognition result and the audio recognition result.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned suspension device control methods is implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the suspension device control method described in any one of the above methods is implemented.

[0036] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned suspension device control methods.

[0037] The suspension device control method, device, equipment and storage medium provided by the present invention recognize the environmental image data and audio data of the target vehicle, and accurately judge whether the target vehicle is currently in a wading environment by combining the image recognition results and the audio recognition results, so as to automatically control the opening of the suspension device of the target vehicle and improve the survival rate of the driver and passengers after an accident. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 This is one of the flow charts of the suspension device control method provided by the present invention;

[0040] Figure 2 This is the second flow chart of the suspension device control method provided by the present invention;

[0041] Figure 3 It is a timing diagram of the suspension device control system provided by the present invention;

[0042] Figure 4 It is a structural schematic diagram of the suspension device control device provided by the present invention;

[0043] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The terms used in one or more embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present invention. The singular forms of "a", "said" and "the" used in one or more embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.

[0046] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present invention, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present invention, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when...".

[0047] Vehicles have become one of the important means of transportation for people to travel. In the process of daily driving, you may encounter mountain torrents, rainstorms, or traffic accidents that wash down rivers and lakes, causing your vehicle to be submerged. When the vehicle is submerged, due to the high external water pressure, the doors and windows are often difficult to open normally. The current coping methods mainly include: window smashing method. However, the window smashing method has high requirements on the quality of the passengers, which poses a safety hazard to the drivers and passengers.

[0048] In view of the above problems, the present invention proposes the following embodiments. Figure 1 FIG. 1 is one of the flow charts of the suspension device control method provided by the present invention. Figure 1 As shown, the suspension device control method includes:

[0049] Step 11, obtaining environmental image data and audio data;

[0050] It should be noted that the environmental image data represents the image data of the target vehicle's current environment. Since the grayscale values ​​in the environmental images when the vehicle is in a wading environment and when the vehicle is driving normally are different, the vehicle's current environmental image can be collected to determine whether the vehicle is in a wading environment. Optionally, image acquisition devices can be installed at the left and right rearview mirrors of the target vehicle. The image acquisition devices include cameras and other devices. In addition, image acquisition devices can also be installed in front of and / or behind the vehicle to comprehensively collect the environmental image data of the vehicle's current location captured by each image acquisition device.

[0051] In addition, since when a vehicle drives into deeper water or a river, it will hit and make a loud water sound, audio data can also be collected to determine whether the target vehicle is in a wading environment. Optionally, the audio data is collected by an audio detection device pre-installed on the outside of the target vehicle.

[0052] Specifically, in one embodiment, when a wading signal of a wading sensor is detected, only the image data captured by the image acquisition device or the audio data collected by the audio acquisition device may be obtained. In order to improve the accuracy of determining the environment in which the vehicle is located, in another embodiment, the wading signal sent by the wading sensor is monitored in real time, so that when the wading signal of the wading sensor is detected, the image acquisition device and the audio acquisition device are controlled to be in an open state, and then the environmental image data collected by each image acquisition device and the audio data collected by the audio detection device are obtained, so that the environment in which the vehicle is located can be accurately determined based on the wading sensor, the image data and the audio data. It should be noted that the wading sensor is a material-type sensor, so that the target vehicle can normally emit a wading signal even after being submerged in water. In addition, during the driving process of the vehicle, when the target vehicle is in a wading environment with a water level below the hood, the driver can usually open the door or window normally. However, when the target vehicle is in a wading environment with a water level above the hood, the external water pressure is relatively large, and the doors and windows are often difficult to open normally. Therefore, preferably, the wading sensor is installed inside the hood of the target vehicle, so that when the water level is submerged to the hood position, the wading sensor can be triggered to emit a wading signal. In another embodiment, in order to prevent the wading sensor inside the front hood of the vehicle from malfunctioning, in addition to installing a wading sensor inside the front hood of the vehicle, a wading sensor can be installed on the left, right and / or rear of the vehicle according to the height of the hood, so that the wading signal of the target vehicle can be accurately detected.

[0053] Step 12, performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result;

[0054] It should be noted that the image recognition result and the audio recognition result represent the recognition result of whether the target vehicle is in a wading environment. Specifically, when performing image recognition on environmental image data, the environmental image data can be input into an image recognition model to determine the image recognition result according to the result output by the image recognition model, wherein the image recognition model is trained based on each image sample and the sample label corresponding to each image sample. Additionally, the audio data is input into an audio recognition model to determine the audio recognition result according to the result output by the audio recognition model, wherein the audio recognition model is trained based on each audio data sample and the audio category label corresponding to each audio data sample.

[0055] Step 13: Based on the image recognition result and the audio recognition result, control the suspension device of the target vehicle to open.

[0056] It should be noted that the target vehicle is equipped with a suspension device, which can allow the target vehicle to float on the water surface when the suspension device is turned on.

[0057] Specifically, based on the image recognition result and the audio recognition result, it is determined whether the target vehicle is in a wading environment. Optionally, if the image recognition result is a recognition result that the vehicle is in a wading environment, and the audio recognition result is a recognition result that the vehicle is in a wading environment, then the target vehicle is determined to be in a wading environment, and then the suspension device of the target vehicle is triggered to open, so that the target vehicle is suspended on the water surface to ensure the life safety of the driving personnel. In addition, if the image recognition result is a recognition result that the vehicle is not in a wading environment, or the audio recognition result is a recognition result that the vehicle is not in a wading environment, then it is determined that the target vehicle is not currently in a wading environment, and water droplets may splash onto the vehicle hood, erroneously triggering the wading sensor. At this time, there is no need to trigger the suspension device of the target vehicle to open.

[0058] The embodiment of the present invention adopts the above scheme, that is, obtaining environmental image data and audio data; performing image recognition on the environmental image data to obtain image recognition results; and performing audio recognition on the audio data to obtain audio recognition results; and controlling the suspension device of the target vehicle to open based on the image recognition results and the audio recognition results. The environmental image data and audio data of the target vehicle are recognized, so as to accurately determine whether the target vehicle is currently in a wading environment in combination with the image recognition results and the audio recognition results, thereby automatically controlling the suspension device of the target vehicle to open, thereby improving the survival rate of the driver and passengers after an accident.

[0059] Figure 2 This is the second flow chart of the suspension device control method provided by the present invention. Figure 2 As shown, in one embodiment of the present invention, after controlling to open the suspension device of the target vehicle based on the image recognition result and the audio recognition result, the method further includes:

[0060] Step 21, obtaining the in-vehicle image data of the target vehicle within a preset time;

[0061] Step 22, performing image recognition on each of the in-vehicle image data to obtain a state recognition result, wherein the state recognition result is used to characterize the state of the driver and passenger in the target vehicle;

[0062] Step 23, determining that the state recognition result belongs to a preset recognition result, and generating rescue prompt information.

[0063] It should be noted that the in-vehicle image data is collected by using a camera installed inside the vehicle. Optionally, when the suspension device is triggered to open, the camera inside the vehicle is controlled to be in an on state, thereby starting to collect in-vehicle image data of the target vehicle. Furthermore, the preset recognition result indicates a recognition result that the driver or passenger is in a coma.

[0064] In one embodiment, each in-vehicle image data within a predetermined time period may be collected, wherein the predetermined time period may be set according to actual conditions and is not specifically limited herein, and then posture detection is performed on the target person in each in-vehicle image data to obtain a posture detection result corresponding to each in-vehicle image data, thereby determining a state recognition result corresponding to the target person based on the posture detection result corresponding to each in-vehicle image data, and further, if the posture of the target person in the posture detection result does not change, then it is determined that the state recognition result belongs to a preset recognition result, that is, the driver and occupant of the target vehicle is in a coma, and then a rescue prompt information is generated to issue a rescue prompt to the outside world, for example, the rescue prompt information is sent to a relevant rescue department.

[0065] In another embodiment, it should be noted that there are two colors of the white and black of the eyeball in the open eyes state, and only the eyelids in the closed eyes state. Therefore, the image area in the eye in-vehicle image data can be color recognized to obtain a state recognition result. Specifically, each in-vehicle image data within a predetermined time is intercepted to obtain an eye image area, and then each eye image area is color recognized to obtain an eye recognition result. The state recognition result is determined based on the eye recognition results corresponding to each of the eye image areas. If all eye recognition results do not have two colors of the white and black of the eyeball, it is determined that the driver and passenger may have been in a state of closing their eyes for a long time. At this time, it can be determined that the driver and passenger are in a state of closing their eyes for a long time, that is, it is determined that the state recognition result corresponding to the driver and passenger is that the driver and passenger is in a coma. Further, a rescue prompt information is generated to send the rescue prompt information to the outside world.

[0066] In addition, in another embodiment, taking into account that the camera in the vehicle may malfunction or have blind spots, and it is impossible to capture an image containing the driver, in one embodiment, a heart rate detection device and a breathing detection device can be used to detect the heart rate and breathing rate of the driver and occupant, and determine the state recognition result based on the heart rate and breathing rate. Optionally, the heart rate detection device and the breathing detection device are installed on the side wall of the seat belt close to the heart of the chest. If it is detected that the heart rate and the breathing rate fluctuate violently in a short period of time, it is determined that the state recognition result belongs to the preset recognition result, that is, the driver and occupant is in a coma, and then a rescue prompt information is generated to send the rescue prompt information to the outside world.

[0067] In addition, in order to improve the accuracy of the detection status recognition results, the above three situations can be combined, that is, if the posture detection results, eye recognition results, and the status recognition results corresponding to the heart rate and breathing rate respectively belong to the preset recognition results, then a rescue prompt information is generated; if one or more of the status recognition results corresponding to the posture detection results, eye recognition results, and the heart rate and breathing rate respectively do not belong to the preset recognition results, then there is no need to generate a rescue prompt information.

[0068] The embodiment of the present invention adopts the above scheme, that is, obtaining the in-vehicle image data; performing image recognition on the in-vehicle image data to obtain the state recognition result; if the state recognition result is that the driver and occupant are in a coma, a rescue prompt message is issued. When the target vehicle is in a wading environment, the in-vehicle camera is turned on to collect the in-vehicle image data to detect in real time whether the driver and occupant are in a coma, so as to provide rescue prompt messages in time and reduce the safety risks of the driver and occupants.

[0069] In one embodiment of the present invention, performing image recognition on the in-vehicle image data to obtain a state recognition result includes:

[0070] Based on the posture detection model, performing posture detection on each of the in-vehicle image data to obtain a posture detection result;

[0071] Performing color recognition on the eye image area in each of the in-vehicle image data to obtain an eye recognition result;

[0072] The state recognition result is determined based on the posture detection result and the eye recognition result.

[0073] Specifically, a pre-trained posture estimation model is used to perform posture detection on the target person in each in-vehicle image data to obtain a posture detection result corresponding to each in-vehicle image data. Optionally, the posture estimation model is trained based on a plurality of image samples and posture labels corresponding to each of the image samples, thereby obtaining a posture detection result corresponding to each in-vehicle image data. It should be noted that if the posture detection results in each in-vehicle image data are the same as the posture of the target person in the in-vehicle image data, it can be preliminarily determined that the driver and passenger in the target vehicle are in a coma.

[0074] In addition, in each in-vehicle image data, an eye image area is extracted, and then each eye image area is divided into a number of fragment images. Further, color recognition is performed on all the fragment images to identify whether all the fragment images have two colors of white and black eyes, and an eye recognition result is obtained. If the eye recognition result does not have two colors of white and black eyes, it is determined that the target person in the in-vehicle image data is in a closed-eye state, and then the driver and passenger are in a closed-eye state. In addition, in one embodiment, in order to improve the accuracy of detecting the eye recognition result, after preliminarily determining that the driver and passenger are in a closed-eye state, further, a number of eyelid fragment images are searched in each fragment image, and then the eyelid fragment images at the same position in all the eye image areas are compared to determine whether there is eyelid movement with or without blinking, so as to obtain the eye recognition result corresponding to the eyelid movement with or without blinking, so as to accurately identify the eye recognition result corresponding to the driver and passenger through eyeball color recognition and eyelid movement detection.

[0075] Furthermore, the state recognition result is determined in combination with the posture detection result and the eye recognition result, that is, if the posture detection result in each in-vehicle image data is that the posture of the target person in the in-vehicle image data is the same, and the eye recognition result is that the target person in the in-vehicle image data is in a closed-eye state, then the state recognition result is determined to be that the driver and occupant are in a coma; if the posture detection result in each in-vehicle image data is that the posture of the target person in the in-vehicle image data is different, or the eye recognition result is that the target person in the in-vehicle image data is not in a closed-eye state, then the state recognition result is determined to be that the driver and occupant is not in a coma.

[0076] The embodiment of the present invention uses the above scheme to perform posture detection on each in-vehicle image data to obtain a posture detection result; and performs color recognition on the eye image area in each in-vehicle image data, so as to accurately determine the status of the driver and passengers in the target vehicle based on the posture detection results and the eye recognition results, thereby reducing the safety hazards of the driver and passengers.

[0077] In one embodiment of the present invention, the above step 13: controlling the opening of the suspension device of the target vehicle based on the image recognition result and the audio recognition result, comprises:

[0078] Based on the image recognition result and the audio recognition result, the vehicle environment information corresponding to the target vehicle is determined; if the vehicle environment information is wading environment information, the suspension device is triggered to open.

[0079] Specifically, if both the image recognition result and the audio recognition result are water wading recognition results, wherein the water wading recognition result is a recognition result that the vehicle is in a water wading environment, that is, if the image recognition result is a recognition result that the vehicle is in a water wading environment, and the audio recognition result is a recognition result that the vehicle is in a water wading environment, then it is determined that the vehicle environment information is water wading environment information, and then an opening signal is sent down to trigger the opening of the suspension device. Additionally, if the image recognition result is not a water wading recognition result, or the audio recognition result is not a water wading recognition result, that is, if the image recognition result is a recognition result that the vehicle is not in a water wading environment, or the audio recognition result is a recognition result that the vehicle is not in a water wading environment, then it is determined that the vehicle environment information is not water wading environment information, that is, the target vehicle is not currently in a water wading environment, and at this time, there is no need to control the opening of the suspension device.

[0080] The embodiment of the present invention uses the above solution, that is, based on the image recognition result and the audio recognition result, the vehicle environment information corresponding to the target vehicle is determined; if the vehicle environment information is water-wading environment information, the suspension device is triggered to be turned on. It is achieved that according to the image recognition result and the audio recognition result, it is accurately determined whether the target vehicle is in a water-wading environment, thereby triggering the suspension device to be turned on, reducing the safety risks of the driver and passengers.

[0081] In one embodiment of the present invention, the above step 12: performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result, includes:

[0082] The environmental image data is input into an image recognition model to obtain an image recognition result output by the image recognition model; the audio data is input into an audio recognition model to obtain an audio recognition result output by the audio recognition model; the image recognition model is trained based on a number of image samples and sample labels corresponding to each of the image samples; the audio recognition model is trained based on each audio data sample and an audio category label corresponding to each of the audio data samples.

[0083] Specifically, the environmental image data is input into an image recognition model to extract the image grayscale value features in the environmental image data, and then based on the image grayscale value features, the image recognition results corresponding to the environmental image data are identified. In addition, the audio recognition model includes a feature extraction module and a prediction output module, and the audio data is input into the feature extraction module to extract the audio features corresponding to the audio data using the feature extraction module, and then the audio features are input into the prediction output module to obtain the audio recognition results output by the prediction output module. Optionally, the audio data can be the original time domain waveform of the audio or the fragment data of the time domain waveform, wherein extracting audio features from the time domain waveform can reduce information loss and improve the audio recognition effect. By combining the recognition of the environmental image data and the audio data, the image recognition results and the audio recognition results are obtained, and then the image recognition results and the audio recognition results are combined to accurately determine whether the target vehicle is in a wading environment.

[0084] In one embodiment, the image recognition model is trained based on the following steps:

[0085] Acquire several image samples, wherein the image samples are marked with sample labels; for any one of the image samples, input the image sample into the image recognition model to be trained to obtain a prediction result output by the image recognition model to be trained; calculate a model loss value based on the prediction result and the sample label corresponding to the image sample; and update the model parameters in the image recognition model to be trained according to the model loss value obtained in each iteration to obtain the image recognition model.

[0086] Specifically, several image samples are collected in advance. It should be noted that the image samples include images of vehicles driving in deep water, wading environments corresponding to rivers, shallow water, and normal driving, wherein each image sample is marked with a sample label. Further, for any of the image samples, the image sample is input into the image recognition model to be trained to determine the prediction result according to the output of the image recognition model to be trained, and then the model loss value is calculated based on the prediction result and sample label corresponding to the image sample using the target loss function. The target loss function includes equal L1 loss function and dice loss function, etc., which are not specifically limited here. After calculating the model loss value, the error back propagation algorithm is used to update the model parameters in the image recognition model to be trained. The training process ends and the next training is then carried out. During the training process, it is determined whether the updated image recognition model to be trained meets the preset training end conditions. If so, the updated image recognition model to be trained is used as the image recognition model. If not, the model training continues, wherein the preset training end conditions include loss convergence and reaching the maximum number of iterations threshold. By training the image recognition model according to image samples in various scenes, it is beneficial to control the loss value of the image recognition model within a preset range, ensuring that the image recognition model can correctly identify wading scenes, thereby helping to improve the image recognition accuracy of the image recognition model.

[0087] In one embodiment, the audio recognition model is trained based on the following steps:

[0088] Acquire a plurality of audio data samples, wherein the audio data samples are marked with audio category labels; and iteratively train an audio recognition model to be trained based on each of the audio data samples and the audio category labels corresponding to each of the audio data samples to obtain the audio recognition model.

[0089] Specifically, several audio data samples are collected in advance. It should be noted that the audio data samples include audio data in scenes such as vehicle wading and normal vehicle driving. The audio data samples are marked with audio category labels. Further, for any audio data sample, the audio data sample is input into the audio recognition model to be trained, so as to determine the prediction result corresponding to the audio data sample according to the output of the audio recognition model to be trained, and then the loss value is calculated based on the prediction result corresponding to the audio data sample and the sample label. After the loss value is calculated, the error back propagation algorithm is used to update the model parameters in the audio recognition model to be trained. This training process ends, and then the next training is carried out. During the training process, it is determined whether the updated audio recognition model to be trained meets the preset training end condition. If it does, the updated audio recognition model to be trained is used as the audio recognition model. If it does not, the model training continues. By training the image recognition model according to the audio data sample, it is beneficial to control the loss value of the audio recognition model within a preset range, ensuring that the audio recognition model can correctly identify the wading scene, thereby improving the accuracy of the audio recognition model in audio recognition.

[0090] Figure 3 is a timing diagram of the suspension device control system provided by the present invention, such as Figure 3 As shown, the suspension device control system includes a camera installed outside the vehicle (that is, the image acquisition device in this embodiment), a camera installed inside the vehicle, a wading sensor, an audio acquisition device, an intelligent judgment module, a suspension device, a suspension control module and a safety monitoring module, wherein:

[0091] The water wading sensor is used to send a water wading signal to the intelligent judgment module;

[0092] The intelligent judgment module is used to control the audio acquisition device and the camera outside the vehicle to be in an open state in response to the wading signal of the wading sensor;

[0093] The camera outside the vehicle is used to send the captured environmental image data to the intelligent judgment module;

[0094] The audio acquisition device is used to send the collected audio data to the intelligent judgment module;

[0095] The intelligent judgment module is further used to perform image recognition on the environmental image data to obtain an image recognition result; and perform audio recognition on the audio data to obtain an audio recognition result;

[0096] The intelligent judgment module is further configured to send an opening signal to the suspension control module if it is determined that the vehicle environment information of the target vehicle is wading environment information based on the image recognition result and the audio recognition result;

[0097] The suspension control module is used to trigger the opening of the suspension device of the target vehicle based on the opening signal;

[0098] The camera in the vehicle is used to collect image data in the vehicle after the suspension device of the target vehicle is turned on, and send the collected image data in the vehicle to the security monitoring module;

[0099] The safety monitoring module is used to determine whether the driver and passenger in the target vehicle are in a coma based on the in-vehicle image data, and if so, issue a rescue prompt message.

[0100] The following is a description of the suspension device control device provided by the present invention. The suspension device control device described below and the suspension device control method described above can be referred to each other.

[0101] Figure 4 Schematic diagram of the structure of the suspension device control device provided by the present invention. Figure 4 As shown, a suspension device control device according to an embodiment of the present invention comprises:

[0102] An acquisition module 41 is used to acquire environmental image data and audio data;

[0103] The recognition module 42 is used to perform image recognition on the environmental image data to obtain an image recognition result; and perform audio recognition on the audio data to obtain an audio recognition result;

[0104] The control module 43 is used to control the opening of the suspension device of the target vehicle based on the image recognition result and the audio recognition result.

[0105] The control module 43 is also used for:

[0106] Determining vehicle environment information corresponding to the target vehicle based on the image recognition result and the audio recognition result;

[0107] If the vehicle environment information is water-wading environment information, the suspension device is triggered to open.

[0108] The identification module 42 is also used for:

[0109] Inputting the environmental image data into an image recognition model to obtain an image recognition result output by the image recognition model;

[0110] Inputting the audio data into an audio recognition model to obtain an audio recognition result output by the audio recognition model;

[0111] The image recognition model is trained based on a number of image samples and sample labels corresponding to each of the image samples;

[0112] The audio recognition model is trained based on each audio data sample and the audio category label corresponding to each audio data sample.

[0113] The suspension device control device is also used for:

[0114] Acquiring in-vehicle image data of the target vehicle within a preset time;

[0115] Performing image recognition on each of the in-vehicle image data to obtain a state recognition result, wherein the state recognition result is used to characterize the state of the driver and passenger in the target vehicle;

[0116] Determine that the status recognition result belongs to a preset recognition result, and generate rescue prompt information.

[0117] The suspension device control device is also used for:

[0118] Based on the posture detection model, performing posture detection on each of the in-vehicle image data to obtain a posture detection result;

[0119] Performing color recognition on the eye image area in each of the in-vehicle image data to obtain an eye recognition result;

[0120] The state recognition result is determined based on the posture detection result and the eye recognition result.

[0121] The suspension device control device is also used for:

[0122] If the image recognition result is a wading recognition result, and the audio recognition result is a wading recognition result, determining that the vehicle environment information is wading environment information;

[0123] If the image recognition result is not a wading recognition result, or the audio recognition result is not a wading recognition result, it is determined that the vehicle environment information is non-wading environment information.

[0124] The acquisition module 41 is also used for:

[0125] In response to a wading signal from the wading sensor, controlling the image acquisition device and the audio acquisition device to be in an on state;

[0126] Acquire the environmental image data captured by the image acquisition device and the audio data collected by the audio acquisition device.

[0127] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0128] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor 510, a memory 520, a communications interface 530 and a communications bus 540, wherein the processor 510, the memory 520 and the communications interface 530 communicate with each other through the communications bus 540. The processor 510 may call the logic instructions in the memory 520 to execute the suspension device control method, which includes: acquiring environmental image data and audio data; performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result; and based on the image recognition result and the audio recognition result, controlling the suspension device of the target vehicle to open.

[0129] In addition, the logic instructions in the above-mentioned memory 520 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0130] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the suspension device control method provided by the above-mentioned methods, the method comprising: acquiring environmental image data and audio data; performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result; based on the image recognition result and the audio recognition result, controlling the suspension device of the target vehicle to open.

[0131] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the suspension device control method provided by the above methods, which includes: acquiring environmental image data and audio data; performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result; based on the image recognition result and the audio recognition result, controlling the suspension device of the target vehicle to open.

[0132] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A suspension device control method, characterized in that: include: Acquire environmental image data and audio data of the target vehicle; Performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result; Based on the image recognition result and the audio recognition result, controlling to open a suspension device of the target vehicle; After controlling to open the suspension device of the target vehicle based on the image recognition result and the audio recognition result, the method further includes: Acquiring in-vehicle image data of the target vehicle within a preset time; Performing image recognition on each of the in-vehicle image data to obtain a state recognition result, wherein the state recognition result is used to characterize the state of the driver and passenger in the target vehicle; Determining that the state recognition result belongs to a preset recognition result, and generating rescue prompt information; The performing image recognition on each of the in-vehicle image data to obtain a state recognition result includes: Based on the posture detection model, performing posture detection on each of the in-vehicle image data to obtain a posture detection result; Performing color recognition on the eye image area in each of the in-vehicle image data to obtain an eye recognition result; Determining the state recognition result based on the posture detection result and the eye recognition result; The performing color recognition on the eye image area in each of the in-vehicle image data to obtain the eye recognition result includes: Extracting an eye image region from each of the in-vehicle image data, and dividing each of the eye image regions into a plurality of fragment images; Performing color recognition on each of the fragment images to identify whether each of the fragment images has two colors, the white of the eye and the black of the eye, to obtain an eye recognition result; Among them, the eye recognition result is used to determine whether the target persons in the in-vehicle image data are all in a closed-eye state; if the eye recognition result does not have two colors of white and black eyes, it is determined that the target persons in the in-vehicle image data are all in a closed-eye state.

2. The suspension device control method according to claim 1, characterized in that: The controlling of opening the suspension device of the target vehicle based on the image recognition result and the audio recognition result includes: Determining vehicle environment information corresponding to the target vehicle based on the image recognition result and the audio recognition result; If the vehicle environment information is water-wading environment information, the suspension device is triggered to open.

3. The suspension device control method according to claim 2, characterized in that: The determining, based on the image recognition result and the audio recognition result, vehicle environment information corresponding to the target vehicle includes: If the image recognition result is a wading recognition result, and the audio recognition result is a wading recognition result, determining that the vehicle environment information is wading environment information; If the image recognition result is not a wading recognition result, or the audio recognition result is not a wading recognition result, it is determined that the vehicle environment information is non-wading environment information.

4. The suspension device control method according to claim 1, characterized in that: The performing image recognition on the environmental image data to obtain an image recognition result; and performing audio recognition on the audio data to obtain an audio recognition result, comprises: Inputting the environmental image data into an image recognition model to obtain an image recognition result output by the image recognition model; Inputting the audio data into an audio recognition model to obtain an audio recognition result output by the audio recognition model; The image recognition model is trained based on a number of image samples and sample labels corresponding to each of the image samples; The audio recognition model is trained based on each audio data sample and the audio category label corresponding to each audio data sample.

5. The suspension device control method according to claim 1, characterized in that: Obtain environmental image data and audio data, including: In response to a wading signal from the wading sensor, controlling the image acquisition device and the audio acquisition device to be in an on state; The environmental image data of the target vehicle captured by the image acquisition device and the audio data of the target vehicle collected by the audio acquisition device are acquired.

6. A suspension device control device, characterized in that: include: An acquisition module, used to acquire environmental image data and audio data; A recognition module, configured to perform image recognition on the environmental image data to obtain an image recognition result; and perform audio recognition on the audio data to obtain an audio recognition result; A control module, used for controlling the opening of a suspension device of a target vehicle based on the image recognition result and the audio recognition result; Acquire in-vehicle image data of the target vehicle within a preset time; perform image recognition on each in-vehicle image data to obtain a state recognition result, wherein the state recognition result is used to characterize the state of the driver and passenger in the target vehicle; Determining that the state recognition result belongs to a preset recognition result, and generating rescue prompt information; The performing image recognition on each of the in-vehicle image data to obtain a state recognition result includes: Based on the posture detection model, performing posture detection on each of the in-vehicle image data to obtain a posture detection result; Performing color recognition on the eye image area in each of the in-vehicle image data to obtain an eye recognition result; Determining the state recognition result based on the posture detection result and the eye recognition result; The performing color recognition on the eye image area in each of the in-vehicle image data to obtain the eye recognition result includes: Extracting an eye image region from each of the in-vehicle image data, and dividing each of the eye image regions into a plurality of fragment images; Performing color recognition on each of the fragment images to identify whether each of the fragment images has two colors, the white of the eye and the black of the eye, to obtain an eye recognition result; Among them, the eye recognition result is used to determine whether the target persons in the in-vehicle image data are all in a closed-eye state; if the eye recognition result does not have two colors of white and black eyes, it is determined that the target persons in the in-vehicle image data are all in a closed-eye state.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the suspension device control method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the suspension device control method according to any one of claims 1 to 5 is implemented.

Images