Behavior reminding method and system and vehicle-mounted equipment

Abstract

The invention discloses a behavior reminding method and system and vehicle-mounted equipment. The behavior reminding method comprises the following steps: analyzing acquired audio data in a vehicle; if it is determined that the audio data at the first moment meet the target condition, a target behavior in a first video image corresponding to the audio data at the first moment is determined, and the first video image is a video image in the vehicle; acquiring a second video image, wherein the second video image is a video image in the vehicle; if it is determined that the target behavior exists in the second video image at the second moment, prompt information is output, and the second moment is later than the first moment.

Description

Technical Field

[0001] The present application relates to the field of monitoring, and in particular to a behavior reminder method, system and vehicle-mounted equipment. Background Art

[0002] At present, the detection of in-vehicle behavior is relatively simple, usually only the driver's behavior is detected, such as driver fatigue detection, but the behavior of non-driver passengers in the vehicle cannot be detected. This results in the inability to provide timely reminders when certain behaviors of passengers in the vehicle affect the driving safety of the vehicle. Summary of the invention

[0003] In view of this, the present application provides a behavior reminder method, system and vehicle-mounted device, and the specific scheme is as follows:

[0004] A behavior reminder method, comprising:

[0005] Analyze the collected audio data inside the vehicle;

[0006] If it is determined that the audio data at the first moment meets the target condition, determining a target behavior in a first video image corresponding to the audio data at the first moment, the first video image being a video image of the interior of the vehicle;

[0007] Acquiring a second video image, where the second video image is a video image of the interior of the vehicle;

[0008] If it is determined that the target behavior exists in the second video image at the second moment, prompt information is output, and the second moment is later than the first moment.

[0009] Furthermore, the analysis of the audio data collected from inside the vehicle includes:

[0010] If it is determined that the audio data at the first moment meets the tone condition, and the audio data includes the target keyword, it is determined that the audio data meets the target condition.

[0011] Further, the determining of the target behavior in the first video image corresponding to the audio data at the first moment includes:

[0012] Determine a first video image at the first moment from video images collected from inside the vehicle;

[0013] The first video image at the first moment is analyzed to determine a target behavior present in the first video image at the first moment.

[0014] Further, the determining of the target behavior in the first video image corresponding to the audio data at the first moment includes:

[0015] Determining the action statement included in the audio data at the first moment;

[0016] A behavior feature corresponding to the behavior statement is determined from the first video image at the first moment, and a behavior corresponding to the behavior feature is determined as a target behavior.

[0017] Furthermore, it also includes:

[0018] Determine a target position corresponding to the target behavior in a first video image corresponding to the audio data at the first moment;

[0019] If it is determined that the target behavior exists in the second video image at the second moment, outputting prompt information includes:

[0020] Analyzing the collected second video image of the interior of the vehicle;

[0021] If it is determined that the target behavior exists at the target position in the second video image at the second moment, prompt information is output.

[0022] Furthermore, it also includes:

[0023] Determining a target object that performs the target behavior in a first video image corresponding to the audio data at the first moment;

[0024] If it is determined that the target behavior exists in the second video image at the second moment, outputting prompt information includes:

[0025] Analyzing the collected second video image of the interior of the vehicle;

[0026] If it is determined that the target object in the second video image at the second moment performs the target behavior, prompt information is output.

[0027] Furthermore, the object that outputs the audio data is different from the target object that performs the target behavior.

[0028] Furthermore, the analysis of the audio data collected from inside the vehicle includes:

[0029] If it is determined that the vehicle meets the monitoring conditions, start the audio acquisition device and the video acquisition device of the vehicle; wherein the audio acquisition device is used to collect audio data inside the vehicle, and the video acquisition device is used to collect image data inside the vehicle;

[0030] The audio data collected from inside the vehicle is analyzed.

[0031] A behavior reminder system, comprising:

[0032] An analysis unit, used for analyzing the collected audio data inside the vehicle;

[0033] a determining unit, configured to determine a target behavior in a first video image corresponding to the audio data at the first moment, when it is determined that the audio data at the first moment meets the target condition, wherein the first video image is a video image of the interior of the vehicle;

[0034] A collection unit, used for collecting a second video image, where the second video image is a video image of the interior of the vehicle;

[0035] An output unit is used to output prompt information when it is determined that the target behavior exists in a second video image at a second moment, and the second moment is later than the first moment.

[0036] A vehicle-mounted device, comprising:

[0037] Sensors for collecting audio data and video images from inside the vehicle;

[0038] The processor is used to analyze the audio data inside the vehicle collected by the sensor, and when it is determined that the audio data at a first moment meets the target condition, determine the target behavior in the first video image corresponding to the audio data at the first moment; control the sensor to collect a second video image, and when it is determined that the target behavior exists in the second video image at a second moment, output prompt information, and the second moment is later than the first moment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of a behavior reminder method disclosed in an embodiment of the present application;

[0041] Figure 2 A flowchart of another behavior reminder method disclosed in an embodiment of the present application;

[0042] Figure 3 A flowchart of another behavior reminder method disclosed in an embodiment of the present application;

[0043] Figure 4 A flowchart of another behavior reminder method disclosed in an embodiment of the present application;

[0044] Figure 5A flowchart of another behavior reminder method disclosed in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the structure of a behavior reminder system disclosed in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of a vehicle-mounted device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0048] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0049] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0050] This application discloses a behavior reminder method, and its flow chart is as follows: Figure 1 As shown, including:

[0051] Step S11, analyzing the collected audio data inside the vehicle;

[0052] Step S12: if it is determined that the audio data at the first moment meets the target condition, determine the target condition in a first video image corresponding to the audio data at the first moment, where the first video image is a video image of the interior of the vehicle;

[0053] Step S13, collecting a second video image, where the second video image is a video image of the interior of the vehicle;

[0054] Step S14: if it is determined that the target behavior exists in the second video image at the second moment, output prompt information, and the second moment is later than the first moment.

[0055] At present, the detection of in-vehicle behavior is relatively simple, usually only the driver's behavior is detected, such as driver fatigue detection, but the behavior of non-driver passengers in the vehicle cannot be detected. This results in the inability to provide timely reminders when certain behaviors of passengers in the vehicle affect the driving safety of the vehicle.

[0056] Based on this, in this scheme, when it is determined that the audio data at the first moment inside the vehicle meets the target conditions, the target behavior in the first video image corresponding to the audio data at the first moment is determined, and when it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output to prompt passengers in the vehicle to avoid performing the target behavior, thereby achieving the purpose of ensuring driving safety through timely reminders.

[0057] An audio collection device is provided inside the vehicle, which can collect or monitor the audio data inside the vehicle.

[0058] Specifically, when it is determined that the vehicle meets the monitoring conditions, the audio collection device installed inside the vehicle is started, and the audio inside the vehicle is collected by the audio collection device. For example, when the vehicle is in a driving state, the audio collection device inside the vehicle collects or monitors the audio data inside the vehicle; or, when it is detected that there are passengers in the vehicle and the vehicle is unlocked or started, the audio collection device inside the vehicle collects or monitors the audio data inside the vehicle to determine whether the collected or monitored audio data meets the target conditions.

[0059] After the audio acquisition device acquires or monitors the audio data inside the vehicle, the audio data is analyzed to determine whether the audio data meets the target condition, that is, to determine whether a certain target behavior is specified in the audio data.

[0060] If it is determined that the audio data at a certain moment does not meet the target conditions, the audio data collected after that moment can be further analyzed to determine whether the audio data collected later meets the target conditions; if it is determined that the audio data at a certain moment meets the target conditions, the subsequent process can be continued, that is, determining the target behavior in the first video image corresponding to the audio data at the first moment.

[0061] If it is determined that the audio data at the first moment meets the target condition, the target behavior in the first video image corresponding to the audio data at the first moment is determined, that is, based on the information specified in the audio data, the target behavior that meets the specified information is found from the first video image.

[0062] The first video image may be a pre-stored video image including a plurality of target behaviors. After the information specified in the audio data is determined, the target behavior corresponding to the information specified in the audio data is searched from the pre-stored video image, that is, the target behavior in the first video image is determined.

[0063] Alternatively, the first video image may also be a video image collected simultaneously with the audio data, that is, when it is determined that the vehicle meets the monitoring condition, the audio acquisition device and the video acquisition device provided inside the vehicle are started, the audio data inside the vehicle is collected by the audio acquisition device, and the video image inside the vehicle is collected by the video acquisition device. When the audio acquisition device inside the vehicle collects or monitors the audio data inside the vehicle, the video acquisition device inside the vehicle simultaneously collects the video image inside the vehicle, and after it is determined that the audio data at the first moment meets the target condition, the target behavior corresponding to the audio data at the first moment is searched from the first video image at the first moment, that is, the target behavior in the first video image is determined.

[0064] After determining the target behavior in the first video image, the target behavior is recorded so that when it is determined that the target behavior occurs again inside the vehicle, a prompt can be given in time.

[0065] The video image inside the vehicle is collected by an image collector, and the collected video image at each moment is analyzed to determine whether the target behavior appears in the video image at each moment.

[0066] After the first moment, when the collected video images of the interior of the vehicle are analyzed in real time, if it is determined that the target behavior is not included in the video images at a certain moment, then when the video images at the next moment are collected, the real-time analysis of the video images at the next moment will continue; if it is determined that the target behavior is included in the video images at a certain moment, a prompt message will be output to prohibit the passengers inside the vehicle from performing the target behavior.

[0067] The audio data at the first moment may be collected in real time, that is, processed in real time after being collected in real time, so as to ensure that as long as the target behavior is collected in the second video image after the first moment, a prompt can be given in time;

[0068] Alternatively, the audio data at the first moment is not collected in real time, but has a certain delay, such as: the delay can be after 30 seconds, or within 1 minute, etc., that is, even if there is a delay, the delay is within a certain time range, so as to avoid the situation where the passengers inside the vehicle have performed the target behavior and are reminded by the passengers (the reminder is the audio data that meets the target conditions), and the passengers in the vehicle perform the target behavior again after a certain period of time, but due to the long delay, the system has not yet detected the audio data that meets the target conditions and cannot give timely reminders.

[0069] The prompt information may be audio prompt information, video prompt information, or image prompt information. Of course, it may also be other prompt information, which is not specifically limited here.

[0070] For example: when the audio acquisition device inside the vehicle detects the audio data of "Little A, don't stick your head out of the window", it is determined that the audio data meets the target condition. At this time, the target behavior of "sticking your head out of the window" is determined from the first video data. After that, the video image inside the vehicle is monitored. If it is determined that the target behavior of "sticking your head out of the window" appears in the second video image at the second moment, at this time, a voice prompt is given, which can be: "Don't stick your head out of the window", realizing automatic reminder of the target behavior, without the need for multiple reminders from the driver or passengers, thereby ensuring driving safety and riding safety.

[0071] The behavior reminder method disclosed in this embodiment analyzes the audio data collected inside the vehicle; if it is determined that the audio data at the first moment meets the target condition, the target condition in the first video image corresponding to the audio data at the first moment is determined, and the first video image is a video image of the interior of the vehicle; a second video image is collected, and the second video image is a video image of the interior of the vehicle; if it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, and the second moment is later than the first moment. When determining that the audio data at the first moment inside the vehicle meets the target condition, this scheme must first determine the target behavior in the first video image corresponding to the audio data at the first moment, and continue to collect the second video image. When it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, which ensures that when the target behavior corresponding to the audio data at the first moment appears multiple times inside the vehicle, a prompt message can be given in time to ensure the driving safety of the vehicle.

[0072] This embodiment discloses a behavior reminder method, and its flow chart is as follows: Figure 2 As shown, including:

[0073] Step S21, analyzing the collected audio data inside the vehicle;

[0074] Step S22: if it is determined that the audio data at the first moment meets the tone condition and the audio data includes the target keyword, it is determined that the audio data meets the target condition, and the target behavior in the first video image corresponding to the audio data at the first moment is determined, where the first video image is a video image of the interior of the vehicle;

[0075] Step S23, collecting a second video image, where the second video image is a video image of the interior of the vehicle;

[0076] Step S24: if it is determined that the target behavior exists in the second video image at the second moment, output prompt information, and the second moment is later than the first moment.

[0077] Audio data from inside the vehicle is collected and analyzed. If it is determined that the audio data at a first moment meets the target condition, a target behavior in a first video image corresponding to the audio data at the first moment is determined. At a second moment after the first moment, if it is determined that the target behavior exists in a second video image collected at the second moment inside the vehicle, a prompt message is output.

[0078] Among them, determining whether the audio data at the first moment meets the target condition can be specifically as follows: if it is determined that the audio data at the first moment meets the tone condition and includes the target keyword, then it is determined that the audio data at the first moment meets the target condition; if the audio data does not meet the tone condition, then even if the audio data includes the target keyword, it will be determined that the audio data does not meet the target condition; if the audio data meets the tone condition but does not include the target keyword, then it is also determined that the audio data does not meet the target condition.

[0079] Among them, the target keywords can be: no, can't, don't and other words that indicate prohibition of doing something, such as: audio data: "Don't stick your hands out of the car window", "Don't fight", etc.; the corresponding tone conditions can be: stern tone, or warning tone, etc.

[0080] Alternatively, the target keyword may be: warning prompt words. The system based on the behavior reminder method disclosed in this embodiment analyzes the audio data to determine whether it includes words or sentences belonging to the category of warning prompt words, such as: "Don't put your hands there, it's too dangerous", "It's not safe to sit like this, sit down quickly", "Put your hands back", etc. At the same time, the system analyzes the tone of the audio data to determine whether it meets the tone condition. If it meets the tone condition and includes warning prompt words, it can be determined that the audio data meets the target condition. At this time, it is necessary to determine the target behavior in the first video image corresponding to the warning prompt word corresponding to the audio data, and record the target behavior so that when the target behavior appears again in the vehicle, a prompt is given. For example, when it is determined that the audio data "Don't put your hands there, it's too dangerous" at the first moment meets the target condition, it is determined where the hand is placed in the first video image at the first moment, and after the determination, it is recorded. For example, if the hand is placed in the window gap, then this behavior is recorded, and in the second video image after the first moment, it is determined that the hand is placed in the window gap again, and then a prompt message is output. There is no need for the passengers in the vehicle to remind multiple times, but the system reminds, which improves the user experience.

[0081] Alternatively, the target keywords include not only words such as "no", "can't", "don't", etc., which indicate that something is prohibited, but also the name of a passenger in the current vehicle, so as to distinguish that the audio data is a reminder of certain behaviors of the passengers in the current vehicle, rather than words that prohibit doing something in the chat content of the passengers in the vehicle. For example, the audio data collected in the vehicle is "Xiao A, you can't do that", Xiao A is not in the vehicle. At this time, the passenger who outputs the audio data may be having a voice call with Xiao A. Then, even if the audio data meets the target condition, there is no need to perform subsequent processes. Therefore, when the audio data inside the vehicle is collected and analyzed, it is necessary not only to determine whether the audio data meets the tone condition and whether it includes words that prohibit doing something, but also to determine whether the name included in the audio data is the name of a passenger in the vehicle. If it is not the name of a passenger, there is no need to perform subsequent processes, and the subsequent audio data can continue to be collected or monitored.

[0082] Only when the tone conditions are met and the target keywords are included at the same time, can the system corresponding to the behavior reminder method disclosed in this embodiment determine that the current audio data is audio data prohibiting certain behaviors of passengers in the vehicle.

[0083] After determining that the audio data at the first moment meets the target condition, determine the target behavior in the first video image corresponding to the audio data at the first moment. Taking the first video image as a video image of the interior of a vehicle as an example, the first video image corresponding to the audio data at the first moment is the first video image at the first moment. The first video image at the first moment is analyzed to determine the target behavior existing in the first video image at the first moment.

[0084] The behavior reminder method disclosed in this embodiment analyzes the audio data collected from inside the vehicle; if it is determined that the audio data meets the tone condition and the audio data includes the target keyword, the audio data is determined to meet the target condition, and the target behavior in the first video image corresponding to the audio data at the first moment is determined, and the first video image is a video image of the interior of the vehicle; a second video image is collected, and the second video image is a video image of the interior of the vehicle; if it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, and the second moment is later than the first moment. This solution analyzes the audio data collected from inside the vehicle, and when it is determined that the audio data meets the tone condition and the audio data includes the target keyword, the audio data is determined to meet the target condition, thereby determining the target behavior corresponding to the audio data, and outputting a prompt message when the target behavior appears again inside the vehicle. By meeting the tone condition and including the target keyword, it is determined that the audio data meets the target condition, thereby ensuring the accuracy of the target condition determination and avoiding the problem of false reminders affecting the user experience.

[0085] This embodiment discloses a behavior reminder method, and its flow chart is as follows: Figure 3 As shown, including:

[0086] Step S31, analyzing the collected audio data inside the vehicle;

[0087] Step S32: if it is determined that the audio data at the first moment meets the target condition, determine the action statement included in the audio data at the first moment;

[0088] Step S33, determining a behavior feature corresponding to the behavior statement from the first audio image at the first moment, and determining the behavior corresponding to the behavior feature as the target behavior;

[0089] Step S34, collecting a second video image, where the second video image is a video image of the interior of the vehicle;

[0090] Step S35: if it is determined that the target behavior exists in the second video image at the second moment, output prompt information, and the second moment is later than the first moment.

[0091] After collecting audio data inside the vehicle, the audio data is analyzed to determine whether it meets the target conditions, and when it is determined that it meets the target conditions, the target behavior is determined. When the target behavior exists in the second video image collected after the determination, prompt information is output to provide timely reminders when passengers inside the vehicle perform the target behavior multiple times, without the need for reminders from passengers or the driver, thereby ensuring driving safety.

[0092] Specifically, when it is determined that the audio data meets the target condition, the audio data may be further analyzed to determine the behavior statements included in the audio data. The behavior statements may be statements used to express behavior, such as: "put your hands there", "sit like this", "pull your hair", etc.

[0093] That is, the analysis of audio data can be divided into two parts. First, determine whether the audio data meets the target conditions. Only when the audio data meets the target conditions will the behavioral statements included in the audio data be analyzed. If the audio data does not meet the target conditions, there is no need to continue the subsequent process. At this time, regardless of whether the audio data includes behavioral statements, there is no need to analyze the behavioral statements in the audio data.

[0094] Determining whether the audio data meets the conditions can be: determining whether the audio data meets the tone conditions and includes the target keywords. This process does not involve analysis of the sentences. Therefore, after determining that the audio data meets the tone conditions and includes the target keywords, the behavioral sentences in the audio data are determined.

[0095] After determining that the audio data at the first moment includes a behavior statement, the first video image of the interior of the vehicle at the first moment is analyzed to determine a behavior feature that matches the behavior statement.

[0096] When it is determined that the audio data at the first moment meets the target condition and there is a behavior statement in the audio data, it can be determined that the target behavior corresponding to the behavior statement must have occurred inside the vehicle at the first moment. Then, the first video image inside the vehicle at the first moment must include an image of the target behavior corresponding to the behavior statement. The audio data at the first moment is a reminder of the target behavior. In order for the system to remind of the target behavior later, it is necessary to analyze the behavior characteristics of the target behavior corresponding to the behavior statement in the first video image at the first moment, and when it is determined that the behavior characteristics are met, the system automatically reminds.

[0097] For example: the audio data "Put your hand back" at the first moment meets the target condition, and then the target behavior included in the audio data at the first moment is determined: the hand is stretched out. Based on this, the first video image at the first moment is analyzed, and the part corresponding to the target behavior of "stretching out the hand" is extracted from the first video image at the first moment. This can be done by capturing the image of this part, and then extracting the behavior features in the captured partial image, and recording the behavior features to facilitate the analysis of the second video image subsequently collected; if the second video image at the second moment is analyzed to determine that the second video image at the second moment meets the behavior features, then it is determined that the target behavior exists in the second video image at the second moment, that is, the hand is stretched out in the second video image at the second moment. At this time, there is a passenger in the vehicle who has stretched out his hand, so the system outputs a prompt message.

[0098] Among them, the prompt information output by the system based on the behavior reminder method disclosed in this embodiment can be the same as the audio data at the first moment, that is, when it is determined that the second video image at the second moment has the target behavior, the output prompt information is the audio data collected at the first moment. In this embodiment, when it is determined that the audio data at the first moment meets the target condition, the audio data at the first moment needs to be stored so that when the system outputs the prompt information, the stored audio data at the first moment can be directly called;

[0099] Alternatively, the prompt information output by the system on which the behavior reminder method disclosed in this embodiment is based is different from the audio data at the first moment, but is based on the audio data at the first moment or based on the prompt information output based on the target behavior, as long as it can achieve the purpose of prompting the passenger to stop performing the target behavior.

[0100] The behavior reminder method disclosed in this embodiment analyzes the audio data collected from inside the vehicle; if it is determined that the audio data at the first moment meets the target condition, the behavior sentence included in the audio data at the first moment is determined; the behavior feature corresponding to the behavior sentence is determined from the first audio image at the first moment, and the behavior corresponding to the behavior feature is determined as the target behavior; a second video image is collected, and the second video image is a video image of the interior of the vehicle; if it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, and the second moment is later than the first moment. When determining that the audio data at the first moment meets the target condition, this scheme determines the behavior sentence included in the audio data, and determines the behavior feature corresponding to the behavior sentence from the first video data, and determines the behavior corresponding to the behavior feature as the target behavior. By searching whether the behavior feature is included in the second video image collected later, it is determined whether the target behavior exists in the second video image, thereby improving the accuracy of the judgment on whether the target behavior is included in the video image and ensuring the output of the prompt message.

[0101] This embodiment discloses a behavior reminder method, and its flow chart is as follows: Figure 4 As shown, including:

[0102] Step S41, analyzing the collected audio data inside the vehicle;

[0103] Step S42: if it is determined that the audio data at the first moment meets the target condition, determine the target behavior in the first video image corresponding to the audio data at the first moment and the target position corresponding to the target behavior;

[0104] Step S43, collecting a second video image, where the second video image is a video image of the interior of the vehicle;

[0105] Step S44, analyzing the collected second video image of the interior of the vehicle;

[0106] Step S45: If it is determined that the target behavior exists at the target position in the second video image at the second moment, output prompt information.

[0107] The collected audio data from inside the vehicle is analyzed. If it is determined that the audio data at the first moment meets the target condition, then the first video image of the interior of the vehicle at the first moment must include the target behavior. The target behavior is determined from the first video image at the first moment. In addition, the target position corresponding to the target behavior in the first video image at the first moment also needs to be determined.

[0108] The target position corresponding to the target behavior in the first video image at the first moment may be: the position of the target behavior in the first video image, or may be: the position where the target behavior occurs inside the vehicle.

[0109] If the image acquisition range of the image acquisition device used to acquire the video image of the interior of the vehicle does not change before and after the target behavior is determined, then the target position can be directly determined as the position of the target behavior in the first video image. The image acquisition range of the video image acquired by the image acquisition device is the entire interior of the vehicle. The image acquisition device can be set at the rearview mirror outside the front windshield of the vehicle. In this case, the image acquisition device can acquire a complete image of the interior of the vehicle; or the image acquisition device can be set at other locations as long as it can acquire a complete image of the interior of the vehicle.

[0110] If the image acquisition range of the image acquisition device does not change, and the target position is the position of the target behavior in the first video image, then, after acquiring the first video image, the image acquisition device determines that the target behavior is located at the upper right corner of the first video image. Since the image acquisition range of the image acquisition device does not change, after determining the target behavior, the image acquisition device continues to acquire video images of the interior of the vehicle, and the second video image acquired is still a complete image of the interior of the vehicle. However, when analyzing the acquired second video image, only the upper right corner portion of the second video image is analyzed to determine whether the target behavior exists in the upper right corner portion of the second video image, and other portions of the second video image except the upper right corner portion do not need to be analyzed, thereby saving the time and data processing required for analyzing the second video image and improving processing efficiency.

[0111] If the image acquisition range of the image acquisition device used to acquire the video image of the interior of the vehicle changes before and after the target behavior is determined, then the target position can be determined as the location where the target behavior occurs inside the vehicle. Before the target behavior is determined, the image acquisition range of the image acquisition device is the entire interior of the vehicle. After the target behavior is determined, the image acquisition range changes and the image acquisition range is adjusted to the location where the target behavior occurs inside the vehicle.

[0112] For example: the first video image at the first moment is analyzed to determine that the position corresponding to the target behavior in the first video image is the upper left corner of the first video image. Based on the position where the image acquisition device is set, it can be determined which position inside the vehicle the upper left corner of the first video image corresponds to. If the image acquisition device is set at the rearview mirror outside the front windshield inside the vehicle, then the upper left corner of the first video image corresponds to the right side of the rear row inside the vehicle, and the right side of the rear row inside the vehicle is determined as the target position. After determining the target position, the image acquisition range of the image acquisition device is adjusted so that the image acquisition device no longer captures the complete image of the interior of the vehicle, but only captures the right side of the rear row inside the vehicle. Then, the second video image captured by the image acquisition device is the image of the right side of the rear row inside the vehicle. At this time, the complete second video image needs to be analyzed to determine whether the target behavior exists in the second video image.

[0113] The behavior reminder method disclosed in this embodiment analyzes the audio data collected from inside the vehicle; if it is determined that the audio data at the first moment meets the target condition, the target behavior in the first video image corresponding to the audio data at the first moment and the target position corresponding to the target behavior are determined; a second video image is collected, and the second video image is a video image of the interior of the vehicle; the second video image collected from the interior of the vehicle is analyzed; if it is determined that the target behavior exists at the target position in the second video image at the second moment, a prompt message is output. When determining that the audio data at the first moment meets the target condition, this scheme determines the target behavior in the first video image corresponding to the audio data at the first moment and the target position where the target behavior is located, and at a subsequent moment, analyzes whether there is a target behavior at the target position in the second video image, and only analyzes part of the second video image to accurately determine whether the same passenger in the vehicle has performed the target behavior multiple times, thereby achieving the purpose of accurate reminder, and reducing the area for analyzing the second video image, thereby improving data processing efficiency.

[0114] This embodiment discloses a behavior reminder method, and its flow chart is as follows: Figure 5 As shown, including:

[0115] Step S51, analyzing the collected audio data inside the vehicle;

[0116] Step S52: if it is determined that the audio data at the first moment meets the target condition, determine a target behavior in the first video image corresponding to the audio data at the first moment and a target object that performs the target behavior;

[0117] Step S53, collecting a second video image, where the second video image is a video image of the interior of the vehicle;

[0118] Step S54, analyzing the collected second video image of the interior of the vehicle;

[0119] Step S55: if it is determined that the target object in the second video image at the second moment performs the target behavior, output prompt information.

[0120] The collected audio data from inside the vehicle is analyzed. If it is determined that the audio data at the first moment meets the target conditions, then the first video image of the interior of the vehicle at the first moment must include the target behavior. The target behavior is determined from the first video image at the first moment. In addition, the target object that performs the target behavior also needs to be determined.

[0121] If the passengers in the vehicle include children, the child may not always stay in the same position when sitting in the back seat, and his position may change. If the child is the target object, when the child's position changes, if the original position of the target object is still analyzed, it may lead to the problem of not being reminded in time when the target behavior occurs. Therefore, in this embodiment, it is necessary to analyze the behavior of the target object.

[0122] After determining the target behavior and the target object that performs the target behavior, the characteristics of the target object are determined, and it is determined whether the image acquisition range of the image acquisition device changes.

[0123] If the image acquisition range of the image acquisition device does not change, then the image acquisition device still acquires a complete image of the interior of the vehicle. After acquiring the second video image, the second video image is analyzed to determine the position of the features that meet the target object in the second video image, and the behavior of the target object at this position is analyzed to determine whether it is consistent with the target behavior. If it is consistent, it means that the target object has performed the target behavior again, and a prompt message is output; if it is inconsistent, no prompt is required, and the behavior of the target object in the second video image acquired later is continued to be analyzed.

[0124] If the image acquisition range of the image acquisition device changes, then the image acquisition range of the image acquisition device may change with the change of the position of the target object, that is, after determining the target behavior and the target object, when the image acquisition device acquires the second video image, it locates the position of the target object and adjusts the image acquisition range of the image acquisition device to the position of the target object. When the position of the target object changes, the image acquisition range of the image acquisition device also changes. It is necessary to ensure that the target object is always included in the image acquisition range of the image acquisition device, and analyze the behavior of the target object in the second video image to determine whether the target object has performed the target behavior again. If it has performed the target behavior again, a prompt message is output. If it has not performed the target behavior again, the image of the position of the target object continues to be acquired and analyzed.

[0125] Furthermore, in the behavior reminder method disclosed in this embodiment, the target object that performs the target behavior is different from the object that outputs the audio data.

[0126] Because the target object has executed the target behavior, other objects in the vehicle will output audio data to remind the target object to stop executing the target behavior. That is, the output of audio data is to remind the target object to stop executing the target behavior. Therefore, the object that executes the target behavior of the target object is different from the object that outputs audio data.

[0127] In addition, in the behavior reminder method disclosed in this embodiment, the reminder for the target behavior can be executed only in one trip of the vehicle, that is, when the vehicle's computer is turned off or the vehicle is turned off, the relevant data of the target behavior recorded in the system is cleared, and the target behavior is re-determined based on the audio data that meets the target conditions when the vehicle is started next time or during driving.

[0128] Alternatively, the data related to the target behavior stored in the system is cleared at every preset time interval, that is, whether the data related to the target behavior stored in the system is cleared is irrelevant to whether the vehicle is restarted;

[0129] Alternatively, when it is determined that the target behavior has not been detected within a specific period of time, the data related to the target behavior stored in the system is cleared to ensure driving safety.

[0130] The behavior reminder method disclosed in this embodiment analyzes the audio data collected from inside the vehicle; if it is determined that the audio data at the first moment meets the target condition, the target behavior in the first video image corresponding to the audio data at the first moment and the target object that performs the target behavior are determined; a second video image is collected, and the second video image is a video image of the interior of the vehicle; the second video image collected from the interior of the vehicle is analyzed; if it is determined that the target object in the second video image at the second moment performs the target behavior, a prompt message is output. This scheme determines the target behavior in the first video image corresponding to the audio data at the first moment and the target object that performs the target behavior when it is determined that the audio data at the first moment meets the target condition, and at a subsequent moment, only when it is determined that the target object in the second video image performs the target behavior, the prompt message is output, without the need to analyze whether other objects perform the target behavior, thereby improving the efficiency of data processing.

[0131] This embodiment discloses a behavior reminder system, and its structural diagram is as follows: Figure 6 As shown, including:

[0132] An analyzing unit 61 , a determining unit 62 , a collecting unit 63 and an output unit 64 .

[0133] The analysis unit 61 is used to analyze the collected audio data inside the vehicle;

[0134] The determination unit 62 is used to determine the target behavior in the first video image corresponding to the audio data at the first moment when it is determined that the audio data at the first moment meets the target condition, and the first video image is a video image of the interior of the vehicle;

[0135] The acquisition unit 63 is used to acquire a second video image, where the second video image is a video image of the interior of the vehicle;

[0136] The output unit 64 is used to output prompt information when it is determined that the target behavior exists in the second video image at the second moment, and the second moment is later than the first moment.

[0137] Furthermore, the analysis unit is used to:

[0138] If it is determined that the audio data at the first moment meets the tone condition, and the audio data includes the target keyword, it is determined that the audio data meets the target condition.

[0139] Furthermore, the determination unit is used to:

[0140] A first video image at a first moment is determined from the video images collected from the interior of the vehicle; and the first video image at the first moment is analyzed to determine a target behavior existing in the first video image at the first moment.

[0141] Furthermore, the determination unit is used to:

[0142] Determine a behavior sentence included in the audio data at the first moment; determine a behavior feature corresponding to the behavior sentence from the first video image at the first moment, and determine the behavior corresponding to the behavior feature as the target behavior.

[0143] Furthermore, the determining unit is further configured to:

[0144] Determine a target position corresponding to the target behavior in a first video image corresponding to the audio data at a first moment;

[0145] Output units are used to:

[0146] The collected second video image of the interior of the vehicle is analyzed; if it is determined that the target behavior exists at the target position in the second video image at the second moment, prompt information is output.

[0147] Furthermore, the determining unit is further configured to:

[0148] Determine a target object performing a target behavior in a first video image corresponding to the audio data at a first moment;

[0149] Output units are used to:

[0150] The collected second video image of the interior of the vehicle is analyzed; if it is determined that the target object in the second video image at the second moment performs the target behavior, prompt information is output.

[0151] Furthermore, the object that outputs the audio data is different from the target object that performs the target behavior.

[0152] Furthermore, the analysis unit is used to:

[0153] If it is determined that the vehicle meets the monitoring conditions, the audio acquisition device and video acquisition device of the vehicle are started; wherein the audio acquisition device is used to collect audio data inside the vehicle, and the video acquisition device is used to collect image data inside the vehicle; the audio data collected inside the vehicle is analyzed.

[0154] The behavior reminder system disclosed in this embodiment is implemented based on the behavior reminder method disclosed in the above embodiment, which will not be described in detail here.

[0155] The behavior reminder system disclosed in this embodiment analyzes the audio data collected from inside the vehicle; if it is determined that the audio data at the first moment meets the target condition, the target condition in the first video image corresponding to the audio data at the first moment is determined, and the first video image is a video image of the interior of the vehicle; a second video image is collected, and the second video image is a video image of the interior of the vehicle; if it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, and the second moment is later than the first moment. When determining that the audio data at the first moment inside the vehicle meets the target condition, this scheme must first determine the target behavior in the first video image corresponding to the audio data at the first moment, and continue to collect the second video image. When it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, which ensures that when the target behavior corresponding to the audio data at the first moment appears multiple times inside the vehicle, a prompt message can be given in time to ensure the driving safety of the vehicle.

[0156] This embodiment discloses a vehicle-mounted device, and its structural schematic diagram is as follows: Figure 7 As shown, including:

[0157] Sensor 71 and processor 72.

[0158] Among them, the sensor 71 is used to collect audio data and video images inside the vehicle;

[0159] The processor 72 is used to analyze the audio data inside the vehicle collected by the sensor, and when it is determined that the audio data at the first moment meets the target condition, determine the target behavior in the first video image corresponding to the audio data at the first moment; control the sensor to collect the second video image, and when it is determined that the target behavior exists in the second video image at the second moment, output a prompt message, and the second moment is later than the first moment.

[0160] The sensor may include: an audio acquisition device and a video acquisition device, the audio acquisition device is used to acquire audio data inside the vehicle, and the video acquisition device is used to acquire video images inside the vehicle.

[0161] The vehicle-mounted device disclosed in this embodiment is implemented based on the behavior reminder method disclosed in the above embodiment, which will not be described in detail here.

[0162] The vehicle-mounted device disclosed in this embodiment analyzes the audio data collected from inside the vehicle; if it is determined that the audio data at the first moment meets the target condition, the target condition in the first video image corresponding to the audio data at the first moment is determined, and the first video image is a video image of the interior of the vehicle; a second video image is collected, and the second video image is a video image of the interior of the vehicle; if it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, and the second moment is later than the first moment. When determining that the audio data at the first moment inside the vehicle meets the target condition, this solution must first determine the target behavior in the first video image corresponding to the audio data at the first moment, and continue to collect the second video image. When it is determined that the target behavior exists in the second video image at the second moment, a prompt message is output, which ensures that when the target behavior corresponding to the audio data at the first moment appears multiple times inside the vehicle, a prompt message can be given in time to ensure the driving safety of the vehicle.

[0163] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0164] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0165] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0166] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A behavior reminder method, comprising: Analyze the collected audio data inside the vehicle; If it is determined that the audio data at the first moment meets the target condition, determining a target behavior in a first video image corresponding to the audio data at the first moment, the first video image being a video image of the interior of the vehicle; Acquiring a second video image, where the second video image is a video image of the interior of the vehicle; If it is determined that the target behavior exists in the second video image at the second moment, prompt information is output, and the second moment is later than the first moment.

2. The method according to claim 1, wherein analyzing the audio data collected from inside the vehicle comprises: If it is determined that the audio data at the first moment meets the tone condition, and the audio data includes the target keyword, it is determined that the audio data meets the target condition.

3. The method according to claim 1, wherein determining the target behavior in the first video image corresponding to the audio data at the first moment comprises: Determine a first video image at the first moment from video images collected from inside the vehicle; The first video image at the first moment is analyzed to determine a target behavior present in the first video image at the first moment.

4. The method according to claim 1, wherein determining the target behavior in the first video image corresponding to the audio data at the first moment comprises: Determining the action statement included in the audio data at the first moment; A behavior feature corresponding to the behavior statement is determined from the first video image at the first moment, and a behavior corresponding to the behavior feature is determined as a target behavior.

5. The method according to claim 1, further comprising: Determine a target position corresponding to the target behavior in a first video image corresponding to the audio data at the first moment; If it is determined that the target behavior exists in the second video image at the second moment, outputting prompt information includes: Analyzing the collected second video image of the interior of the vehicle; If it is determined that the target behavior exists at the target position in the second video image at the second moment, prompt information is output.

6. The method according to claim 1, further comprising: Determining a target object that performs the target behavior in a first video image corresponding to the audio data at the first moment; If it is determined that the target behavior exists in the second video image at the second moment, outputting prompt information includes: Analyzing the collected second video image of the interior of the vehicle; If it is determined that the target object in the second video image at the second moment performs the target behavior, prompt information is output. The method according to claim 6 , wherein the object outputting the audio data is different from the target object performing the target behavior.

8. The method according to claim 1, wherein analyzing the audio data collected from inside the vehicle comprises: If it is determined that the vehicle meets the monitoring conditions, start the audio acquisition device and the video acquisition device of the vehicle; wherein the audio acquisition device is used to collect audio data inside the vehicle, and the video acquisition device is used to collect image data inside the vehicle; The audio data collected from inside the vehicle is analyzed.

9. A behavior reminder system, comprising: An analysis unit, used for analyzing the collected audio data inside the vehicle; a determining unit, configured to determine a target behavior in a first video image corresponding to the audio data at the first moment, when it is determined that the audio data at the first moment meets the target condition, wherein the first video image is a video image of the interior of the vehicle; A collection unit, used for collecting a second video image, where the second video image is a video image of the interior of the vehicle; An output unit is used to output prompt information when it is determined that the target behavior exists in a second video image at a second moment, and the second moment is later than the first moment.

10. A vehicle-mounted device, comprising: Sensors for collecting audio data and video images from inside the vehicle; A processor, configured to analyze the audio data inside the vehicle collected by the sensor, and when it is determined that the audio data at the first moment meets the target condition, determine a target behavior in a first video image corresponding to the audio data at the first moment; The sensor is controlled to collect a second video image, and when it is determined that the target behavior exists in the second video image at a second moment, prompt information is output, and the second moment is later than the first moment.

Images