Vehicle-mounted video recording method, device, equipment, readable storage medium and program product

Abstract

The invention relates to a vehicle-mounted video recording method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a video recording condition instruction input by a user, and acquiring first audio and video data of an outside-vehicle environment and second audio and video data of an inside-vehicle behavior; respectively identifying a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data; and analyzing the video recording condition instruction, determining a video recording condition, and controlling the video recording equipment to carry out video recording under the condition that any one of the first scene and the second scene meets the video recording condition. By adopting the method, the safety of the driving process can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a vehicle-mounted video recording method, device, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] The in-vehicle camera systems currently on the market are mainly divided into two categories: one is the dashcam that focuses on safety, and the other is a simple event recording camera used to preserve accident evidence. These systems usually record continuously or perform emergency recording of collisions, and cannot intelligently trigger recording based on specific situations. For example, a dashcam can usually only start saving the first few minutes of recording when a physical trigger such as a collision occurs. If the user wants to record specific footage, they usually need to manually operate the recording function. Therefore, traditional in-vehicle recording methods bring safety risks to the driving process. Summary of the invention

[0003] Based on this, it is necessary to provide a vehicle-mounted video recording method, device, computer equipment, computer-readable storage medium and computer program product to address the above technical problems, which can improve the safety of the driving process.

[0004] In a first aspect, the present application provides a vehicle-mounted video recording method, comprising:

[0005] Obtaining a video recording condition instruction input by a user, and obtaining first audio and video data for detecting an environment outside the vehicle and second audio and video data for detecting a behavior inside the vehicle;

[0006] Respectively identifying a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data;

[0007] The recording condition instruction is parsed to determine the recording condition, and when either the first scene or the second scene meets the recording condition, the recording device is controlled to record.

[0008] In one embodiment, obtaining a recording condition instruction input by a user includes:

[0009] In response to a data input operation performed by a user on a vehicle-mounted central control screen, obtaining a video recording condition instruction sent by the vehicle-mounted central control screen;

[0010] Alternatively, when a user input signal sent by a preset application is received, the user input signal is used as a recording condition instruction.

[0011] In one embodiment, the first audio and video data includes environmental video data and vehicle surrounding audio data; the step of identifying the first scene corresponding to the first audio and video data includes:

[0012] Perform environmental recognition on environmental video data to determine the category of the external environment;

[0013] Perform scene classification on the audio data around the vehicle to determine the category of the scene outside the vehicle;

[0014] Based on the external vehicle environment category and the external vehicle scene category, a first scene is determined.

[0015] In one embodiment, the second audio and video data includes driver video data, passenger video data and in-vehicle audio data; the step of identifying the second scene corresponding to the second audio and video data includes:

[0016] Performing driver status detection on the driver video data to determine the driver status;

[0017] Performing occupant behavior detection on occupant video data to determine occupant behavior;

[0018] Perform scene classification on the in-car audio data to determine the in-car scene category;

[0019] The second scene is determined based on the driver status, occupant behavior and in-vehicle scene category.

[0020] In one embodiment, the video recording device includes an external video recording device and an internal video recording device; when either the first scene or the second scene meets the video recording condition, the video recording device is controlled to record the video, including:

[0021] When the first scene meets the recording conditions, the video recording device outside the vehicle is controlled to record the video;

[0022] When the second scene meets the recording conditions, the in-vehicle recording device is controlled to record.

[0023] In one embodiment, the vehicle video recording method further includes:

[0024] Get video data;

[0025] Identify video clips that meet the video recording conditions in the video data, and edit and synthesize the video clips to generate a synthetic video.

[0026] In a second aspect, the present application also provides a vehicle-mounted video recording device, comprising:

[0027] An acquisition module, used to acquire a recording condition instruction input by a user, and acquire first audio and video data for detecting an environment outside the vehicle and second audio and video data for detecting a behavior inside the vehicle;

[0028] An identification module, used to respectively identify a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data;

[0029] The control module is used to parse the recording condition instruction, determine the recording condition, and control the recording device to record when either the first scene or the second scene meets the recording condition.

[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Obtaining a video recording condition instruction input by a user, and obtaining first audio and video data for detecting an environment outside the vehicle and second audio and video data for detecting a behavior inside the vehicle;

[0032] Respectively identifying a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data;

[0033] The recording condition instruction is parsed to determine the recording condition, and when either the first scene or the second scene meets the recording condition, the recording device is controlled to record.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0035] Obtaining a video recording condition instruction input by a user, and obtaining first audio and video data for detecting an environment outside the vehicle and second audio and video data for detecting a behavior inside the vehicle;

[0036] Respectively identifying a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data;

[0037] The recording condition instruction is parsed to determine the recording condition, and when either the first scene or the second scene meets the recording condition, the recording device is controlled to record.

[0038] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0039] Obtaining a video recording condition instruction input by a user, and obtaining first audio and video data for detecting an environment outside the vehicle and second audio and video data for detecting a behavior inside the vehicle;

[0040] Respectively identifying a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data;

[0041] The recording condition instruction is parsed to determine the recording condition, and when either the first scene or the second scene meets the recording condition, the recording device is controlled to record.

[0042] The above-mentioned vehicle-mounted video recording method, device, computer equipment, computer-readable storage medium and computer program product can respectively obtain the first audio and video data for detecting the external environment of the vehicle and the second audio and video data for detecting the behavior inside the vehicle, and can identify the first scene corresponding to the external environment of the vehicle and the second scene corresponding to the behavior inside the vehicle through the audio and video data. At the same time, the user can specify the recording conditions for recording by inputting the recording condition instructions. Since the recording conditions are specified by the user, the user's personalized needs for vehicle-mounted video recording are met; the recording device can control the recording device to automatically start recording when either the first scene corresponding to the external environment of the vehicle or the second scene corresponding to the behavior inside the vehicle meets the recording conditions, without the need for manual operation by the user, thereby greatly improving the safety of the driving process. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A diagram showing an application environment of a vehicle-mounted video recording method in an embodiment;

[0045] Figure 2 A schematic diagram of a process of a vehicle-mounted video recording method in one embodiment;

[0046] Figure 3 1 is a schematic diagram of the overall process of a vehicle-mounted video recording method in one embodiment;

[0047] Figure 4 is a structural block diagram of a vehicle-mounted video recording device in one embodiment;

[0048] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The vehicle-mounted video recording method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. This embodiment uses the method applied to the terminal 102 as an example. It can be understood that the method can also be applied to the server, and can also be applied to the system including the terminal and the server, and is implemented through the interaction between the terminal and the server. The terminal 102 obtains the recording condition instruction input by the user, and obtains the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle; respectively identifies the first scene corresponding to the first audio and video data and the second scene corresponding to the second audio and video data; parses the recording condition instruction, determines the recording condition, and controls the recording device to record when any one of the first scene or the second scene meets the recording condition. Among them, the terminal 102 can be an electronic control unit (ECU) integrated in the vehicle, or it can be a terminal device independent of the vehicle. The terminal device can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart car device, a projection device, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0051] In an exemplary embodiment, Figure 2 As shown, a vehicle-mounted video recording method is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, including the following steps 202 to 206. Among them:

[0052] Step 202, obtaining the recording condition instruction input by the user, and obtaining the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle.

[0053] Among them, the recording condition instruction refers to the instruction used to instruct the recording device under what conditions to record. The recording device can be a driving recorder outside the vehicle, or a camera of the DMS (Driver Monitor System) or OMS (Occupant Monitoring System) inside the vehicle. Among them, the driving recorder often starts to save the previous few minutes of recording when a physical trigger such as a collision occurs, and the camera of the DMS or OMS often records continuously. The recording condition instruction is input by the user and can be flexibly set according to the user's needs, so that the vehicle recording is more in line with the user's personalized needs.

[0054] The first audio and video data refers to the audio and video data collected by the video recording device deployed outside the vehicle and the audio recording device during the detection of the environment outside the vehicle. The video recording device deployed outside the vehicle can be a driving recorder.

[0055] The second audio and video data refers to the audio and video data collected by the video recording device deployed in the car and the audio and video data collected by the recording device during the detection of the in-car behavior. The video recording device deployed in the car can be a camera of the DMS or OMS.

[0056] Step 204 , respectively identifying a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data.

[0057] The first scene refers to the scene corresponding to the environment outside the vehicle, and the second scene refers to the scene corresponding to the behavior of the driver or passengers in the vehicle.

[0058] The first scene and the second scene can be respectively identified by the first audio and video data and the second audio and video data. Exemplarily, each frame of the first audio and video data is identified to determine the type of object contained in each image frame, and if a preset number of image frames are identified to contain sheep and grassland, the first scene is determined to be a sheep herd in the grassland.

[0059] Step 206, parsing the recording condition instruction, determining the recording condition, and controlling the recording device to record when either the first scene or the second scene meets the recording condition.

[0060] The recording condition instruction is an instruction input by the user through an input device, so the terminal needs to parse the recording condition instruction and use the parsing result as the recording condition to instruct the recording device to turn on or off recording. For example, the input device can be a human-computer interaction interface or an application of a mobile terminal.

[0061] The terminal compares the first scene and the second scene with the recording conditions respectively, and can control the recording device to start recording if any scene meets the recording conditions. For example, if the recording condition is to include a flock of sheep, and the first scene is a flock of sheep in the grassland, the first scene meets the recording conditions, and the terminal can control the recording device to record the flock of sheep in the sheep grass outside the car, thereby realizing automatic recording that meets user needs.

[0062] In the above-mentioned in-vehicle recording method, by respectively acquiring the first audio and video data for detecting the external environment of the vehicle and the second audio and video data for detecting the behavior inside the vehicle, the first scene corresponding to the external environment of the vehicle and the second scene corresponding to the behavior inside the vehicle can be identified through the audio and video data. At the same time, the user can specify the recording conditions for recording by inputting the recording condition instructions. Since the recording conditions are specified by the user, the user's personalized needs for in-vehicle recording are met. The recording device can control the recording device to automatically start recording when either the first scene corresponding to the external environment of the vehicle or the second scene corresponding to the behavior inside the vehicle meets the recording conditions, without the need for manual operation by the user, thereby greatly improving the safety of the driving process.

[0063] In an exemplary embodiment, obtaining a recording condition instruction input by a user includes: in response to a data input operation performed by a user on a vehicle-mounted central control screen, obtaining a recording condition instruction sent by the vehicle-mounted central control screen; or, when receiving a user input signal sent by a preset application, using the user input signal as a recording condition instruction.

[0064] Among them, the vehicle-mounted central control screen is a human-computer interaction interface installed in the vehicle. The vehicle-mounted central control screen includes a recording condition input area. In response to the user's data input operation on the recording condition input area, a recording condition instruction is obtained, and the vehicle-mounted central control screen sends the recording condition instruction to the terminal.

[0065] The preset application refers to an application pre-loaded in the mobile terminal, and the preset application can communicate with the terminal. The preset application includes a recording condition input window. The mobile terminal responds to the data input operation performed by the user in the recording condition input window in the preset application, obtains the user input signal, and sends the user input signal to the terminal. When the terminal receives the user input signal from the preset application, it uses the user input signal as a recording condition instruction.

[0066] In some embodiments, the user input is often text data, and the vehicle-mounted central control screen converts the text data into a recording condition instruction and sends it to the terminal. After the terminal parses the recording condition instruction according to a preset protocol, the recording condition instruction can be restored to text data. Similarly, the preset application converts the text data into a user input signal and sends it to the terminal. After the terminal parses the user input signal according to a preset protocol, the recording condition instruction can be restored to text data.

[0067] In some embodiments, parsing the recording condition instruction and determining the recording condition includes: parsing the recording condition instruction into a recording condition text according to a preset protocol, identifying keywords in the recording condition text; and querying a preset mapping relationship to determine the recording condition corresponding to the keyword. The preset mapping relationship is used to indicate the correspondence between the keyword and the recording condition. For example, if the keyword included in the recording condition text is a flock of sheep, the corresponding recording condition may be to include a flock of sheep.

[0068] In this embodiment, the user can input recording condition instructions through the vehicle's central control screen or a preset application program, and obtain the user's setting requirements for recording conditions in a variety of ways, thereby realizing personalized settings of recording conditions, which is conducive to improving user experience.

[0069] In an exemplary embodiment, the first audio and video data includes environmental video data and vehicle surrounding audio data; the step of identifying the first scene corresponding to the first audio and video data includes: performing environmental recognition on the environmental video data to determine the category of the external environment; performing scene classification on the vehicle surrounding audio data to determine the category of the external scene; and determining the first scene based on the external environment category and the external scene category.

[0070] The environmental video data refers to the video data obtained by collecting the environment outside the vehicle. The vehicle surrounding audio data refers to the audio data obtained by collecting the environment outside the vehicle. The audio device outside the vehicle (MIC) is used for the vehicle surrounding audio data.

[0071] The terminal may use a preset image recognition algorithm to perform image recognition on the image frames in the environmental video data to obtain the object type contained in each image frame, and determine the category of the external environment based on the object type contained in each image frame.

[0072] The terminal can also use a pre-trained machine learning model to identify the environmental video data and determine the category of the external environment. For example, the external environment categories include grassland, high-rise, and sheep.

[0073] The terminal can use a preset audio recognition algorithm to recognize the audio data around the vehicle and determine the category of the scene outside the vehicle. For example, the category of the scene outside the vehicle includes the category of animal calls, the category of vehicle horn sounds, etc.

[0074] The first scene can be based on the external environment category and the external scene category. For example, the first mapping relationship is queried to obtain the first scene corresponding to the external environment category and the external scene category. The first mapping relationship pre-stores the correspondence between the external environment category, the external scene category and the scene.

[0075] In this embodiment, by respectively identifying the environmental video data and the audio data around the vehicle, the category of the outside vehicle environment and the category of the outside vehicle scene are obtained, and then the first scene corresponding to the outside vehicle environment is determined, which is conducive to determining the accurate first scene, thereby ensuring that the in-vehicle video recording meets the needs of users.

[0076] In an exemplary embodiment, the second audio and video data includes driver video data, passenger video data and in-vehicle audio data, and the steps of identifying the second scene corresponding to the second audio and video data include: performing driver status detection on the driver video data to determine the driver status; performing passenger behavior detection on the passenger video data to determine the passenger behavior; performing scene classification on the in-vehicle audio data to determine the in-vehicle scene category; and determining the second scene based on the driver status, passenger behavior and in-vehicle scene category.

[0077] The DMS camera in the car is used to collect driver video data for the driver. The OMS camera in the car is used to collect passenger video data for passengers. The audio device (MIC) in the car is used to collect audio data in the car.

[0078] The terminal can use a driver status recognition algorithm to recognize the driver video data and determine the driver status. For example, the driver status includes making or receiving phone calls, fatigue driving status, etc.

[0079] The terminal can use an occupant behavior recognition algorithm to recognize the occupant video data and determine the occupant behavior. For example, the occupant behavior includes singing, clapping, etc.

[0080] The terminal may use a preset audio recognition algorithm to recognize the in-car audio data and determine the in-car scene category. For example, the in-car scene category includes singing category, clapping category, etc.

[0081] The second scene can be based on the driver's state, the passenger's behavior and the in-vehicle scene category. For example, the second mapping relationship is queried to obtain the second scene corresponding to the driver's state, the passenger's behavior and the in-vehicle scene category. The second mapping relationship pre-stores the correspondence between the driver's state, the passenger's behavior, the in-vehicle scene category and the scene.

[0082] In this embodiment, by respectively identifying the driver video data, the passenger video data and the in-car audio data, the driver status, the passenger behavior and the in-car scene category are obtained, and then the second scene corresponding to the in-car behavior is determined, which is conducive to determining the accurate second scene, thereby ensuring that the in-car video recording meets the user's needs.

[0083] In an exemplary embodiment, the recording device includes an external recording device and an internal recording device; when either a first scene or a second scene meets the recording conditions, the recording device is controlled to record, including: when the first scene meets the recording conditions, the external recording device is controlled to record; when the second scene meets the recording conditions, the internal recording device is controlled to record.

[0084] The external video recording device refers to a video recording device deployed outside the vehicle, for example, it can be a driving recorder. The internal video recording device refers to a video recording device outside the vehicle, for example, it can be a DMS camera or an OMS camera.

[0085] When the first scene meets the recording conditions, the terminal can control the video recording device outside the vehicle to record the scene to collect the video recording outside the vehicle that meets the user's needs.

[0086] When the second scene meets the recording conditions, the terminal can control the in-vehicle recording device to record the scene in the car to collect the scene video that meets the user's needs.

[0087] In this embodiment, by controlling the corresponding recording device to record when the first scene or the second scene meets the recording conditions, it is beneficial to record the scenes that meet the user's needs, avoid the safety risks caused by the user's manual operation of the recording device, and thus improve the safety of the driving process.

[0088] In an exemplary embodiment, the vehicle-mounted video recording method further includes: acquiring video data; identifying video clips in the video data that meet the video recording conditions, and editing and synthesizing the video clips to generate a synthesized video.

[0089] The terminal can save the video data collected by the video recording device, and can also save the video data after further processing.

[0090] The method for processing the video data includes: identifying the video data, determining the video segments in the video data that meet the video conditions, and editing and synthesizing the video data using a preset video synthesis tool to obtain a synthesized video.

[0091] In this embodiment, by identifying the video clips that meet the video recording conditions in the video data, manual screening from a large amount of video data is avoided, and the video clips are determined efficiently and accurately; and the video data can be automatically synthesized with a high degree of automation, which is conducive to improving the user experience.

[0092] To explain the vehicle-mounted video recording method and effect in detail, the following is a most detailed embodiment:

[0093] The vehicle video recording method is applied to the vehicle video recording system, which includes a user input module, a command parsing module, an automatic capture unit, a data processing unit and an output module. Figure 3 The figure is a schematic diagram of the overall process of the vehicle-mounted video recording method in one embodiment.

[0094] Among them, the user input module can respond to the user's data input operation on the vehicle's central control screen to obtain the recording condition instruction sent by the vehicle's central control screen; or, when receiving the user input signal sent by a preset application, use the user input signal as the recording condition instruction.

[0095] The instruction parsing module can parse the recording condition instruction and determine the recording condition.

[0096] The automatic capture unit can control the video recording device outside the vehicle to record the video when the first scene meets the recording condition, and control the video recording device inside the vehicle to record the video when the second scene meets the recording condition.

[0097] The data processing unit can obtain the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle, and respectively identify the first scene corresponding to the first audio and video data and the second scene corresponding to the second audio and video data; obtain the video data, identify the video clips in the video data that meet the video conditions, and edit and synthesize the video clips to generate a synthesized video.

[0098] Among them, the first audio and video data includes environmental video data and audio data around the vehicle; the identification step of the first scene corresponding to the first audio and video data includes: performing environmental recognition on the environmental video data to determine the category of the environment outside the vehicle; performing scene classification on the audio data around the vehicle to determine the category of the scene outside the vehicle; and determining the first scene based on the category of the environment outside the vehicle and the category of the scene outside the vehicle.

[0099] The second audio and video data includes driver video data, passenger video data and in-vehicle audio data; the identification steps of the second scene corresponding to the second audio and video data include: performing driver status detection on the driver video data to determine the driver status; performing passenger behavior detection on the passenger video data to determine the passenger behavior; performing scene classification on the in-vehicle audio data to determine the in-vehicle scene category; determining the second scene based on the driver status, passenger behavior and in-vehicle scene category.

[0100] The output module is used to upload the synthesized video to a preset application for users to view, edit and share the synthesized video data. In addition, the output module is also used to upload the synthesized video to TBOX (telematics box, car networking system), then upload it to the cloud server, and then send it from the cloud server to the mobile terminal.

[0101] The above-mentioned in-vehicle recording method, by respectively acquiring the first audio and video data for detecting the external environment of the vehicle and the second audio and video data for detecting the behavior inside the vehicle, can identify the first scene corresponding to the external environment of the vehicle and the second scene corresponding to the behavior inside the vehicle through the audio and video data. At the same time, the user can specify the recording conditions for recording by inputting the recording condition instructions. Since the recording conditions are specified by the user, the user's personalized needs for in-vehicle recording are met. The recording device can control the recording device to automatically start recording when either the first scene corresponding to the external environment of the vehicle or the second scene corresponding to the behavior inside the vehicle meets the recording conditions, without the need for manual operation by the user, thereby greatly improving the safety of the driving process.

[0102] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0103] Based on the same inventive concept, the embodiment of the present application also provides a vehicle-mounted video recording device for implementing the vehicle-mounted video recording method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more vehicle-mounted video recording device embodiments provided below can refer to the limitations of the vehicle-mounted video recording method above, and will not be repeated here.

[0104] In an exemplary embodiment, Figure 4 As shown, a vehicle-mounted video recording device 100 is provided, comprising: an acquisition module 120, an identification module 140 and a control module 160, wherein:

[0105] The acquisition module 120 is used to acquire the video recording condition instruction input by the user, and acquire the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle;

[0106] An identification module 140, configured to respectively identify a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data;

[0107] The control module 160 is used to parse the recording condition instruction, determine the recording condition, and control the recording device to record when either the first scene or the second scene meets the recording condition.

[0108] The above-mentioned vehicle-mounted video recording device can identify the first scene corresponding to the environment outside the vehicle and the second scene corresponding to the behavior inside the vehicle through the audio and video data, by respectively acquiring the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle. At the same time, the user can specify the recording conditions for recording by inputting the recording condition instructions. Since the recording conditions are specified by the user, the user's personalized needs for vehicle-mounted video recording are met. The recording device can control the recording device to automatically start recording when either the first scene corresponding to the environment outside the vehicle or the second scene corresponding to the behavior inside the vehicle meets the recording conditions, without the need for manual operation by the user, thereby greatly improving the safety of the driving process.

[0109] In one embodiment, the recording condition instruction input by the user is obtained, and the acquisition module 120 is also used to: obtain the recording condition instruction sent by the vehicle-mounted central control screen in response to the user's data input operation on the vehicle-mounted central control screen; or, when receiving the user input signal sent by a preset application, use the user input signal as the recording condition instruction.

[0110] In one embodiment, the first audio and video data includes environmental video data and audio data around the vehicle; in the step of identifying the first scene corresponding to the first audio and video data, the identification module 140 is also used to: perform environmental recognition on the environmental video data to determine the category of the external environment; perform scene classification on the audio data around the vehicle to determine the category of the external scene; and determine the first scene based on the category of the external environment and the category of the external scene.

[0111] In one embodiment, the second audio and video data includes driver video data, passenger video data and in-vehicle audio data. The identification module 140 is also used to: perform driver status detection on the driver video data to determine the driver status; perform passenger behavior detection on the passenger video data to determine the passenger behavior; perform scene classification on the in-vehicle audio data to determine the in-vehicle scene category; and determine the second scene based on the driver status, passenger behavior and in-vehicle scene category.

[0112] In one embodiment, the recording device includes an external recording device and an internal recording device; when either the first scene or the second scene meets the recording conditions, the recording device is controlled to record. The control module 160 is also used to: when the first scene meets the recording conditions, control the external recording device to record; when the second scene meets the recording conditions, control the internal recording device to record.

[0113] In one embodiment, the vehicle-mounted video recording device 100 further includes a synthesis module, which is used to: obtain video data; identify video clips that meet the video recording conditions in the video data, and edit and synthesize the video clips to generate a synthesized video.

[0114] Each module in the above-mentioned vehicle-mounted video recording device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0115] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a vehicle-mounted video recording method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0116] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program. The processor implements the following steps when executing the computer program:

[0118] Obtain the recording condition instruction input by the user, and obtain the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle; respectively identify the first scene corresponding to the first audio and video data and the second scene corresponding to the second audio and video data; parse the recording condition instruction, determine the recording condition, and control the recording device to record when either the first scene or the second scene meets the recording condition.

[0119] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0120] In response to the user's data input operation on the vehicle's central control screen, a recording condition instruction sent by the vehicle's central control screen is obtained; or, when a user input signal sent by a preset application is received, the user input signal is used as a recording condition instruction.

[0121] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0122] The first audio and video data includes environmental video data and audio data around the vehicle; the environmental video data is subjected to environmental recognition to determine the category of the external environment; the audio data around the vehicle is subjected to scene classification to determine the category of the external scene; based on the category of the external environment and the category of the external scene, the first scene is determined.

[0123] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0124] The second audio and video data includes driver video data, passenger video data and in-car audio data. The driver status is detected on the driver video data to determine the driver status; the passenger behavior is detected on the passenger video data to determine the passenger behavior; the in-car audio data is scene classified to determine the in-car scene category; and the second scene is determined based on the driver status, passenger behavior and in-car scene category.

[0125] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0126] The video recording device includes an external video recording device and an internal video recording device. When a first scene meets the video recording condition, the external video recording device is controlled to record the video. When a second scene meets the video recording condition, the internal video recording device is controlled to record the video.

[0127] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0128] Acquire video data; identify video clips that meet the video conditions in the video data, and edit and synthesize the video clips to generate a synthesized video.

[0129] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0130] Obtain the recording condition instruction input by the user, and obtain the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle; respectively identify the first scene corresponding to the first audio and video data and the second scene corresponding to the second audio and video data; parse the recording condition instruction, determine the recording condition, and control the recording device to record when either the first scene or the second scene meets the recording condition.

[0131] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0132] In response to the user's data input operation on the vehicle's central control screen, a recording condition instruction sent by the vehicle's central control screen is obtained; or, when a user input signal sent by a preset application is received, the user input signal is used as a recording condition instruction.

[0133] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0134] The first audio and video data includes environmental video data and audio data around the vehicle; the environmental video data is subjected to environmental recognition to determine the category of the external environment; the audio data around the vehicle is subjected to scene classification to determine the category of the external scene; based on the category of the external environment and the category of the external scene, the first scene is determined.

[0135] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0136] The second audio and video data includes driver video data, passenger video data and in-car audio data; the driver status is detected on the driver video data to determine the driver status; the passenger behavior is detected on the passenger video data to determine the passenger behavior; the in-car audio data is scene classified to determine the in-car scene category; based on the driver status, passenger behavior and in-car scene category, the second scene is determined.

[0137] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0138] The video recording device includes an external video recording device and an internal video recording device. When a first scene meets the video recording condition, the external video recording device is controlled to record the video. When a second scene meets the video recording condition, the internal video recording device is controlled to record the video.

[0139] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0140] Acquire video data; identify video clips that meet the video conditions in the video data, and edit and synthesize the video clips to generate a synthesized video.

[0141] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0142] Obtain the recording condition instruction input by the user, and obtain the first audio and video data for detecting the environment outside the vehicle and the second audio and video data for detecting the behavior inside the vehicle; respectively identify the first scene corresponding to the first audio and video data and the second scene corresponding to the second audio and video data; parse the recording condition instruction, determine the recording condition, and control the recording device to record when either the first scene or the second scene meets the recording condition.

[0143] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0144] In response to the user's data input operation on the vehicle's central control screen, a recording condition instruction sent by the vehicle's central control screen is obtained; or, when a user input signal sent by a preset application is received, the user input signal is used as a recording condition instruction.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0146] The first audio and video data includes environmental video data and audio data around the vehicle; the environmental video data is subjected to environmental recognition to determine the category of the external environment; the audio data around the vehicle is subjected to scene classification to determine the category of the external scene; based on the category of the external environment and the category of the external scene, the first scene is determined.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0148] The second audio and video data includes driver video data, passenger video data and in-car audio data; the driver status is detected on the driver video data to determine the driver status; the passenger behavior is detected on the passenger video data to determine the passenger behavior; the in-car audio data is scene classified to determine the in-car scene category; based on the driver status, passenger behavior and in-car scene category, the second scene is determined.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] The video recording device includes an external video recording device and an internal video recording device. When a first scene meets the video recording condition, the external video recording device is controlled to record the video. When a second scene meets the video recording condition, the internal video recording device is controlled to record the video.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0152] Acquire video data; identify video clips that meet the video conditions in the video data, and edit and synthesize the video clips to generate a synthesized video.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0154] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0155] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0156] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A vehicle-mounted video recording method, characterized in that: The method comprises: Obtaining a video recording condition instruction input by a user, and obtaining first audio and video data for detecting an environment outside the vehicle and second audio and video data for detecting a behavior inside the vehicle; Respectively identifying a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data; The recording condition instruction is parsed to determine the recording condition, and when either the first scene or the second scene meets the recording condition, the recording device is controlled to record.

2. The method according to claim 1, characterized in that The step of obtaining the video recording condition instruction input by the user includes: In response to a data input operation performed by a user on a vehicle-mounted central control screen, obtaining a video recording condition instruction sent by the vehicle-mounted central control screen; Alternatively, when a user input signal sent by a preset application is received, the user input signal is used as a recording condition instruction.

3. The method according to claim 1, characterized in that The first audio and video data includes environmental video data and vehicle surrounding audio data; the first scene recognition step corresponding to the first audio and video data includes: Performing environmental recognition on the environmental video data to determine the category of the external environment of the vehicle; Performing scene classification on the audio data around the vehicle to determine the category of the scene outside the vehicle; Based on the external vehicle environment category and the external vehicle scene category, a first scene is determined.

4. The method according to claim 1, characterized in that: The second audio and video data includes driver video data, passenger video data and in-vehicle audio data, and the step of identifying the second scene corresponding to the second audio and video data includes: Performing driver status detection on the driver video data to determine the driver status; Performing occupant behavior detection on the occupant video data to determine occupant behavior; Performing scene classification on the in-vehicle audio data to determine an in-vehicle scene category; Based on the driver status, the occupant behavior and the in-vehicle scene category, a second scene is determined.

5. The method according to claim 1, characterized in that The video recording device includes an external video recording device and an internal video recording device; when either the first scene or the second scene meets the video recording condition, controlling the video recording device to record includes: When the first scene meets the recording condition, controlling the external recording device to record the video; When the second scene meets the recording condition, the in-vehicle recording device is controlled to record the scene.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Get video data; The video clips meeting the video condition are identified in the video data, and the video clips are edited and synthesized to generate a synthesized video.

7. A vehicle-mounted video recording device, characterized in that: The device comprises: An acquisition module, used to acquire a recording condition instruction input by a user, and acquire first audio and video data for detecting an environment outside the vehicle and second audio and video data for detecting a behavior inside the vehicle; an identification module, configured to respectively identify a first scene corresponding to the first audio and video data and a second scene corresponding to the second audio and video data; The control module is used to parse the recording condition instruction, determine the recording condition, and control the recording device to record when either the first scene or the second scene meets the recording condition.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images