Vehicle-mounted auxiliary control method, device and system, product and medium

By using visible light and infrared image data in the on-board assisted driving system combined with different deep learning models, the problem of poor environmental adaptability in on-board assisted driving is solved, and efficient assisted driving effect in different environments is achieved.

CN119018174APending Publication Date: 2024-11-26YANTAI RAYTRON TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411432509.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-14
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In vehicle-assisted driving applications, deep learning is difficult to fully utilize its advantages in different environments due to high data demand and weak transfer learning ability, and the difference in image quality affects processing performance.

Method used

Provides vehicle-mounted auxiliary control methods, which can obtain visible light and infrared image data, determine the target display mode according to the working mode instructions, and use different deep learning models for processing, and perform targeted processing for different types of image data to reduce training difficulty and image quality. difference.

Benefits of technology

Effectively utilizing the advantages of deep learning models in vehicle-assisted driving, improving image processing performance in different environments, and enhancing the real-time and flexibility of assisted driving systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119018174A_ABST
    Figure CN119018174A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a vehicle-mounted auxiliary control method, device and system, a product and a medium. The vehicle-mounted auxiliary control method comprises the steps that visible light image data and infrared image data which are synchronously collected are acquired; obtaining a working mode instruction, and determining a target display mode according to the working mode instruction; if the target display mode is a visible light mode, displaying an auxiliary driving picture according to an output obtained by taking the visible light image data as input according to a visible light data model; wherein the visible light data model comprises a visible light target detection model and a lane line detection model; if the target display mode is an infrared mode, displaying an auxiliary driving picture according to an output obtained by taking the infrared image data as input according to an infrared data model; the infrared data model comprises an infrared target detection model and a super-division conversion model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of assisted driving processing, and in particular to a vehicle-mounted auxiliary control method, a vehicle-mounted electronic device, a vehicle-mounted assisted driving system, a computer program product, and a computer-readable storage medium. Background Art

[0002] With the development of the times, cars have become an important means of transportation in our daily lives, and the birth of intelligent assisted driving equipment has made our driving safer. The most efficient and convenient way to obtain the road conditions in front of the vehicle in assisted driving is to perform targeted target detection and lane line detection through the front image data collected by the camera, and then process these detected data into sound and light to remind the driver of potential dangers in driving. With the widespread application of deep learning technology in the fields of computer vision and image processing, target detection, lane line detection, etc. have gradually become the core functions of real-time visual systems, which play a key role in applications such as object recognition, scene analysis and assisted driving.

[0003] However, due to the high data requirements and dependence of deep learning, weak transfer learning and generalization capabilities, and the significant differences in image quality in different vehicle driving environments, it is difficult for deep learning to fully exert its advantages in vehicle assisted driving applications. Summary of the invention

[0004] In order to solve the existing technical problems, the present application provides a vehicle-mounted auxiliary control method, a vehicle-mounted electronic device, a vehicle-mounted auxiliary driving system, a computer program product and a computer-readable storage medium that can effectively and fully utilize the advantages of deep learning in vehicle-mounted assisted driving applications.

[0005] In a first aspect, a vehicle auxiliary control method is provided, comprising:

[0006] Acquire synchronously collected visible light image data and infrared image data;

[0007] Obtaining a working mode instruction, and determining a target display mode according to the working mode instruction;

[0008] If the target display mode is a visible light mode, an output display auxiliary driving screen is obtained by taking the visible light image data as input according to a visible light data model; wherein the visible light data model includes a visible light target detection model and a lane line detection model;

[0009] If the target display mode is the infrared mode, an output display auxiliary driving picture is obtained by taking the infrared image data as input according to the infrared data model; the infrared data model includes an infrared target detection model and a super-resolution conversion model.

[0010] In a second aspect, there is provided an in-vehicle electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory;

[0011] When the computer program is executed by the processor, the vehicle auxiliary control method described in any embodiment of the present application is implemented.

[0012] In a third aspect, a vehicle-mounted assisted driving system is provided, including an image data acquisition device and the vehicle-mounted electronic device described in any embodiment of the present application;

[0013] The image data acquisition device is used to acquire synchronized visible light image data and infrared image data, and send the synchronized visible light image data and infrared image data to the vehicle-mounted electronic device.

[0014] In a fourth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the vehicle-mounted auxiliary control method described in any embodiment of the present application is implemented.

[0015] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the vehicle-mounted auxiliary control method described in any embodiment of the present application is implemented.

[0016] The vehicle-mounted auxiliary control method provided in the above embodiment includes a visible light mode and an infrared mode, and supports determining the target display mode according to the working mode instruction. In the visible light mode and the infrared mode, a visible light data model with visible light image data as input and an infrared data model with infrared image data as input are respectively provided. In this way, different target display modes can be selected according to the type of image data in different vehicle driving environments, and deep learning models corresponding to different types of image data in different display modes are used for corresponding processing. Different deep learning models are set correspondingly for different types of image data for data processing, which is beneficial to reducing the training difficulty of the deep learning model, and is also beneficial to reducing the image quality difference that affects the processing performance of the deep learning model, and can effectively and fully give play to the advantages of deep learning in vehicle-mounted assisted driving applications.

[0017] The vehicle-mounted electronic equipment, vehicle-mounted assisted driving system, computer program product and computer-readable storage medium provided in the above embodiments belong to the same concept as the corresponding vehicle-mounted auxiliary control method embodiments, and thus have the same technical effects as the corresponding vehicle-mounted auxiliary control method embodiments, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of optional application scenarios of the vehicle auxiliary control method in one embodiment.

[0019] Figure 2Schematic diagram of a flow chart of a vehicle auxiliary control method in one embodiment.

[0020] Figure 3 Schematic diagram of the auxiliary driving screen in the visible light mode in one embodiment.

[0021] Figure 4 Schematic diagram of the assisted driving screen in infrared mode in one embodiment.

[0022] Figure 5 Schematic diagram of the architecture of a vehicle-mounted assisted driving system in one embodiment.

[0023] Figure 6 It is a flowchart of a vehicle auxiliary control method in an optional example.

[0024] Figure 7 FIG. 1 is a schematic diagram of the structure of an on-vehicle electronic device in an embodiment. DETAILED DESCRIPTION

[0025] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0027] In the following description, the expression "some embodiments" is involved, which describes a subset of all possible embodiments. It should be noted that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0028] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may also be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "inside", "outside", "left", "right" and similar expressions used herein are only for the purpose of explanation in conjunction with the embodiments of the accompanying drawings and do not represent the only implementation method.

[0029] The inventors of this application found in the application research of deep learning models to vehicle-mounted assisted driving that the quality of images in vehicle-mounted driving scenes is one of the main factors that directly affect the accuracy of deep learning models, so there is an increasing demand for efficient visual solutions that adapt to different environmental lighting conditions. High-resolution, multi-information images of visible light have a good effect on target detection under ideal lighting conditions, and deep learning models can be used to easily find targets of interest in images and determine their specific locations. However, due to the influence of darkness, strong light, bad weather, etc., the quality of visible light images drops sharply, while infrared images can greatly alleviate the above problems. By fully combining the advantages of the two, vehicle-mounted equipment with dual-light cameras can be better used in the field of assisted driving.

[0030] However, due to the high data demand and dependence of deep learning models, and weak transfer learning and generalization capabilities, in order to be compatible with the image quality differences of dual-light cameras in different vehicle driving environments, the inventors of the present application proposed a technical idea of ​​setting different target display modes according to the image data types in different vehicle driving environments, and using deep learning models corresponding to different types of image data in different display modes to perform corresponding processing. Targeted deep learning models for data processing are set for different types of image data, which is conducive to reducing the training difficulty of deep learning models, and also conducive to reducing the image quality differences that affect the processing performance of deep learning models, and can effectively and fully give play to the advantages of deep learning in vehicle-mounted assisted driving applications.

[0031] See also Figure 1 , which is a schematic diagram of an optional application scenario of the vehicle-mounted auxiliary control method provided in an embodiment of the present application. The vehicle-mounted auxiliary control method is applied to a vehicle-mounted assisted driving system, including a vehicle-mounted electronic device 10 and a dual-light camera 20. The vehicle-mounted electronic device 10 includes a host 11 and a touch screen 12 installed in the cab. The host 11 is loaded with a computer program for executing the vehicle-mounted auxiliary control method provided in an embodiment of the present application. During the driving of the vehicle, the dual-light camera 20 collects visible light image data and infrared image data of the vehicle-mounted scene in real time, and sends the visible light image data and infrared image data to the host 11. The host 11 executes the vehicle-mounted auxiliary control method, and according to the current display mode, processes and outputs the data model for processing the corresponding image data type in the current display mode, and displays the corresponding assisted driving screen through the touch screen 12. The touch screen 12 can provide the user with touch operation to switch between the visible light mode and the infrared mode.

[0032] See also Figure 2 , which is a vehicle auxiliary control method provided by an embodiment, can be applied Figure 1 The vehicle-mounted electronic device shown includes the following steps.

[0033] S101, acquiring synchronously collected visible light image data and infrared image data.

[0034] In the vehicle-mounted auxiliary control method, visible light image data and infrared image data generally refer to vehicle-mounted scene image data collected in real time while the vehicle is driving. The synchronously collected visible light image data and infrared image data may refer to the visible light image data and infrared image data collected by the visible light vehicle-mounted camera and the infrared vehicle-mounted camera installed on the vehicle at the same time, respectively; or may refer to the visible light image data and infrared image data collected by the dual-light camera.

[0035] S103, obtaining a working mode instruction, and determining a target display mode according to the working mode instruction.

[0036] The working mode instruction refers to various instructions that can be used to determine the current display mode of the vehicle-mounted electronic device, such as an automatic instruction based on the satisfaction of set conditions, or an operation instruction obtained based on a set user operation, etc. The vehicle-mounted electronic device is preset with multiple display modes, and the target display mode can be any one of the preset multiple display modes.

[0037] S105, if the target display mode is a visible light mode, an output display auxiliary driving picture is obtained by taking the visible light image data as input according to a visible light data model; wherein the visible light data model includes a visible light target detection model and a lane line detection model.

[0038] In this embodiment, the multiple display modes preset by the on-board electronic device include at least a visible light mode. For the visible light mode, a visible light data model is preset that takes the visible light image data as input, and the visible light image data is processed by the visible light data model, and the current assisted driving screen is obtained according to the output of the visible light data model. The visible light data model is a trained deep learning model, including a visible light target detection model and a lane line detection model. The visible light target detection model is a deep learning model that takes the visible light image data as input and outputs the detection coordinates of the detection targets preset in the visible light image data, such as various vehicles, pedestrians, cyclists, and animals. The lane line detection model is a deep learning model that takes the visible light image data as input and outputs the point coordinates corresponding to the road lane line in the visible light image data. Please refer to Figure 3 The assisted driving picture obtained according to the output of the visible light target detection model and the lane line detection model is a visible light image picture that marks and displays the detected targets and lane lines in the visible light image data.

[0039] S107, if the target display mode is the infrared mode, outputting a driving assistance screen obtained by taking the infrared image data as input according to an infrared data model; the infrared data model includes an infrared target detection model and a super-resolution conversion model.

[0040] In this embodiment, the multiple display modes preset by the on-board electronic device include at least an infrared mode. For the infrared mode, an infrared data model that takes infrared image data as input is preset. The infrared image data is processed by the infrared data model, and the current assisted driving screen is obtained according to the output of the infrared data model. The infrared data model is a trained deep learning model, including an infrared target detection model and a super-resolution conversion model. The infrared target detection model is a deep learning model that takes infrared image data as input and outputs the detection coordinates of preset detection targets contained in the infrared image data, such as vehicles, pedestrians, cyclists, and animals. The super-resolution conversion model is a deep learning model that takes infrared image data as input and outputs infrared image data magnified by a specified multiple. Please refer to Figure 4 The assisted driving picture obtained according to the output of the infrared target detection model and the super-resolution conversion model is an infrared image picture that is magnified by a specified multiple and has marked and displayed the detected targets in the infrared image data.

[0041] The vehicle-mounted auxiliary control method provided in the above embodiment includes a visible light mode and an infrared mode, and supports determining the target display mode according to the working mode instruction. In the visible light mode and the infrared mode, a visible light data model with visible light image data as input and an infrared data model with infrared image data as input are respectively provided. In this way, different target display modes can be selected according to the type of image data in different vehicle driving environments, and deep learning models corresponding to different types of image data in different display modes are used for corresponding processing. Different deep learning models are set correspondingly for different types of image data for data processing, which is beneficial to reducing the training difficulty of the deep learning model, and is also beneficial to reducing the image quality difference that affects the processing performance of the deep learning model, and can effectively and fully give play to the advantages of deep learning in vehicle-mounted assisted driving applications.

[0042] Among them, the display mode includes at least visible light mode and infrared mode. In the visible light mode, visible light target detection and lane line detection are performed using the visible light target detection model and lane line detection model, which can provide more information in scenes with sufficient light. In the infrared mode, infrared target detection and super-resolution display are performed using the infrared target detection model and super-resolution conversion model to provide efficient visual solutions in low-light or complex environments.

[0043] In some embodiments, step S101 includes:

[0044] Gets a picture-in-picture video stream of visible light and infrared video from a dual-light camera.

[0045] In this embodiment, the image data acquisition device for synchronously acquiring visible light image data and infrared image data is a dual-light camera. The dual-light camera can transmit the vehicle scene video stream data acquired in real time during vehicle driving to the vehicle electronic device in the form of picture-in-picture. The visible light and infrared video streams transmitted in the form of picture-in-picture are the same video data formed by superimposing the data of each infrared image frame in the infrared video stream on the designated position of each corresponding visible light image frame in the visible light video stream.

[0046] In the above embodiment, the transmission in the form of picture-in-picture can improve the efficiency of image data transmission, reduce the deployment of transmission cables, and reduce the probability of errors such as packet loss. The vehicle-mounted electronic device obtains the visible light and infrared video streams collected and transmitted by the dual-light camera in the form of picture-in-picture, and can process and separate the video streams to obtain infrared and visible light video stream signals.

[0047] In some embodiments, the working mode instruction is a power-on instruction; step S103 includes:

[0048] A power-on instruction is obtained, and an initial display mode is determined as a target display mode according to the power-on instruction; or a last display mode before the last shutdown exit is determined as a target display mode according to the power-on instruction.

[0049] The form of the working mode instruction for determining the target display mode of the vehicle-mounted electronic device is different at different use stages. In this embodiment, the user can generate a power-on instruction by turning on the vehicle-mounted electronic device. When the vehicle-mounted electronic device is started according to the power-on instruction, the vehicle-mounted electronic device can be set to use the preset initial display mode as the target display mode to enter the working state; or, when the vehicle-mounted electronic device is started, the last display mode before the last shutdown exit is set as the target display mode.

[0050] In the above embodiments, when the vehicle-mounted electronic device is started, the default initial display mode is used as the target display mode, or the last display mode determined according to the historical usage record is used as the target display mode. This can support the vehicle-mounted electronic device to enter the working state more quickly when started, reduce delays, and improve the real-time display of the assisted driving screen.

[0051] In some embodiments, the working mode instruction is a mode switching instruction; step S103 includes:

[0052] A mode switching instruction is obtained, and according to the mode switching instruction, the selected display mode is determined to be the target display mode.

[0053] The mode switching instruction refers to the working mode instruction of the vehicle-mounted electronic device to determine the target display mode according to the mode switching operation of the user to switch the current display mode after the vehicle-mounted electronic device is started. Optionally, the vehicle-mounted electronic device can provide the user with a touch button for mode switching operation through the main interface displayed on the touch screen, and the vehicle-mounted electronic device continuously monitors the input status of the user touching the touch screen, such as a sliding gesture, using the user's left and right sliding gestures in the main interface to select switching to the visible light mode or infrared mode; or a button click, using the user's button touch in the main interface to select switching to the visible light mode or infrared mode.

[0054] In the above embodiment, after the vehicle-mounted electronic device is started, it supports the user to switch in real time to the currently required display mode through human-computer interaction according to the needs of the actual application scenario. Through user-friendly touch screen operation, the user can instantly switch between different display modes to achieve instant image data acquisition and processing in different visual environments. This interactive design not only improves the flexibility and response speed of the vehicle-mounted electronic device, but also provides users with customized visual perception and analysis capabilities.

[0055] In some embodiments, if the target display mode is a visible light mode, the vehicle auxiliary control method further includes:

[0056] Stop inputting the infrared image data into the corresponding infrared data model, or stop outputting the infrared data model;

[0057] If the target display mode is infrared mode, it also includes:

[0058] Stop inputting the visible light image data into the corresponding visible light data model, or stop outputting the visible light data model.

[0059] When the on-board electronic device switches from the current infrared mode to the visible light mode according to the mode switching instruction, the output of the infrared data model currently being output is first stopped, and the output of the visible light data model is immediately outputted, so as to realize the switching display of the auxiliary driving screen. Stopping the output of the infrared data model currently being outputted includes: stopping the input of the infrared image data into the corresponding infrared data model, or stopping the output of the infrared data model.

[0060] Correspondingly, when the on-board electronic device switches from the current visible light mode to the infrared mode according to the mode switching instruction, the output of the currently output visible light data model is first stopped, and the output of the infrared data model is immediately output, so as to realize the switching display of the auxiliary driving screen. Stopping the output of the currently output visible light data model includes: stopping the input of the visible light image data into the corresponding visible light data model, or stopping the output of the visible light data model.

[0061] In this embodiment, the on-board electronic device completes the loading of the visible light data model and the infrared data model in the initial state, and maintains the real-time processing of the visible light data model and the infrared data model with the visible light image data and the infrared image data as input respectively, and only selects the output of the visible light data model or the infrared data model according to the currently selected display mode to obtain the currently displayed assisted driving screen. When the on-board electronic device triggers the mode switching process according to the mode switching instruction, it can directly stop the output of the current visible light data model and immediately output the output of the infrared data model, or directly stop the output of the current infrared data model and immediately output the output of the visible light data model, thereby effectively reducing the delay and improving the switching fluency and user experience.

[0062] In some embodiments, before step S101, the following steps are included:

[0063] In the initialization state, preloading of data processing models required for output display of multiple display modes is completed; the data processing models include visible light data models and infrared data models.

[0064] When the device is turned on and initialized, the data processing models required for the output display of all display modes are preloaded, so that the data processing models required for subsequent switching between different display modes can be quickly accessed. In this embodiment, the display mode includes a visible light mode, and the data processing models required for the corresponding output display include a visible light target detection model and a lane line detection model; the display mode includes an infrared mode, and the data processing models required for the corresponding output display include an infrared target detection model and a super-resolution conversion model.

[0065] After the preloaded data processing model is completed, the image data of the corresponding data type can be kept as input for processing, and the processing unit can control the output of only the data results of the corresponding data processing model according to the currently selected display mode, so as to ensure that the delay when switching the display mode and the response time of the processing unit are reduced, thereby improving the user experience and the consistency of visual perception.

[0066] In order to ensure a more comprehensive understanding of the vehicle auxiliary control method provided in the embodiment of the present application, please refer to Figure 5 and 6 The vehicle-mounted auxiliary driving system includes a vehicle-mounted electronic device and a dual-light camera, and the vehicle-mounted device includes a host and a touch screen. The vehicle-mounted auxiliary control method includes:

[0067] S11, video stream acquisition and transmission processing.

[0068] When the vehicle-mounted electronic equipment is started, the visible light and infrared video streams collected by the dual-light camera are transmitted to the host in the form of picture-in-picture. The host processes the visible light and infrared video streams and divides them into two video signals, infrared and visible light.

[0069] S12, touch screen input detection.

[0070] The host continuously monitors the user's input status on the touch screen. These inputs may be sliding gestures or button clicks. For example, the left and right sliding gestures on the main interface can be used to select to switch to the visible light or infrared screen.

[0071] S13, model preloading.

[0072] In the initial state of the device, all trained data processing models required for all display modes are preloaded (including visible light target detection model, lane line detection model, infrared target detection model, and super-resolution conversion model) for quick access. In the default infrared mode, the host controls the output of only the data results of the infrared target detection model and the super-resolution conversion model to achieve the optimization of infrared target drawing and infrared display effect.

[0073] S14, the user selects the visible light mode.

[0074] When the host detects that the user selects the visible light mode through the touch screen, the mode switching process is triggered. The host first stops the output data of the super-resolution conversion model and infrared target detection model that are currently being output, and immediately outputs the output data of the visible light target detection model and visible light lane line model that are kept loaded.

[0075] S15, visible light video data display post-processing.

[0076] In visible light mode, after the visible light data model starts transmitting, the host starts to feed the visible light video stream into the visible light data model in real time. The visible light target detection model identifies and tracks target objects in the visible light video stream, such as vehicles and pedestrians, and outputs corresponding location information and category labels; the lane line detection model analyzes the road surface in the visible light video stream, identifies the location and shape of the lane line, and provides vehicle driving trajectory prediction and environmental perception.

[0077] S16, the user selects infrared mode.

[0078] When the user chooses to switch to infrared mode, the host detects the user's sliding input again and triggers the switching process. The host stops the output data of the visible light target detection model and visible light lane line model currently being output, and immediately outputs the output data of the infrared target detection model and super-resolution conversion model that remain loaded.

[0079] S17, infrared video data display post-processing.

[0080] In infrared mode, when the infrared data model starts transmitting, the host starts to feed the infrared video stream into the infrared data model in real time. The super-resolution conversion model improves the resolution and details of the infrared video screen and enhances visual perception. The infrared target detection model adapts to low-light environmental conditions, identifies and tracks targets in the infrared video screen, such as vehicles, pedestrians, and animals, providing accurate target analysis and scene perception.

[0081] In the above embodiment, the visible light and infrared video streams collected by the dual-light camera are transmitted to the back-end processing unit (host of the vehicle-mounted electronic device) in the form of picture-in-picture, and the processing unit cuts the video and displays it according to the user's choice. The device monitors the user's touch screen input in real time, supports the user to slide the device screen left and right through the sliding touch function, and realizes the dual-light instant switching of the vehicle-mounted electronic device. Once it is detected that the user chooses to switch from infrared mode to visible light mode, the deep learning model switching process is immediately triggered, and the processing unit quickly stops the data transmission of the currently running super-resolution conversion model and infrared target detection model, and then starts the data transmission of the pre-trained visible light target detection model and visible light lane line model. After the vehicle-mounted electronic device is turned on, the processing unit will start the post-processing process of the visible light video stream, including real-time target detection frame and lane line display. These tasks are optimized by efficient algorithms to ensure accurate visual analysis results at high frame rates. When the user selects infrared mode, the deep learning model switching process will also be carried out in a similar manner. The processing unit immediately responds to the input of the touch screen, stops the data transmission of the currently running visible light target detection model and visible light lane line model, and starts the data transmission of the super-resolution conversion model and infrared target detection model. Among them, the input of the visible light target detection model and the infrared target detection model are the corresponding visible light video stream and infrared video stream, and the output data are the various detection targets in each image frame in the visible light video stream and the infrared video stream, including but not limited to the detection coordinates of vehicles, pedestrians, cyclists, and animals; the input of the lane line detection model is the visible light video stream, and the output data are the point coordinates corresponding to the road lane line in each image frame in the visible light video stream; the input of the super-resolution conversion model is the infrared video stream, and the output data are each image frame in the infrared video stream with a magnification multiple.

[0082] Among them, the visible light data model and the infrared data model are pre-trained with visible light image data and infrared image data as input respectively. The visible light data model can effectively handle target detection and lane line analysis tasks in daytime high-definition videos, and the infrared data model can effectively handle target detection and analysis tasks under the glare and low-light conditions caused by car lights and street lights in night videos. By pre-loading all models during device initialization and controlling the data output of different models when switching models, the delay can be greatly reduced, the real-time nature of data transmission is taken into account, and the switching smoothness and user experience are improved.

[0083] On the other hand, please see Figure 7 , and also provides an in-vehicle electronic device, including a memory 112 and a processor 111, wherein the memory 112 stores a computer program; when the computer program is executed by the processor 111, the in-vehicle auxiliary control method described in any embodiment of the present application is implemented. Optionally, the in-vehicle electronic device may refer to an in-vehicle device host installed in the vehicle cab.

[0084] In another aspect, the present application also provides a vehicle-mounted assisted driving system, comprising an image data acquisition device and the vehicle-mounted electronic device described in the above embodiment. The image data acquisition device is used to acquire synchronized visible light image data and infrared image data and send them to the vehicle-mounted electronic device.

[0085] Optionally, the image data acquisition device is a dual-light camera, and the dual-light camera includes a visible light image data acquisition module and an infrared image data acquisition module.

[0086] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the vehicle-mounted auxiliary control method described in any embodiment of the present application.

[0087] On the other hand, the embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the process of each of the above-mentioned vehicle auxiliary control method embodiments is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0088] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, vehicle-mounted device, etc.) to execute the methods described in each embodiment of the present invention.

[0090] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A vehicle auxiliary control method, characterized in that: include: Acquire synchronously collected visible light image data and infrared image data; Obtaining a working mode instruction, and determining a target display mode according to the working mode instruction; If the target display mode is a visible light mode, an output display auxiliary driving screen is obtained by taking the visible light image data as input according to a visible light data model; wherein the visible light data model includes a visible light target detection model and a lane line detection model; If the target display mode is the infrared mode, an output display auxiliary driving screen is obtained by taking the infrared image data as input according to the infrared data model; The infrared data model includes an infrared target detection model and a super-resolution conversion model.

2. The vehicle-mounted auxiliary control method according to claim 1, characterized in that: The step of acquiring the synchronously collected visible light image data and infrared image data comprises: Gets a picture-in-picture video stream of visible light and infrared video from a dual-light camera.

3. The vehicle-mounted auxiliary control method according to claim 1, characterized in that: The working mode instruction is a power-on instruction; the obtaining of the working mode instruction and determining the target display mode according to the working mode instruction include: A power-on instruction is obtained, and an initial display mode is determined as a target display mode according to the power-on instruction; or a last display mode before the last shutdown exit is determined as a target display mode according to the power-on instruction.

4. The vehicle-mounted auxiliary control method according to claim 1, characterized in that: The working mode instruction is a mode switching instruction; the obtaining of the working mode instruction, and determining the target display mode according to the working mode instruction, comprises: A mode switching instruction is obtained, and according to the mode switching instruction, the selected display mode is determined to be the target display mode.

5. The vehicle-mounted auxiliary control method according to claim 4, characterized in that: If the target display mode is the visible light mode, it also includes: Stop inputting the infrared image data into the corresponding infrared data model, or stop outputting the infrared data model; If the target display mode is infrared mode, it also includes: Stop inputting the visible light image data into the corresponding visible light data model, or stop outputting the visible light data model.

6. The vehicle-mounted auxiliary control method according to claim 1, characterized in that: Before acquiring the synchronously collected visible light image data and infrared image data, the method includes: In the initialization state, preloading of data processing models required for output display of multiple display modes is completed; the data processing models include visible light data models and infrared data models.

7. A vehicle-mounted electronic device, characterized in that: The device comprises a memory and a processor, wherein the memory stores a computer program; When the computer program is executed by the processor, the vehicle auxiliary control method according to any one of claims 1 to 6 is implemented.

8. A vehicle-mounted assisted driving system, characterized in that: It includes an image data acquisition device and the vehicle-mounted electronic device as claimed in claim 7; The image data acquisition device is used to acquire synchronized visible light image data and infrared image data, and send the synchronized visible light image data and infrared image data to the vehicle-mounted electronic device.

9. The vehicle-mounted assisted driving system according to claim 8, characterized in that: The image data acquisition device is a dual-light camera, and the dual-light camera includes a visible light image data acquisition module and an infrared image data acquisition module.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle auxiliary control method according to any one of claims 1 to 6 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle-mounted auxiliary control method according to any one of claims 1 to 6 is implemented.