Intelligent cockpit infrared data acquisition method and system

Abstract

The invention provides an intelligent cockpit infrared data acquisition method and system, and the method comprises the steps: carrying out the monitoring and acquisition of the interior of an intelligent cockpit, obtaining a continuous image frame set, enabling the previous frame of two adjacent frames to contain infrared data, and enabling the next frame of the two adjacent frames to not contain infrared data; marking the subset containing the infrared frame or the subset not containing the infrared frame in the acquisition process; and after the subset containing the infrared frames and the subset not containing the infrared frames are identified through the marks, pixel subtraction is carried out on the subset containing the infrared frames and the subset not containing the infrared frames, and an infrared data set is obtained. According to the invention, the odd-numbered frame set or the even-numbered frame set is marked, so that the odd-numbered frame set or the even-numbered frame set can be conveniently identified and calculated to obtain the infrared data set, the limitation is eliminated to a great extent, the adaptability of various scenes is improved, and the infrared data acquisition difficulty is reduced under the condition of no optical filter.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent cockpits, and in particular to an infrared data acquisition method for an intelligent cockpit; in addition, the present invention also relates to an infrared data acquisition system for an intelligent cockpit. Background Art

[0002] The main functions of the smart cockpit are driver fatigue detection and in-car passenger detection. Driver fatigue detection is used to identify whether the driver has dangerous driving behavior. Image data is used by the algorithm, but in order to eliminate other interference and improve the accuracy of back-end recognition, infrared data must be used for recognition processing. The current mainstream practice in the market is to add a 940nm filter to the front of the lens, so that the data given to the back end are pure infrared data, but this practice undoubtedly increases costs.

[0003] In order to control costs, some manufacturers have removed the filter and extracted infrared data through algorithm optimization. In the first method, the virtual channel method in the mobile industry processor interface is used to extract the transmission infrared data. In the second method, frame counting is added to the electronic mechanical braking system and the transmission infrared data is extracted through the intermediate module transparent transmission.

[0004] However, in the first method, only the data output by the mobile industry processor interface can be used. In the second method, first, the infrared sensor needs to support the electronic mechanical braking system, and second, the middle part from the infrared sensor to the back end can transparently transmit the electronic mechanical braking system (including image signal processing transparent transmission), but this has high requirements for the middle part of the image signal processing, and many image signal processing are not supported. The above two methods have limitations, so they cannot adapt to various scenarios, increasing the difficulty of infrared data collection. Summary of the invention

[0005] In order to solve the problems existing in the prior art, at least one embodiment of the present invention provides an intelligent cockpit infrared data collection method, which largely gets rid of the limitations and improves the adaptability of various scenes, and reduces the difficulty of infrared data collection without a filter. To this end, at least one intelligent cockpit infrared data collection system of the present invention.

[0006] In a first aspect, an embodiment of the present invention provides a method for collecting infrared data of an intelligent cockpit, comprising:

[0007] The intelligent cockpit is monitored and collected to obtain a continuous set of image frames, wherein the first frame of two adjacent frames contains infrared data, and the second frame of two adjacent frames does not contain infrared data;

[0008] During the acquisition process, a subset of frames containing infrared or a subset of frames not containing infrared are marked;

[0009] After identifying the subset of frames containing infrared and the subset of frames not containing infrared by marking, the subset of frames containing infrared and the subset of frames not containing infrared are pixel-wise subtracted to obtain the infrared data set.

[0010] In some embodiments, the present invention provides a smart cockpit infrared data collection method, wherein the first frame of two adjacent frames contains infrared data, and the second frame of two adjacent frames does not contain infrared data, including:

[0011] The acquisition process of the odd-numbered frame sets of the image frame set is to capture with infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture without infrared turned on; or, the acquisition process of the odd-numbered frame sets of the image frame set is to capture without infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture with infrared turned on.

[0012] In some embodiments, the present invention provides an intelligent cockpit infrared data acquisition method that identifies odd frame sets and even frame sets through frame counting.

[0013] In some embodiments, the present invention provides a smart cockpit infrared data acquisition method, in which marking a subset containing infrared frames or a subset not containing infrared frames during the acquisition process includes:

[0014] The odd frame set or the even frame set is marked by an OSD character.

[0015] In some embodiments, the present invention provides a smart cockpit infrared data acquisition method, marking odd frame sets or even frame sets with OSD characters, including:

[0016] When an odd-numbered frame is identified, the infrared is turned on for acquisition and the OSD characters are turned on for marking. When an even-numbered frame is identified, the infrared is turned off for acquisition and the OSD characters are turned off. Alternatively, when an even-numbered frame is identified, the infrared is turned on for acquisition and the OSD characters are turned on for marking. When an odd-numbered frame is identified, the infrared is turned off for acquisition and the OSD characters are turned off.

[0017] In some embodiments, the present invention provides a smart cockpit infrared data acquisition method, wherein pixel subtraction of a subset of infrared frames and a subset of non-infrared frames comprises:

[0018] Subtract pixels from the odd-numbered frames and the even-numbered frames.

[0019] In a second aspect, an embodiment of the present invention further provides an intelligent cockpit infrared data acquisition system, characterized in that it includes:

[0020] The acquisition module is used to monitor and acquire the intelligent cockpit to obtain a continuous set of image frames, wherein the first frame of two adjacent frames contains infrared data, and the second frame of two adjacent frames does not contain infrared data;

[0021] A marking module, used for marking a subset of frames containing infrared or a subset of frames not containing infrared during the acquisition process;

[0022] An identification module, used for identifying the subset containing infrared frames and the subset not containing infrared frames by marking;

[0023] The pixel subtraction module is used to perform pixel subtraction between the subset of frames containing infrared and the subset of frames not containing infrared to obtain an infrared data set.

[0024] In a third aspect, an embodiment of the present invention further provides an intelligent cockpit infrared data acquisition device, comprising at least one processor; a memory coupled to the at least one processor, the memory storing executable instructions, which when executed by the at least one processor implement the steps of any one of the methods of the first aspect above.

[0025] In a fourth aspect, an embodiment of the present invention further provides a chip for executing the steps of the method in the first aspect. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip is used to execute the steps of the method in the first aspect.

[0026] In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the methods in the first aspect above are implemented.

[0027] It can be seen that the intelligent cockpit infrared data acquisition method and system of the embodiment of the present invention, by marking odd-numbered frame sets or even-numbered frame sets, makes them easy to identify and calculate to obtain infrared data sets, thereby getting rid of limitations to a large extent, improving the adaptability to various scenes, and reducing the difficulty of infrared data acquisition in the absence of filters. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0029] Figure 1 Shown is a flow chart of a smart cockpit infrared data collection method according to an embodiment of the present invention;

[0030] Figure 2 Shown is a schematic diagram of the framework of an intelligent cockpit infrared data acquisition system in an embodiment of the present invention. Specific implementation plan

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0033] [Example 1]

[0034] The inventor of the present solution found that in the prior art, some manufacturers removed the filter in order to control costs and extracted infrared data through algorithm optimization. In the first method, the virtual channel method in the mobile industry processor interface is adopted to extract and transmit infrared data. In the second method, a frame count is added to the electronic mechanical braking system and the transmission infrared data is extracted through the intermediate module transparent transmission. However, in the first method, only the data output by the mobile industry processor interface can be used. In the second method, firstly, the infrared sensor is required to support the electronic mechanical braking system, and secondly, the middle part from the infrared sensor to the back end can transparently transmit the electronic mechanical braking system (including image signal processing transparent transmission), but this has high requirements on the middle part of the image signal processing, and many image signal processing are not supported. The above two methods have limitations, so they cannot adapt to various scenarios, which increases the difficulty of infrared data collection. Example 1 of the present invention provides the following scheme:

[0035] like Figure 1 As shown, this embodiment provides a method for collecting infrared data of an intelligent cockpit, and the method includes the following steps:

[0036] Step 101 , monitor and collect data in the intelligent cockpit to obtain a continuous set of image frames, wherein the first frame between two adjacent frames contains infrared data, and the second frame between two adjacent frames does not contain infrared data.

[0037] In some embodiments, the acquisition process of the odd-numbered frame sets of the image frame set is to capture with infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture without infrared turned on; or, the acquisition process of the odd-numbered frame sets of the image frame set is to capture without infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture with infrared turned on.

[0038] It should be noted that after monitoring and collecting the intelligent cockpit, a video sequence is formed, and the frame numbers are 1, 2, 3, 4, 5, 6, etc., and the frame numbers 1, 3, and 5 are odd frames, and the frame numbers 2, 4, and 6 are even frames. When collecting frame numbers 1, 3, and 5, the infrared sensor is turned on to collect infrared data; when collecting frame numbers 2, 4, and 6, the infrared sensor is not turned on and infrared data is not collected. Alternatively, when collecting frame numbers 1, 3, and 5, the infrared sensor is not turned on and infrared data is not collected; when collecting frame numbers 2, 4, and 6, the infrared sensor is turned on and infrared data is collected.

[0039] Specifically, a DMS camera is used to monitor and collect data in the intelligent cockpit. The image processor model used is FH8310, the CMOS image sensor model is AR0144, and the graphics signal processor model is TI933. The infrared sensor inputs data through the parallel port and generates a digital output in BT.656 format through image signal processing, with a resolution of 1280x720, a frame rate of 30 frames per second, a clock frequency of 74.25MHz, and a low level of the field is effective and a high level of the line is effective.

[0040] Step 102: marking the subset of frames containing infrared or the subset of frames not containing infrared during the acquisition process.

[0041] In some embodiments, odd-numbered frame sets or even-numbered frame sets are marked by OSD characters.

[0042] In some embodiments, when an odd frame is identified, the infrared is turned on for collection and the OSD character is turned on for marking, and when an even frame is identified, the infrared is turned off for collection and the OSD character is turned off. Alternatively, when an even frame is identified, the infrared is turned on for collection and the OSD character is turned on for marking, and when an odd frame is identified, the infrared is turned off for collection and the OSD character is turned off.

[0043] It should be noted that the OSD characters will be refreshed and displayed in real time every frame. The default display information of the OSD includes exposure status, brightness, contrast, and frame rate format, etc. This method is suitable for retaining only a single marking character in the OSD reality, such as "I". The display position is not limited, and the preferred display position is in the upper left corner of the image frame.

[0044] Step 103 , after identifying the subset of frames containing infrared and the subset of frames not containing infrared by marking, subtract pixels from the subset of frames containing infrared and the subset of frames not containing infrared to obtain an infrared data set.

[0045] In some embodiments, the odd frame set and the even frame set are identified by frame counting, and then the odd frame set and the even frame set are distinguished and identified by the above-mentioned OSD character mark, and finally the odd frame set and the even frame set are pixel-subtracted.

[0046] It can be understood that after the image signal is processed, the output is a black-and-white picture with a frame rate of 30 frames per second, with one bright and one black. Based on the two adjacent pictures, the picture without the OSD mark is subtracted from the picture with the OSD mark, and finally a pure infrared data set is obtained.

[0047] [Example 2]

[0048] like Figure 2 As shown, this embodiment provides an intelligent cockpit infrared data acquisition system, including:

[0049] The acquisition module 201 is used to monitor and acquire the inside of the intelligent cockpit to obtain a continuous set of image frames, wherein the previous frame between two adjacent frames contains infrared data, and the next frame between two adjacent frames does not contain infrared data.

[0050] In some embodiments, the acquisition process of the odd-numbered frame sets of the image frame set is to capture with infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture without infrared turned on; or, the acquisition process of the odd-numbered frame sets of the image frame set is to capture without infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture with infrared turned on.

[0051] It should be noted that after monitoring and collecting the intelligent cockpit, a video sequence is formed, and the frame numbers are 1, 2, 3, 4, 5, 6, etc., and the frame numbers 1, 3, and 5 are odd frames, and the frame numbers 2, 4, and 6 are even frames. When collecting frame numbers 1, 3, and 5, the infrared sensor is turned on to collect infrared data; when collecting frame numbers 2, 4, and 6, the infrared sensor is not turned on and infrared data is not collected. Alternatively, when collecting frame numbers 1, 3, and 5, the infrared sensor is not turned on and infrared data is not collected; when collecting frame numbers 2, 4, and 6, the infrared sensor is turned on and infrared data is collected.

[0052] Specifically, a DMS camera is used to monitor and collect data in the intelligent cockpit. The image processor model used is FH8310, the CMOS image sensor model is AR0144, and the graphics signal processor model is TI933. The infrared sensor inputs data through the parallel port and generates a digital output in BT.656 format through image signal processing, with a resolution of 1280x720, a frame rate of 30 frames per second, a clock frequency of 74.25MHz, and a low level of the field is effective and a high level of the line is effective.

[0053] The marking module 202 is used to mark the subset containing infrared frames or the subset not containing infrared frames during the acquisition process.

[0054] In some embodiments, odd-numbered frame sets or even-numbered frame sets are marked by OSD characters.

[0055] In some embodiments, when an odd frame is identified, the infrared is turned on for collection and the OSD character is turned on for marking, and when an even frame is identified, the infrared is turned off for collection and the OSD character is turned off. Alternatively, when an even frame is identified, the infrared is turned on for collection and the OSD character is turned on for marking, and when an odd frame is identified, the infrared is turned off for collection and the OSD character is turned off.

[0056] It should be noted that the OSD characters will be refreshed and displayed in real time every frame. The default display information of the OSD includes exposure status, brightness, contrast, and frame rate format, etc. This method is suitable for retaining only a single marking character in the OSD reality, such as "I". The display position is not limited, and the preferred display position is in the upper left corner of the image frame.

[0057] The identification module 203 is used to identify the subset containing infrared frames and the subset not containing infrared frames by marking.

[0058] The pixel subtraction module 204 is used to perform pixel subtraction between the subset of frames containing infrared and the subset of frames not containing infrared to obtain an infrared data set.

[0059] In some embodiments, the odd frame set and the even frame set are identified by frame counting, and then the odd frame set and the even frame set are distinguished and identified by the above-mentioned OSD character mark, and finally the odd frame set and the even frame set are pixel-subtracted.

[0060] It can be understood that after the image signal is processed, the output is a black-and-white picture with a frame rate of 30 frames per second, with one bright and one black. Based on the two adjacent pictures, the picture without the OSD mark is subtracted from the picture with the OSD mark, and finally a pure infrared data set is obtained.

[0061] [Example 3]

[0062] This embodiment provides an intelligent cockpit infrared data acquisition device, including:

[0063] At least one processor; a memory coupled to the at least one processor, the memory storing executable instructions, wherein the executable instructions, when executed by the at least one processor, implement the method steps of embodiment 1 of the present invention.

[0064] In an intelligent cockpit infrared data acquisition device provided by an embodiment of the present invention, a processor and a memory may be provided separately or integrated together.

[0065] For example, the memory may include a random access memory, a flash memory, a read-only memory, a programmable read-only memory, a non-volatile memory or a register, etc. The processor may be a central processing unit (CPU), etc. or a graphic processing unit (GPU). The memory may store executable instructions. The processor may execute the executable instructions stored in the memory to implement the various processes described herein.

[0066] It can be understood that the memory in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a ROM (Read-Only Memory), a PROM (Programmable ROM), an EPROM (Erasable PROM), an EEPROM (Electrically EPROM), or a flash memory. The volatile memory can be a RAM (Random Access Memory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as SRAM (Static RAM), DRAM (Dynamic RAM), SDRAM (Synchronous DRAM), DDR SDRAM (Double Data Rate SDRAM), ESDRAM (Enhanced SDRAM), SLDRAM (Synchlink DRAM), and DRRAM (Direct Rambus RAM). The memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0067] In some embodiments, the memory stores the following elements, upgrade packages, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system and an application program.

[0068] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program includes various application programs, which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application program.

[0069] In the embodiment of the present invention, the processor calls a program or instruction stored in the memory, specifically, a program or instruction stored in an application program, and the processor is used to execute the method steps of the embodiment 1 of the present invention.

[0070] [Example 4]

[0071] This embodiment provides a chip for executing the method of the above-mentioned embodiment 1 of the present invention. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip is used to execute the method of the embodiment 1 of the present invention.

[0072] [Example 5]

[0073] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method of Embodiment 1 of the present invention are implemented.

[0074] For example, machine-readable storage media may include, but are not limited to, various known and unknown types of non-volatile memory.

[0075] In summary, embodiments 1-5 of the present invention provide an intelligent cockpit infrared data acquisition method and system, which mark odd-numbered frame sets or even-numbered frame sets to facilitate identification and calculation to obtain infrared data sets, thereby getting rid of limitations to a large extent, improving adaptability to various scenes, and reducing the difficulty of infrared data acquisition in the absence of filters.

[0076] It will be appreciated by those skilled in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0077] In the embodiments of the present application, the disclosed systems, devices and methods can be implemented in other ways. For example, the division of units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system. In addition, the coupling between the various units can be direct coupling or indirect coupling. In addition, the various functional units in the embodiments of the present application can be integrated into a processing unit, or can be separate physical existences, etc.

[0078] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0079] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a machine-readable storage medium. Therefore, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a machine-readable storage medium, which can include several instructions to enable an electronic device to perform all or part of the technical solution described in the embodiments of the present application. The above-mentioned storage medium may include various media that can store program codes, such as ROM, RAM, removable disk, hard disk, magnetic disk or optical disk.

[0080] The above contents are only specific implementation methods of the present application, and the protection scope of the present application is not limited thereto. Those skilled in the art may make changes or substitutions within the technical scope disclosed in the present application, and these changes or substitutions shall be within the protection scope of the present application.

Claims

1. A method for collecting infrared data in an intelligent cockpit, characterized in that: include: The intelligent cockpit is monitored and collected to obtain a continuous set of image frames, wherein the first frame of two adjacent frames contains infrared data, and the second frame of two adjacent frames does not contain infrared data; During the acquisition process, a subset of frames containing infrared or a subset of frames not containing infrared are marked; After the infrared frame subset and the non-infrared frame subset are identified by the marker, the infrared frame subset and the non-infrared frame subset are pixel-subtracted to obtain an infrared data set.

2. The intelligent cockpit infrared data acquisition method according to claim 1, characterized in that: The preceding frame of the two adjacent frames contains infrared data, and the following frame of the two adjacent frames does not contain infrared data, including: The acquisition process of the odd-numbered frame sets of the image frame set is to capture with infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture without infrared turned on; or, the acquisition process of the odd-numbered frame sets of the image frame set is to capture without infrared turned on, and the acquisition process of the even-numbered frame sets of the image frame set is to capture with infrared turned on.

3. The intelligent cockpit infrared data acquisition method according to claim 2 is characterized in that: The odd frame set and the even frame set are identified by a frame count.

4. The intelligent cockpit infrared data acquisition method according to claim 3 is characterized in that: The marking of the subset containing infrared frames or the subset not containing infrared frames during the acquisition process includes: The odd frame set or the even frame set is marked by an OSD character.

5. The intelligent cockpit infrared data acquisition method according to claim 4 is characterized in that: The marking of the odd frame set or the even frame set by using OSD characters comprises: When an odd-numbered frame is identified, the infrared is turned on for acquisition and the OSD characters are turned on for marking. When an even-numbered frame is identified, the infrared is turned off for acquisition and the OSD characters are turned off. Alternatively, when an even-numbered frame is identified, the infrared is turned on for acquisition and the OSD characters are turned on for marking. When an odd-numbered frame is identified, the infrared is turned off for acquisition and the OSD characters are turned off.

6. The intelligent cockpit infrared data acquisition method according to claim 2, characterized in that: Subtracting pixels from the infrared-containing frame subset and the infrared-free frame subset comprises: Subtract pixels from the odd-numbered frame set and the even-numbered frame set.

7. An intelligent cockpit infrared data acquisition system, characterized in that: include: The acquisition module is used to monitor and acquire the intelligent cockpit to obtain a continuous set of image frames, wherein the first frame of two adjacent frames contains infrared data, and the second frame of two adjacent frames does not contain infrared data; A marking module, used for marking a subset of frames containing infrared or a subset of frames not containing infrared during the acquisition process; An identification module, used for identifying the subset containing infrared frames and the subset not containing infrared frames by using the mark; The pixel subtraction module is used to perform pixel subtraction between the subset of frames containing infrared and the subset of frames not containing infrared to obtain an infrared data set.

8. An intelligent cockpit infrared data acquisition device, comprising at least one processor; a memory coupled to the at least one processor, the memory storing executable instructions, characterized in that: The executable instructions, when executed by the at least one processor, enable the steps of the method according to any one of claims 1 to 6 to be implemented.

9. A chip, characterized in that: It comprises a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes the steps of the method as claimed in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images