Dual-camera frame synchronization processing method, device, user terminal and storage medium

Abstract

A dual-camera frame synchronization processing method, device, user terminal and storage medium, including an infrared camera and a visible light camera, including: a parameter acquisition step, an infrared image data acquisition step, a visible light image data acquisition step, a timestamp retrieval step and a pairing judgment step. The dual-camera frame synchronization processing method, device, user terminal and storage medium adopt a soft synchronization method to more concisely and efficiently process the frame synchronization problem of infrared images and visible light images, and can process the frame synchronization problem of binocular cameras with different frame rates or the same frame rate, thereby improving the consistency and timeliness of infrared image and visible light image processing.

Description

Technical Field

[0001] The present application relates to the field of image data processing technology, and in particular to a dual-camera frame synchronization processing method, device, user terminal and storage medium. Background Art

[0002] Most current camera devices have dual cameras at the same end, one of which is an infrared camera and the other is a visible light camera. When generating a thermal imaging panoramic picture, it is necessary to synchronize the frame data of the infrared image obtained by the infrared camera and the visible light image obtained by the visible light camera, and then perform subsequent stitching processing.

[0003] In the process of implementing the technical solutions in the embodiments of the present application, the inventors of the present application found that the above technology has at least the following technical problems:

[0004] Cameras on the market use hard synchronization to achieve frame synchronization between infrared images and visible light images. Specifically, the same hardware is used to simultaneously issue trigger acquisition commands to achieve time synchronization of acquisition and measurement of each sensor. For example, infrared cameras and visible light cameras use the same hardware pulse signal to achieve simultaneous exposure control. However, this method can only process infrared cameras and visible light cameras with the same frame rate, which has certain limitations.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In view of at least one of the above technical problems, the present application provides a dual-camera frame synchronization processing method, device, user terminal and storage medium, which solves the problem that most camera devices on the market currently use hard synchronization to achieve frame synchronization of infrared images and visible light images. This processing method can only process infrared cameras and visible light cameras with the same frame rate, and has certain limitations.

[0007] To achieve the above-mentioned purpose, according to an embodiment of the first aspect of the present application, a dual-camera frame synchronization processing method is provided, including an infrared camera and a visible light camera, including:

[0008] Parameter acquisition step: acquiring a first camera parameter of the infrared camera and a second camera parameter of the visible light camera, and generating a first gap trigger time and a second gap trigger time according to the first camera parameter and the second camera parameter, wherein the first camera parameter includes a first frame rate, a first resolution and a first image processing bandwidth, and the second camera parameter includes a second frame rate, a second resolution and a second image processing bandwidth;

[0009] Infrared image data acquisition step: acquiring an infrared image data frame and storing the infrared image data frame in a first data buffer pool, wherein the infrared image data frame has a first timestamp;

[0010] Visible light image data acquisition step: acquiring a visible light image data frame and storing the visible light image data frame in a second data buffer pool, wherein the visible light image data frame has a second timestamp;

[0011] Timestamp retrieval step: taking out infrared image data frames from the first data buffer pool based on the first gap trigger time, and taking out visible light image data frames from the second data buffer pool based on the second gap trigger time;

[0012] Pairing judgment step: judging whether the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame, and if so, optimizing the extracted infrared image data frame and the visible light image data frame.

[0013] In one implementation, the pairing determination step further includes:

[0014] When the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame, generating an identification mark on the infrared image data frame and the visible light image data frame;

[0015] Identify whether there is an identification mark on the infrared image data frame and the visible light image data frame;

[0016] If yes, stop the timestamp retrieval step.

[0017] In one implementation, after optimizing the infrared image data frame and the visible light image data frame with the identification mark, the identification mark of the infrared image data frame and the visible light image data frame is removed;

[0018] Repeat the timestamp retrieval step.

[0019] In one implementation, if the first data buffer pool is not empty, a timestamp retrieval step is started;

[0020] If the second data buffer pool is not empty, the timestamp retrieval step is started.

[0021] In one implementation, the timestamp retrieval step further includes: a timeout range interval;

[0022] After the infrared image data frame is taken out from the first data buffer pool based on the first gap trigger time, a waiting time is generated;

[0023] or, after taking out the visible light image data frame from the second data buffer pool based on the second gap trigger time, generating a waiting time;

[0024] If the waiting time exceeds the timeout range, the timestamp retrieval step is performed again.

[0025] In one implementation, if the first frame rate is the same as the second frame rate, the timeout range is 0 to 50 ms.

[0026] In one implementation, if the first frame rate is different from the second frame rate, the timeout range is 0 to 100 ms.

[0027] To achieve the above-mentioned purpose, according to an embodiment of the second aspect of the present application, a dual-camera frame synchronization processing device is provided, comprising:

[0028] A parameter acquisition module, used to acquire a first camera parameter of the infrared camera and a second camera parameter of the visible light camera, and generate a first gap trigger time and a second gap trigger time according to the first camera parameter and the second camera parameter, wherein the first camera parameter includes a first frame rate, a first resolution and a first image processing bandwidth, and the second camera parameter includes a second frame rate, a second resolution and a second image processing bandwidth;

[0029] A first acquisition module, used for acquiring an infrared image data frame and storing the infrared image data frame in a first data buffer pool, wherein the infrared image data frame has a first timestamp;

[0030] A second acquisition module, used for acquiring a visible light image data frame and storing the visible light image data frame in a second data buffer pool, wherein the visible light image data frame has a second timestamp;

[0031] A timestamp retrieval module, used to retrieve infrared image data frames from a first data buffer pool based on a first gap trigger time, and retrieve visible light image data frames from a second data buffer pool based on a second gap trigger time;

[0032] The pairing determination module is used to determine whether the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame.

[0033] The dual-camera frame synchronization processing method or device of the present application adopts a soft synchronization method to more concisely and efficiently process the frame synchronization problems of infrared images and visible light images, and can process the frame synchronization problems of binocular cameras with different frame rates or the same frame rates, thereby improving the consistency and timeliness of infrared image and visible light image processing.

[0034] To achieve the above object, according to an embodiment of the third aspect of the present application, a user terminal is provided, including a memory and a processor;

[0035] Storage for storing computer programs;

[0036] A processor is used to implement the dual-camera frame synchronization processing method according to the first aspect of the present application when executing a computer program.

[0037] To achieve the above-mentioned purpose, according to an embodiment of the fourth aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the dual-camera frame synchronization processing method according to the first aspect of the present application is implemented.

[0038] The present invention is further described below in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 This is a first principle schematic diagram of a dual-camera frame synchronization processing method according to an embodiment of the present application;

[0041] Figure 2 This is a second principle diagram of the dual-camera frame synchronization processing method according to an embodiment of the present application.

[0042] Figure 3 Schematic diagram of the principle of the pairing determination step of the embodiment of the present application.

[0043] Figure 4 Schematic diagram of a dual-camera frame synchronization processing device according to an embodiment of the present application.

[0044] Figure 5 A block diagram of an exemplary user terminal for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0046] The terms used in the embodiments of the present application will be introduced below.

[0047] Frame rate: is a measure used to measure the number of displayed frames.

[0048] Resolution: refers to the amount of information stored in an image, which is the number of pixels per inch of the image.

[0049] Timestamp: It is the data generated by digital signature technology. The signature object includes the original file information, signature parameters, signature time and other information.

[0050] Existing camera devices use hard synchronization to achieve frame synchronization between infrared images and visible light images. Specifically, the same hardware is used to simultaneously issue trigger acquisition commands to achieve time synchronization of acquisition and measurement of each sensor. For example, infrared cameras and visible light cameras use the same hardware pulse signal to achieve simultaneous exposure control. This method can only process infrared cameras and visible light cameras with the same frame rate. Moreover, due to inconsistent resolutions, there will be a certain delay in data transmission for infrared cameras and visible light cameras with the same frame rate. The sampled image size data is inconsistent, which will lead to certain differences in the timestamps of the images sampled by the two. If hard synchronization is used to process infrared cameras and visible light cameras with different frame rates, due to inconsistent frame rates and resolutions, data transmission will inevitably be inconsistent, and there will be large differences in the timestamps of the images sampled by the two.

[0051] The implementation environment involved in this application includes: target scenes and terminal devices with infrared cameras and visible light cameras. The terminal devices include mobile phones, computers, tablet computers and infrared thermal imagers. The dual-camera frame synchronization processing method of this application is mainly used in infrared thermal imagers. After the infrared camera and visible light camera of the infrared thermal imager acquire images, a hook function is used to retrieve infrared images and visible light images with consistent timestamps, and then the infrared images and visible light images are processed.

[0052] Figure 1 This is a first principle schematic diagram of the dual-camera frame synchronization processing method according to an embodiment of the present application. Figure 1 As shown, an embodiment of the first aspect of the present application provides a dual-camera frame synchronization processing method, including an infrared camera and a visible light camera, including:

[0053] Step 100 is a parameter acquisition step: acquiring the first camera parameter of the infrared camera and the second camera parameter of the visible light camera, and generating a first gap trigger time and a second gap trigger time according to the first camera parameter and the second camera parameter, wherein the first camera parameter includes a first frame rate, a first resolution and a first image processing bandwidth, and the second camera parameter includes a second frame rate, a second resolution and a second image processing bandwidth.

[0054] In step 100, the first camera parameter of the infrared camera is a specific parameter of the infrared camera. Similarly, the second camera parameter of the visible light camera is also a specific parameter of the visible light camera. The first gap trigger time and the second gap departure time are generated by obtaining the parameters of the infrared camera and the visible light camera and calculating according to the hook function.

[0055] The first gap trigger time refers to the gap time between two adjacent infrared image data frames taken out by the hook function in the first data buffer pool. The second gap trigger time refers to the gap time between two adjacent visible light image data frames taken out by the hook function in the second data buffer pool. In this way, the search rate of the hook function in the data buffer pool can be improved.

[0056] The calculation formula of the hook function is:

[0057]

[0058] When the frame rates of the infrared camera and the visible light camera are the same, take the visible light camera as an example: the processing bandwidth is the second image processing bandwidth, the frame rate is set to the second frame rate, the resolution is the second resolution, and the trigger frequency is the sum of the reciprocal of the first frame rate and the reciprocal of the second frame rate.

[0059] When the frame rates of the infrared camera and the visible light camera are the same, take the infrared camera as an example: the processing bandwidth is the first image processing bandwidth, the setting frequency is the first frequency, the resolution is the first resolution, and the trigger frequency is the sum of the reciprocal of the first frame rate and the reciprocal of the second frame rate.

[0060] When the frame rates of the infrared camera and the visible light camera are different, taking the visible light camera as an example, the processing bandwidth is the second image processing bandwidth, the frame rate is set to the second frame rate, the resolution is the second resolution, and the trigger frequency is the sum of the reciprocal of the first frame rate and the reciprocal of the second frame rate.

[0061] When the frame rates of the infrared camera and the visible light camera are different, take the infrared camera as an example: the processing bandwidth is the first image processing bandwidth, the setting frequency is the first frequency, the resolution is the first resolution, and the trigger frequency is the sum of the reciprocal of the first frame rate and the reciprocal of the second frame rate.

[0062] Therefore, the intermittent triggering time of the hook function is determined by the first camera parameter and the second camera parameter.

[0063] Step 200 is an infrared image data acquisition step: acquiring an infrared image data frame and storing the infrared image data frame in a first data buffer pool, wherein the infrared image data frame has a first timestamp.

[0064] In step 200, an infrared image data frame is captured by an infrared camera. After the infrared image data frame is captured, the infrared image data frame is stored in a first data buffer pool, wherein the first data buffer pool is specifically used to store infrared image data frames. A first timestamp is generated on each infrared image data frame.

[0065] Step 300 is a visible light image data acquisition step: acquiring a visible light image data frame and storing the visible light image data frame in a second data buffer pool, wherein the visible light image data frame has a second timestamp.

[0066] In step 300, a visible light image data frame is captured by a visible light camera. After the visible light image data frame is acquired, the visible light image data frame is stored in a second data buffer pool, wherein the second data buffer pool is specifically used to store visible light image data frames. A second timestamp is generated on each visible light image data frame.

[0067] Step 400 is a timestamp retrieval step: taking out infrared image data frames from a first data buffer pool based on a first gap trigger time, and taking out visible light image data frames from a second data buffer pool based on a second gap trigger time.

[0068] In step 400, when an infrared image data frame is stored in the first data buffer pool, the hook function starts to capture the infrared image data frame in the first data buffer pool, and captures the infrared image data frame once, and then captures the infrared image data frame again after the first gap trigger time, and repeats this cycle until there are no infrared image data frames in the first data buffer pool. When a visible light image data frame is stored in the second data buffer pool, the principle is the same as when an infrared image data frame is stored in the first data buffer pool, and will not be repeated here. In this way, by setting the first gap trigger time and the second gap trigger time, data can be taken out from the first data buffer pool and the second data buffer pool separately and quickly, thereby improving the acquisition rate of infrared image data frames and visible light image data frames.

[0069] Step 500 is a pairing determination step: determining whether the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame. If so, optimizing the extracted infrared image data frame and the visible light image data frame.

[0070] In step 500, it is determined whether the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the visible light image data frame. If they are the same, optimization processing is started on the extracted infrared image data frame and the visible light image data frame. If they are not the same, the extracted infrared image data frame and the visible light image data frame are put back into the corresponding first data buffer pool and the second data buffer pool so that the subsequent hook function can re-capture them.

[0071] Figure 2 This is a second principle diagram of the dual-camera frame synchronization processing method according to an embodiment of the present application. Figure 3 FIG. 1 is a schematic diagram showing the principle of the pairing determination step in the embodiment of the present application. Figure 2 and Figure 3 As shown, in some other embodiments, the pairing determination step further includes:

[0072] When the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame, generating an identification mark on the infrared image data frame and the visible light image data frame;

[0073] Step 510 is an identification step: identifying whether there is an identification mark on the infrared image data frame and the visible light image data frame;

[0074] If yes, stop the timestamp retrieval step.

[0075] In this embodiment, the identification mark is used to identify whether the infrared image data frame and the visible light image data frame have completed the timestamp pairing, and the timestamp retrieval step is stopped according to the identification mark, that is, the hook function is stopped from continuing to take out the infrared image data frame and the visible light image data frame from the first data buffer pool and the second data buffer pool. Specifically, when the first timestamp of the taken infrared image data frame is different from the second timestamp of the taken visible light image data frame, the identification mark will not be generated on the infrared image data frame and the visible light image data frame.

[0076] In other embodiments, after optimizing the infrared image data frame and the visible light image data frame with the identification mark, the identification mark of the infrared image data frame and the visible light image data frame is eliminated;

[0077] Repeat the timestamp retrieval step.

[0078] In this embodiment, after the infrared image data frame and the visible light image data frame with the identification mark are optimized, the identification mark of the infrared image data frame and the visible light image data frame is removed. The identification mark is identified again, and if the identification mark is not identified, the timestamp retrieval step is performed again.

[0079] In some other embodiments, if the first data buffer pool is not empty, then starting the timestamp retrieval step;

[0080] If the second data buffer pool is not empty, the timestamp retrieval step is started.

[0081] In this embodiment, the hook function captures data in the first data buffer pool and the second data buffer pool respectively. In this way, when the first data buffer pool and / or the second data buffer pool is empty, the hook function can quickly respond to capture data, thereby improving the capture efficiency of infrared image data frames and visible light image data frames.

[0082] In some other embodiments, the timestamp retrieval step further includes: a timeout range interval;

[0083] After the infrared image data frame is taken out from the first data buffer pool based on the first gap trigger time, a waiting time is generated;

[0084] or, after taking out the visible light image data frame from the second data buffer pool based on the second gap trigger time, generating a waiting time;

[0085] If the waiting time exceeds the timeout range, the timestamp retrieval step is performed again.

[0086] In this embodiment, by setting the timeout range interval, it is helpful to shorten the waiting time after the infrared image data frame or the visible light image data frame is taken out, thereby reducing the problem of occupying processing resources.

[0087] In some other embodiments, if the first frame rate is the same as the second frame rate, the timeout range is 0 to 50 ms.

[0088] In some other embodiments, if the first frame rate is different from the second frame rate, the timeout range is 0 to 100 ms.

[0089] like Figure 4 As shown, according to an embodiment of the second aspect of the present application, a dual-camera frame synchronization processing device 600 is provided, comprising:

[0090] The parameter acquisition module 610 is used to acquire the first camera parameter of the infrared camera and the second camera parameter of the visible light camera, and generate the first gap trigger time and the second gap trigger time according to the first camera parameter and the second camera parameter, wherein the first camera parameter includes a first frame rate, a first resolution and a first image processing bandwidth, and the second camera parameter includes a second frame rate, a second resolution and a second image processing bandwidth;

[0091] A first acquisition module 620, for acquiring an infrared image data frame and storing the infrared image data frame in a first data buffer pool, wherein the infrared image data frame has a first timestamp;

[0092] A second acquisition module 630, configured to acquire a visible light image data frame and store the visible light image data frame in a second data buffer pool, wherein the visible light image data frame has a second timestamp;

[0093] A timestamp retrieval module 640, configured to retrieve infrared image data frames from a first data buffer pool based on a first gap trigger time, and retrieve visible light image data frames from a second data buffer pool based on a second gap trigger time;

[0094] The pairing determination module 650 is used to determine whether the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame.

[0095] The dual-camera frame synchronization processing method or device of the present application adopts a soft synchronization method to more concisely and efficiently process the frame synchronization problems of infrared images and visible light images, and can process the frame synchronization problems of binocular cameras with different frame rates or the same frame rates, thereby improving the consistency and timeliness of infrared image and visible light image processing.

[0096] Figure 4 This is a schematic block diagram of a dual-camera frame synchronization processing device according to an embodiment of the present application. For more specific implementation methods of the various modules of the dual-camera frame synchronization processing device 600, please refer to the description of the dual-camera frame synchronization processing method based on the present invention, and it has similar beneficial effects, which will not be repeated here.

[0097] According to an embodiment of the third aspect of the present application, there is provided a user terminal, including a memory and a processor;

[0098] Storage for storing computer programs;

[0099] A processor is used to implement the dual-camera frame synchronization processing method according to the first aspect of the present application when executing a computer program.

[0100] The user terminal of the third aspect of the present application can be implemented with reference to the contents specifically described in the embodiment of the first aspect of the present application, and has similar beneficial effects as the dual-camera frame synchronization processing method according to the embodiment of the first aspect of the present application, which will not be repeated here.

[0101] Figure 5 A block diagram of an exemplary user terminal for implementing the embodiments of the present application is shown in FIG. Figure 5 The user terminal shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0102] like Figure 5 As shown, the user terminal 10 can be implemented as a general computing device. The components of the user terminal 10 may include but are not limited to: one or more processors or processing units 11, a system memory 12, and a bus 13 connecting different system components (including the system memory 12 and the processing unit 11).

[0103] The bus 13 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (hereinafter referred to as: ISA) bus, Micro Channel Architecture (hereinafter referred to as: MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (hereinafter referred to as: VESA) local bus and Peripheral Component Interconnection (hereinafter referred to as: PCI) bus.

[0104] The user terminal 10 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the user terminal 10, including volatile and non-volatile media, removable and non-removable media.

[0105] The memory 12 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 14 and / or cache memory 15. The user terminal 10 may further include other removable / non-removable, volatile / non-volatile computer readable storage media. By way of example only, the storage system 16 may be used to read and write non-removable, non-volatile magnetic media (not shown in the figure, usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the bus 13 via one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0106] A program / utility 18 having a set (at least one) of program modules 17 may be stored in, for example, a memory, such program modules 17 including but not limited to an operating system, one or more application programs, other program modules 17, and program data, each of which or some combination may include an implementation of a network environment. The program modules 17 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0107] The user terminal 10 may also communicate with one or more external devices 19 (e.g., keyboard, pointing device, display 20, etc.), one or more devices that enable a user to interact with the computer system / server, and / or any device that enables the computer system / server to communicate with one or more other user terminals 10 (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface 21. Furthermore, the user terminal 10 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 22. As shown in the figure, the network adapter 22 communicates with other modules of the user terminal 10 via the bus 13. It should be noted that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the user terminal 10, including but not limited to: microcode, device drivers, redundant processing units 11, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 16, etc.

[0108] The processing unit 11 executes various functional applications and data processing by running programs stored in the system memory 12, such as implementing the methods mentioned in the above embodiments.

[0109] The user terminal 10 in the embodiment of the present application may be a server or a terminal device with limited computing power.

[0110] According to an embodiment of the fourth aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the dual-camera frame synchronization processing method according to the first aspect of the present application is implemented.

[0111] Generally speaking, the computer instructions for implementing the method of the present invention can be carried by any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media can include any computer-readable media except for the signal itself that is temporarily propagating.

[0112] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM14), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0113] Computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages, in particular, Python suitable for neural network computing and platform frameworks based on TensorFlow, PyTorch, etc. can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0114] The above are only preferred embodiments of the present application, and do not constitute any formal limitation on the present application. Any technician familiar with the art can use the above disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present application without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, all equivalent changes made according to the shape, structure and principle of the present application without departing from the content of the technical solution of the present application should be included in the protection scope of the present application.

Claims

1. A dual-camera frame synchronization processing method, characterized in that: include: Parameter acquisition step: acquiring a first camera parameter of the infrared camera and a second camera parameter of the visible light camera, and generating a first gap trigger time and a second gap trigger time according to the first camera parameter and the second camera parameter, wherein the first camera parameter includes a first frame rate, a first resolution and a first image processing bandwidth, and the second camera parameter includes a second frame rate, a second resolution and a second image processing bandwidth; Infrared image data acquisition step: acquiring an infrared image data frame and storing the infrared image data frame in a first data buffer pool, the infrared image data frame having a first timestamp; Visible light image data acquisition step: acquiring a visible light image data frame and storing the visible light image data frame in a second data buffer pool, the visible light image data frame having a second timestamp; Timestamp retrieval step: taking out infrared image data frames from the first data buffer pool based on the first gap trigger time, and taking out visible light image data frames from the second data buffer pool based on the second gap trigger time; The first gap trigger time is the gap time when the hook function takes out two adjacent frames of infrared image data frames from the first data buffer pool; the second gap trigger time is the gap time when the hook function takes out two adjacent frames of visible light image data frames from the second data buffer pool; The calculation formula of the hook function is: Pairing judgment step: judging whether the first timestamp of the infrared image data frame taken out is the same as the second timestamp of the visible light image data frame taken out, and if so, optimizing the infrared image data frame and the visible light image data frame taken out; The pairing determination step further includes: When the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame, generating an identification mark on the infrared image data frame and the visible light image data frame; Identify whether the infrared image data frame and the visible light image data frame have identification marks; If yes, stop the timestamp retrieval step.

2. The dual-camera frame synchronization processing method according to claim 1, characterized in that: After optimizing the infrared image data frame and the visible light image data frame with the identification mark, the identification marks of the infrared image data frame and the visible light image data frame are eliminated; Repeat the timestamp retrieval step.

3. The dual-camera frame synchronization processing method according to claim 1, characterized in that: If the first data buffer pool is not empty, then start the timestamp retrieval step; If the second data buffer pool is not empty, the timestamp retrieval step is started.

4. The dual-camera frame synchronization processing method according to claim 1, characterized in that: The timestamp retrieval step also includes: a timeout range interval; After the infrared image data frame is taken out from the first data buffer pool based on the first gap trigger time, a waiting time is generated; or, after taking out the visible light image data frame from the second data buffer pool based on the second gap trigger time, generating a waiting time; If the waiting time exceeds the timeout range, the timestamp retrieval step is performed again.

5. The dual-camera frame synchronization processing method according to claim 4, characterized in that: If the first frame rate is the same as the second frame rate, the timeout range is 0 to 50 ms.

6. The dual-camera frame synchronization processing method according to claim 4, characterized in that: If the first frame rate is different from the second frame rate, the timeout range is 0 to 100 ms.

7. A dual-camera frame synchronization processing device implementing the dual-camera frame synchronization processing method of claim 1, characterized in that: include: A parameter acquisition module, used to acquire a first camera parameter of the infrared camera and a second camera parameter of the visible light camera, and generate a first gap trigger time and a second gap trigger time according to the first camera parameter and the second camera parameter, wherein the first camera parameter includes a first frame rate, a first resolution and a first image processing bandwidth, and the second camera parameter includes a second frame rate, a second resolution and a second image processing bandwidth; A first acquisition module, used for acquiring an infrared image data frame and storing the infrared image data frame in a first data buffer pool, wherein the infrared image data frame has a first timestamp; A second acquisition module, configured to acquire a visible light image data frame and store the visible light image data frame in a second data buffer pool, wherein the visible light image data frame has a second timestamp; A timestamp retrieval module, configured to retrieve infrared image data frames from the first data buffer pool based on the first gap trigger time, and retrieve visible light image data frames from the second data buffer pool based on the second gap trigger time; The pairing determination module is used to determine whether the first timestamp of the extracted infrared image data frame is the same as the second timestamp of the extracted visible light image data frame.

8. A user terminal, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the dual-camera frame synchronization processing method as described in any one of claims 1 to 6 when executing the computer program.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the dual-camera frame synchronization processing method as described in any one of claims 1 to 6 is implemented.

Images