Camera data transmission method and device based on kernel drive and related product

Abstract

The invention provides a camera data transmission method and device based on a kernel driver and a related product, and the method and device can create an independent memory space for a camera based on the kernel driver. According to the method, the camera data is acquired and cached to the pre-created memory space, and finally the camera data is acquired in the memory space, that is, the camera data can be acquired in the pre-created memory space, and the data packet is transferred to the business layer through the kernel driving layer, so that system call caused by traditional use of socket is avoided, and the system performance is improved. 80000 + times of system calling can be reduced for a single picture, 80000000 + CPU instructions can be reduced, system calling is reduced, switching between a user mode and a kernel mode is reduced, and CPU consumption is reduced.

Description

[Technical field]

[0001] The present invention relates to the technical field of industrial cameras, and in particular to a camera data transmission method and device based on kernel driver and related products. [Background technology]

[0002] Industrial cameras based on the GIGE VERSION protocol transmit data to industrial computers through the network port. When the shooting accuracy is higher, the camera pixel is also required to be larger, and the transmitted data will also be larger. Taking an 8K linear array black and white camera as an example, a single scan has 10,000 lines, and the size of a single picture is 8192*10000=80MB. The picture data is transmitted from the camera to the industrial computer through the UDP protocol. If the size of a single UDP data packet is 1024 bytes, then a picture needs to be split into 80,000 UDP data packets for transmission. Figure 1 As shown in the figure, when the industrial computer uses the traditional method (such as Socket) to receive UDP data packets, each image will cause more than 80,000 system calls. Each system call will switch between user mode and kernel mode, and will execute about 1,000 CPU instructions, which seriously consumes CPU resources. Figure 2 As shown, in the prior art, the CPU occupancy is 25.8% during camera data transmission, which is relatively high.

[0003] In view of this, it is necessary to provide a new camera data transmission method, device and related products based on kernel driver to overcome the above defects. [Summary of the invention]

[0004] The purpose of the present invention is to provide a camera data transmission method, device and related products based on kernel driver, which can reduce system calls and reduce CPU consumption.

[0005] In order to achieve the above object, in a first aspect, the present invention provides a camera data transmission method based on kernel driver, comprising the following steps:

[0006] Create independent memory space for the camera based on kernel driver;

[0007] Receive camera data transmitted from the camera network port to the industrial computer network port;

[0008] Acquire the camera data and cache the camera data into a pre-created memory space;

[0009] The camera data is obtained in the memory space.

[0010] In a preferred embodiment, the step of creating an independent memory space for the camera based on the kernel driver includes:

[0011] Create a character device driver module;

[0012] When the business application program calls to open the character device driver module, the memory space is allocated;

[0013] An ordered circular buffer queue is constructed in the memory space.

[0014] In a preferred embodiment, after the step of creating a character device driver module, the following steps are included:

[0015] When the service application calls to close the character device driver module, the memory space is released.

[0016] In a preferred embodiment, when the service application calls to open the character device driver module, after the step of allocating the memory space, the following steps are included:

[0017] The address of the memory space is mapped to the user state of the business application program for access by the business application program.

[0018] In a preferred embodiment, the step of acquiring the camera data and caching the camera data into a pre-created memory space comprises:

[0019] Hook the data packets that business applications need to capture;

[0020] Delete the IP header and UDP header of the data packet and then add the private header;

[0021] The data packet added with the private packet header is cached in the ring buffer queue.

[0022] In a preferred embodiment, the step of acquiring the camera data in the memory space includes:

[0023] The service application obtains the memory address of the ring buffer queue, and obtains the data packet from the ring buffer queue based on a private packet header protocol.

[0024] In a preferred embodiment, the camera data is photo data.

[0025] In a second aspect, the present invention provides a camera data transmission device based on kernel driver, comprising:

[0026] The memory space creation module is used to create an independent memory space for the camera based on the kernel driver;

[0027] A camera data receiving module is used to receive camera data transmitted from the camera network port to the industrial computer network port;

[0028] A camera data cache module, used for acquiring the camera data and caching the camera data into a pre-created memory space;

[0029] A camera data acquisition module is used to acquire the camera data in the memory space.

[0030] In a third aspect, the present invention further provides a computer device, comprising: a memory and one or more processors; the memory is used to store one or more computer programs; when the one or more computer programs are executed by the one or more processors, the kernel-driven camera data transmission method described in any embodiment of the first aspect of the present invention is implemented.

[0031] In a fourth aspect, the present invention further provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the kernel-driven camera data transmission method described in any embodiment of the first aspect of the present invention is implemented.

[0032] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the kernel-driven camera data transmission method described in any embodiment of the first aspect of the present invention.

[0033] Compared with the prior art, the kernel-driven camera data transmission method, device and related products provided by the present invention can create an independent memory space for the camera based on the kernel driver, and when receiving camera data transmitted from the camera network port to the industrial computer network port, obtain the camera data and cache the camera data to a pre-created memory space, and finally obtain the camera data in the memory space, that is, the camera data can be obtained in the pre-created memory space, and the data packet is transferred to the business layer through the kernel driver layer, thereby avoiding the system call caused by the traditional use of sockets. A single picture can reduce 80,000+ system calls, reduce 80,000,000+ CPU instructions, reduce system calls, reduce switching between user mode and kernel mode, and reduce CPU consumption.

Brief Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 It is a structural diagram of data transmission between a camera and an industrial computer in the prior art;

[0036] Figure 2 for Figure 1 The structure shown is a schematic diagram of CPU usage when transmitting camera data;

[0037] Figure 3 A principle block diagram of a camera data transmission device based on a kernel driver provided by the present invention;

[0038] Figure 4 A flowchart of a camera data transmission method based on a kernel driver provided by the present invention;

[0039] Figure 5 This is a structural diagram of the camera and the industrial computer transmitting data in the present invention;

[0040] Figure 6 for Figure 5 The structure shown is a schematic diagram of CPU usage when transmitting camera data;

[0041] Figure 7 This is a functional block diagram of the computer device provided by the present invention. [Specific implementation method]

[0042] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0043] See also Figure 3 , which is a principle block diagram of a camera data transmission device based on kernel driver provided by the present invention. The camera data transmission device based on kernel driver 100 provided by the present invention comprises a memory space creation module 10, a camera data receiving module 20, a camera data buffer module 30 and a camera data acquisition module 40.

[0044] The memory space creation module 10 is used to create an independent memory space for the camera based on the kernel driver; the camera data receiving module 20 is used to receive the camera data transmitted from the camera network port to the industrial computer network port; the camera data cache module 30 is used to obtain the camera data and cache the camera data to the pre-created memory space; the camera data acquisition module 40 is used to obtain the camera data in the memory space. Specifically, the camera data is photo data.

[0045] In some implementations, the memory space creation module 10 is used to create an independent memory space for the camera based on the kernel driver, including:

[0046] Create a character device driver module;

[0047] When the business application program calls to open the character device driver module, the memory space is allocated;

[0048] An ordered circular buffer queue is constructed in the memory space.

[0049] In some implementations, after creating a character device driver module, the following steps are included:

[0050] When the service application calls to close the character device driver module, the memory space is released.

[0051] In some implementations, when the service application calls to open the character device driver module, after the memory space is allocated, the following steps are performed:

[0052] The address of the memory space is mapped to the user state of the business application program for access by the business application program.

[0053] In some implementations, the camera data acquisition module 40 acquires the camera data and caches the camera data into a pre-created memory space, including:

[0054] Hook the data packets that business applications need to capture;

[0055] Delete the IP header and UDP header of the data packet and then add the private header;

[0056] The data packet added with the private packet header is cached in the ring buffer queue.

[0057] In some implementations, the camera data acquisition module 40 acquires the camera data in the memory space, including:

[0058] The service application obtains the memory address of the ring buffer queue, and obtains the data packet from the ring buffer queue based on a private packet header protocol.

[0059] It is understandable that the above functional modules can be stored in the memory in the form of software programs and executed by the processor. In alternative embodiments, the above functional modules can also be hardware with specific functions, such as a chip with a specific software program burned in it.

[0060] Combine the following Figure 4 A detailed introduction is given to the above functional modules.

[0061] like Figure 4As shown, it is a flow chart of the kernel-driven camera data transmission method provided by the present invention. It should be noted that the method of the present invention is not limited to the order of the following steps, and in other embodiments, the method of the present invention may include only part of the following steps, or some of the steps may be deleted. In implementation, the kernel-driven camera data transmission method can be implemented by the kernel-driven camera data transmission device, and the kernel-driven camera data transmission device adopts one or more specific implementations of the kernel-driven camera data transmission method described in this embodiment to realize data transmission between the camera and the industrial computer.

[0062] Specifically, in the prior art, industrial computers use traditional Sockets to receive UDP data packets. Each image will cause more than 80,000 system calls. Each system call will switch between user mode and kernel mode, and will execute about 1,000 CPU instructions, which seriously consumes CPU resources. That is, a large image will be divided into many data packets, and the industrial computer service will receive data packets in a traditional way, which will consume a lot of CPU. It can be understood that Socket refers to two programs on the network that exchange data through a two-way communication connection. One end of this connection is called a socket, which can be used to realize communication between different virtual machines or different computers.

[0063] About the user state and kernel state of the system: User state is the running state of the application. When the application runs in user state, its permissions are strictly limited. Programs in user state cannot directly access hardware devices or execute privileged instructions. All access to hardware must be performed through the operating system. Programs in user state request services provided by the operating system through system calls, such as file operations and network communications. Kernel state is the running state of the operating system kernel. The CPU in kernel state can execute all instructions, access all memory addresses, and has the highest permissions. Programs in kernel state can access all resources of the system, including CPU, memory, and I / O devices. Kernel state is responsible for processing interrupt requests, system calls, and hardware events to ensure the stable operation of the system. User state and kernel state together constitute the operating environment of the operating system. User state programs interact with kernel state through system calls to complete access to hardware resources and service requests of the operating system. This design ensures the security and stability of the system. User state programs request operating system services through system calls, and the CPU switches to kernel state to perform the corresponding operations and returns to user state after completion. When an exception or interrupt occurs when the CPU is executing a user state program (such as a peripheral device interrupt), the CPU will switch to kernel state to handle these events and return to user state after processing.

[0064] The camera data transmission method based on kernel driver provided by the present invention can be applied to the operating system of industrial computers, such as Linux system. The camera data transmission method based on kernel driver includes the following steps:

[0065] Step S10: Creating an independent memory space for the camera based on the kernel driver. Specifically, the memory space creating module 10 can create an independent memory space for the camera based on the kernel driver.

[0066] In some implementations, the step of creating an independent memory space for the camera based on the kernel driver includes:

[0067] Create a character device driver module; specifically, the character device driver module is also the kernel driver;

[0068] When the business application program calls to open the character device driver module, the memory space is allocated;

[0069] An ordered circular buffer queue is constructed in the memory space.

[0070] In some implementations, after the step of creating a character device driver module, the following steps are included:

[0071] When the service application calls to close the character device driver module, the memory space is released.

[0072] In some implementations, when the service application calls to open the character device driver module, after the step of allocating the memory space, the following steps are included:

[0073] The address of the memory space is mapped to the user state of the business application program for access by the business application program.

[0074] Step S20: receiving camera data transmitted from the camera network port to the industrial computer network port. Specifically, the camera data receiving module 20 can receive camera data transmitted from the camera network port to the industrial computer network port. The camera data receiving module can be, for example, a network card, and the camera data is photo data.

[0075] Step S30: Acquire the camera data and cache the camera data in a pre-created memory space. Specifically, the camera data cache module 30 can acquire the camera data and cache the camera data in a pre-created memory space.

[0076] In some implementations, the step of acquiring the camera data and caching the camera data into a pre-created memory space includes:

[0077] Hook the data packets that the business application needs to capture; specifically, the data packets are the transmitted photo data;

[0078] Delete the IP header and UDP header of the data packet and then add the private header;

[0079] The data packet added with the private packet header is cached in the ring buffer queue.

[0080] Step S40: Acquire the camera data in the memory space. Specifically, the camera data acquisition module can acquire the camera data in the memory space.

[0081] In some implementations, the step of acquiring the camera data in the memory space includes:

[0082] The service application obtains the memory address of the ring buffer queue, and obtains the data packet from the ring buffer queue based on a private packet header protocol.

[0083] The camera data transmission method based on kernel driver provided by the present invention can create an independent memory space for the camera based on kernel driver, and when receiving camera data transmitted from the camera network port to the industrial computer network port, the camera data is obtained and cached in a pre-created memory space, and finally the camera data is obtained in the memory space, that is, the camera data can be obtained in the pre-created memory space, and the data packet is transferred to the business layer through the kernel driver layer, thereby avoiding the system call caused by the traditional use of the socket, and a single picture can reduce 80,000+ system calls, reduce 80,000,000+ CPU instructions, reduce system calls, reduce the switching between user state and kernel state, and reduce CPU consumption.

[0084] The camera data transmission method based on kernel driver provided by the present invention reduces CPU consumption by focusing on reducing system calls and reducing the switching between user state and kernel state. The present invention uses kernel driver technology to develop a character device driver program that resides in the operating system. The specific method is as follows:

[0085] 1) First create a character device driver. The business application (for example, a coil material detection business program) creates an independent memory area for each camera. When a camera is associated, the driver is opened once, and the independent memory space of the camera is opened. When each camera closes the driver, its own memory space is also released. Specifically, this logic can be implemented through the open and release functions in the kernel file structure file_operations. Business applications can generally call the open and close functions to open and close the driver.

[0086] 2) When the business application calls to open the driver, the driver uses vmalloc_user to open a large memory block. In this way, a large memory block can be opened, that is, memory space is allocated, and this memory address can be mapped to the user state of the business application through mmap for access by the business application. After the camera creates its own memory area, it uses the head pointer and tail pointer technology in this memory area to build an orderly circular buffer queue for subsequent processing of data packets.

[0087] 3) Use the hook mechanism of the Linux kernel protocol stack netfilter and the nf_hook_ops hook function to hook out the data packets that the business application needs to capture, remove the redundant IP header and UDP header (28 bytes), and then add the private header (8 bytes) to ensure that the business application can receive the data packets in an orderly manner. Finally, cache the data packets in the buffer queue of this camera for the business application to obtain.

[0088] 4) The business application uses mmap to obtain the memory address of the ring queue of this camera at the driver layer, and obtains the data packet from the ring queue according to the private packet header protocol.

[0089] See also Figure 5 , Figure 5 This is a structural diagram of the data transmission between the camera and the industrial computer in the present invention. Specifically, after the camera is triggered to take a photo, the photo data is transmitted from the camera network port to the industrial computer network port through the network; the kernel driver obtains the photo data from the network card and then caches it in an independent memory space; the business application obtains the photo data in the driver memory space.

[0090] See also Figure 2 and Figure 6 , Figure 2 for Figure 1 The structure shown is a schematic diagram of CPU usage when transmitting camera data. Figure 6 for Figure 5 The structure shown is a schematic diagram of CPU usage when transmitting camera data. Figure 2 and Figure 6 , where PR represents the process priority, NI represents the nice priority switching level, VIRT represents the size of virtual memory, RES represents the size of resident memory, SHR represents the size of shared memory, %CPU represents the percentage of CPU time usage, and %MEM represents the percentage of physical memory used. The advantage of the present invention after optimization is that the CPU consumption is reduced from 25.8% ( Figure 2 ) dropped to 13.9% ( Figure 6 ).

[0091] See also Figure 7The present invention further provides a computer device, including: a memory 210 and one or more processors 220.

[0092] Specifically, the memory 210 is used to store one or more computer programs; when the one or more computer programs are executed by one or more processors 220, the camera data transmission method based on kernel driver described in any one of the above implementations is implemented.

[0093] The memory 210 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 210 is used to store programs, and the processor 220 runs the program after receiving an execution instruction to implement the camera data transmission method based on kernel drive described in any of the above embodiments. It can be understood that the processor 220 and other possible components can access the memory 210 under the control of the storage controller.

[0094] The processor 220 may be an integrated circuit chip with signal processing capabilities. The processor 220 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., and may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the disclosed methods and steps in any embodiment of the present invention.

[0095] The present invention also provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the camera data transmission method based on kernel driving described in any of the above embodiments is implemented.

[0096] The present invention also provides a computer program product, including a computer program or an instruction, and when the computer program or the instruction is executed by a processor, the camera data transmission method based on kernel driving described in any of the above embodiments is implemented.

[0097] In summary, the kernel-driven camera data transmission method, device and related products provided by the present invention can create an independent memory space for the camera based on the kernel driver, and when receiving camera data transmitted from the camera network port to the industrial computer network port, obtain the camera data and cache the camera data to a pre-created memory space, and finally obtain the camera data in the memory space, that is, the camera data can be obtained in the pre-created memory space, and the data packet is transferred to the business layer through the kernel driver layer, thereby avoiding the system call caused by the traditional use of sockets. A single picture can reduce 80,000+ system calls, reduce 80,000,000+ CPU instructions, reduce system calls, reduce switching between user mode and kernel mode, and reduce CPU consumption.

[0098] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A camera data transmission method based on kernel driver, characterized in that: The steps include: Create independent memory space for the camera based on kernel driver; Receive camera data transmitted from the camera network port to the industrial computer network port; Acquire the camera data and cache the camera data into a pre-created memory space; The camera data is obtained in the memory space.

2. The camera data transmission method based on kernel driver according to claim 1, characterized in that: The steps to create an independent memory space for the camera based on the kernel driver include: Create a character device driver module; When the business application program calls to open the character device driver module, the memory space is allocated; An ordered circular buffer queue is constructed in the memory space.

3. The camera data transmission method based on kernel driver according to claim 2, characterized in that: The steps to create a character device driver module include: When the service application calls to close the character device driver module, the memory space is released.

4. The camera data transmission method based on kernel driver according to claim 2, characterized in that: When the service application calls to open the character device driver module, after the step of allocating the memory space, the method includes: The address of the memory space is mapped to the user state of the business application program for access by the business application program.

5. The camera data transmission method based on kernel driver according to claim 2, characterized in that: The step of acquiring the camera data and caching the camera data into a pre-created memory space comprises: Hook the data packets that business applications need to capture; Delete the IP header and UDP header of the data packet and then add the private header; The data packet added with the private packet header is cached in the ring buffer queue.

6. The camera data transmission method based on kernel driver according to claim 5, characterized in that: The step of acquiring the camera data in the memory space comprises: The service application obtains the memory address of the ring buffer queue, and obtains the data packet from the ring buffer queue based on a private packet header protocol.

7. The camera data transmission method based on kernel driver according to any one of claims 1 to 6, characterized in that: The camera data is photo data.

8. A camera data transmission device based on kernel driver, characterized in that: include: The memory space creation module is used to create an independent memory space for the camera based on the kernel driver; A camera data receiving module is used to receive camera data transmitted from the camera network port to the industrial computer network port; A camera data cache module, used for acquiring the camera data and caching the camera data into a pre-created memory space; A camera data acquisition module is used to acquire the camera data in the memory space.

9. A computer device comprising: memory and one or more processors; The memory is used to store one or more computer programs; characterized in that when the one or more computer programs are executed by the one or more processors, the kernel-driven camera data transmission method according to any one of claims 1 to 7 is implemented.

10. A computer storage medium having a computer program stored therein, characterized in that: When the computer program is executed by a processor, the kernel-driven camera data transmission method according to any one of claims 1 to 7 is implemented.

Images