Airport monitoring video transmission method and device, storage medium and electronic equipment

Abstract

The invention provides an airport monitoring video transmission method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining the comprehensive complexity corresponding to each coding tree unit in each frame of image in an image group, and accurately dividing the current frame of image into a key region and a non-key region based on a preset condition; according to the comprehensive complexity of the kth merging area in the current frame image and the total bit budget of the current frame image, determining the allocation bit number corresponding to the kth merging area in the current frame image; coding the pixel points in the kth merging area based on the allocation bit number corresponding to the kth merging area; and after all frame images in the image group are coded, transmitting coded data. Dynamic bit allocation is carried out according to the comprehensive complexity of the key area and the non-key area, the coding accuracy of the key area is improved, and the definition and the overall monitoring efficiency of the key area are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of images, and in particular to an airport monitoring video transmission method, device, storage medium and electronic equipment. Background Art

[0002] In recent years, thanks to the rapid economic development, people's living standards have improved significantly, and the number of people who choose to travel by plane has continued to increase. As a vital transportation hub, the airport and the flight area are of utmost importance, and its safety issues have always received widespread attention.

[0003] With the rapid development of intelligent monitoring technology, as an area with high security requirements, airports generally install high-resolution, multi-angle monitoring systems to achieve real-time monitoring of key areas such as runways, aprons, and terminals. However, the existing bandwidth is usually not enough to support the transmission of high-resolution video. The bit allocation method in bit rate control is a key link in the video transmission process. It is responsible for allocating the number of bits for the frames to be encoded during transmission, thereby determining the number of bits that the frames to be encoded can obtain. However, the existing bit allocation methods usually cannot meet the needs of low-bandwidth video transmission. Summary of the invention

[0004] The object of the present invention is to provide an airport surveillance video transmission method, device, storage medium and electronic equipment to improve the above-mentioned problem.

[0005] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for transmitting airport surveillance video, the method comprising:

[0007] Obtaining the comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, wherein the frame images in the image group are images in the airport surveillance video;

[0008] Merge the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into a key area, and merge the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into a non-key area; wherein the first condition is that the comprehensive complexity corresponding to the coding tree unit is greater than the first complexity threshold, the second condition is that the two coding tree units to be merged are adjacent, the third condition is that the absolute value of the comprehensive complexity difference between the two coding tree units to be merged is less than the second complexity threshold, and the fourth condition is that the comprehensive complexity corresponding to the coding tree unit is less than or equal to the first complexity threshold;

[0009] Determine the number of allocated bits corresponding to the kth merged region in the current frame image according to the comprehensive complexity of the kth merged region in the current frame image and the total bit budget of the current frame image, wherein the comprehensive complexity of the kth merged region is the sum of the comprehensive complexities corresponding to the coding tree units of the kth merged region;

[0010] Encoding the pixels in the kth merged area based on the number of allocated bits corresponding to the kth merged area;

[0011] After all frame images in the image group are encoded, the encoded data is transmitted.

[0012] In a second aspect, an embodiment of the present invention provides an airport surveillance video transmission device, the device comprising:

[0013] A first processing unit is used to obtain a comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, wherein the frame image in the image group is an image in the airport surveillance video;

[0014] The first processing unit is further used to merge the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into a key area, and merge the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into a non-key area; wherein the first condition is that the comprehensive complexity corresponding to the coding tree unit is greater than the first complexity threshold, the second condition is that the two coding tree units to be merged are adjacent, the third condition is that the absolute value of the comprehensive complexity difference between the two coding tree units to be merged is less than the second complexity threshold, and the fourth condition is that the comprehensive complexity corresponding to the coding tree unit is less than or equal to the first complexity threshold;

[0015] The first processing unit is further configured to determine the number of allocated bits corresponding to the kth merging region in the current frame image according to the comprehensive complexity of the kth merging region in the current frame image and the total bit budget of the current frame image, wherein the comprehensive complexity of the kth merging region is the sum of the comprehensive complexities corresponding to the coding tree units of the kth merging region;

[0016] The first processing unit is further configured to encode the pixels in the kth merging area based on the number of allocated bits corresponding to the kth merging area;

[0017] The second processing unit is used to transmit the encoded data after all frame images in the image group are encoded.

[0018] In a third aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, and the computer program implements the above method when executed by a processor.

[0019] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above method is implemented.

[0020] Compared with the prior art, the airport surveillance video transmission method, device, storage medium and electronic device provided by the embodiment of the present invention obtain the comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, merge the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into the key area, and merge the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into the non-key area; determine the number of allocated bits corresponding to the kth merged area in the current frame image according to the comprehensive complexity of the kth merged area in the current frame image and the total bit budget of the current frame image; encode the pixels in the kth merged area based on the number of allocated bits corresponding to the kth merged area; after all the frame images in the image group are encoded, transmit the encoded data. The key area and the non-key area can be accurately identified, and dynamic bit allocation is performed according to the comprehensive complexity of the key area and the non-key area, so as to realize the accurate bit allocation of the video frame in the case of low-bandwidth transmission of high-resolution video frames, improve the encoding accuracy of the key area, thereby improving the overall quality of the video frame, and can effectively solve the problem of image quality degradation caused by insufficient resource allocation when the bandwidth is low, and significantly improve the clarity of the key area and the overall monitoring efficiency. It is suitable for complex scenarios with low bandwidth and requiring high-quality monitoring, such as airport runways, traffic sections, etc. It can optimize image quality based on existing monitoring equipment and promote the further development of smart transportation and smart security.

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0024] Figure 2 One of the flow charts of the airport surveillance video transmission method provided in an embodiment of the present invention.

[0025] Figure 3 The second flowchart of the airport surveillance video transmission method provided in the embodiment of the present invention.

[0026] Figure 4 A unit schematic diagram of an airport surveillance video transmission device provided in an embodiment of the present invention.

[0027] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 301 - first processing unit; 302 - second processing unit. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0029] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0031] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0032] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, or are directions or positional relationships in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0033] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "disposed" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0035] The current bit allocation strategy cannot meet the existing high data volume requirements, resulting in a significant reduction in video frame quality. It is difficult to transmit and encode the key parts of high-resolution video frames with high precision under low bandwidth conditions.

[0036] Current bit allocation strategies have difficulty accurately identifying key areas in video frames that really need to be protected first, such as moving objects or areas with abnormal activities. The allocation of regional weights in video frames is usually static or rough, and cannot adapt to dynamic changes in the scene in real time.

[0037] Current bit allocation strategies often only focus on the bit requirements in the current frame or a short period of time, ignoring the importance of global optimization over a longer time span, such as within a Group of Pictures (GOP) or the entire surveillance sequence, which may lead to unbalanced resource allocation and fluctuations in video quality.

[0038] In order to overcome the above problems, an embodiment of the present invention provides an airport surveillance video transmission method, which realizes the low-bandwidth video transmission scenario for airport flight zone monitoring, proposes a dynamic bit allocation optimization method, and comprehensively improves the video frame quality based on scene characteristics and feedback mechanism. An optimization method based on dynamic weight allocation of key areas, a long-time span resource allocation method based on global feedback, and an adaptive adjustment method combining resource utilization maximization and detail priority guarantee are constructed, which can effectively solve the problem of image quality degradation caused by insufficient resource allocation under low bandwidth. It can accurately identify important areas in the video frame, achieve optimal allocation of bit resources under long-term and multi-scenario conditions, and significantly improve the clarity of key areas and overall monitoring efficiency. It is suitable for complex scenes with low bandwidth and high-quality monitoring, such as airport runways, traffic sections, etc., to achieve image quality optimization based on existing monitoring equipment, and promote the further development of smart transportation and smart security.

[0039] The embodiment of the present invention provides an electronic device, which may be a mobile phone, a computer, a server, etc. Figure 1 , a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12, and the processor 10 is used to execute an executable module stored in the memory 11, such as a computer program.

[0040] The processor 10 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the airport surveillance video transmission method can be completed by the hardware integrated logic circuit in the processor 10 or the instructions in the form of software. The above-mentioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0041] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0042] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Although only one bidirectional arrow is used in the figure, it does not mean that there is only one bus 12 or only one type of bus 12 .

[0043] The memory 11 is used to store programs, such as programs corresponding to the airport surveillance video transmission device. The airport surveillance video transmission device includes at least one software function module that can be stored in the memory 11 in the form of software or firmware or fixed in the operating system (OS) of the electronic device. After receiving the execution instruction, the processor 10 executes the program to implement the airport surveillance video transmission method.

[0044] Possibly, the electronic device provided by the embodiment of the present invention further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0045] It should be understood that Figure 1 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0046] The airport surveillance video transmission method provided by the embodiment of the present invention can be applied to but not limited to Figure 1 For detailed procedures, please refer to the electronic equipment shown in Figure 2 , the airport surveillance video transmission method includes: S11, S12, S13, S14, S15 and S16, which are specifically described as follows.

[0047] S11, obtaining the comprehensive complexity corresponding to each coding tree unit in each frame image in the image group.

[0048] The frame images in the image group are images in the airport surveillance video, and the coding tree unit (Coding Tree Unit, CTU for short) includes a plurality of pixel points.

[0049] Optionally, S11, the step of obtaining the comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, includes: S111 and S112, which are specifically described as follows.

[0050] S111, obtaining the gradient complexity, texture complexity and motion complexity of the coding tree unit.

[0051] S112, performing weighted calculation according to the gradient complexity, the texture complexity and the motion complexity to determine the comprehensive complexity.

[0052] Regarding the calculation method of gradient complexity, texture complexity and motion complexity, the embodiment of the present invention also provides an optional implementation method, please refer to the following.

[0053]

[0054] C i =w g ·G i +w t ·T i +w m ·M i

[0055] Among them, C i represents the comprehensive complexity corresponding to the i-th coding tree unit, w g represents the gradient weight, w t represents the texture weight, w m represents the motion weight, G i represents the gradient complexity, T i Indicates the texture complexity, M i Represents motion complexity, N represents the number of pixels in the i-th coding tree unit, CTU i represents the i-th coding tree unit, I(x,y) represents the brightness value of the pixel (x,y), I(x+1,y) represents the brightness value of the pixel (x+1,y), I(x,y+1) represents the brightness value of the pixel (x,y+1), LBP(x,y) represents the texture value of the pixel (x,y), Indicates the motion value of the pixel (x, y).

[0056] S12, merging the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into a key area.

[0057] S13, merging the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into a non-key area.

[0058] Among them, the first condition is that the comprehensive complexity corresponding to the coding tree unit is greater than the first complexity threshold, the second condition is that the two coding tree units to be merged are adjacent, the third condition is that the absolute value of the comprehensive complexity difference between the two coding tree units to be merged is less than the second complexity threshold, and the fourth condition is that the comprehensive complexity corresponding to the coding tree unit is less than or equal to the first complexity threshold.

[0059] Assume that the comprehensive complexity corresponding to the i-th coding tree unit and the comprehensive complexity corresponding to the j-th coding tree unit are both greater than the first complexity threshold, and the two are adjacent. If the formula [|C i -C j |<ε holds true, then the i-th coding tree unit and the j-th coding tree unit can be merged into the key area. It should be noted that if the comprehensive complexity corresponding to the h-th coding tree unit adjacent to the j-th coding tree unit is also greater than the first complexity threshold, and the formula [|C h -C j |<ε holds true, the h-th coding tree unit can be merged into the key area.

[0060] Assume that the comprehensive complexity corresponding to the i-th coding tree unit and the comprehensive complexity corresponding to the j-th coding tree unit are both less than or equal to the first complexity threshold, and the two are adjacent. If the formula [|C i -C j |<ε holds true, then the i-th coding tree unit and the j-th coding tree unit can be merged into a non-key area. It should be noted that if the comprehensive complexity corresponding to the h-th coding tree unit adjacent to the j-th coding tree unit is also less than or equal to the first complexity threshold, and the formula [|C h -C j |<ε holds true, the hth coding tree unit can be merged into the non-key area.

[0061] Among them, C i represents the i-th coding tree unit, C j represents the jth coding tree unit, C h represents the hth coding tree unit, and ε represents the second complexity threshold.

[0062] In an optional implementation, the merging may not be performed, and each coding tree unit is regarded as a region, and the coding tree unit is determined to be a key region or a non-key region based on the first complexity threshold.

[0063] S14, determining the number of allocated bits corresponding to the kth merged region in the current frame image according to the comprehensive complexity of the kth merged region in the current frame image and the total bit budget of the current frame image.

[0064] The comprehensive complexity of the k-th merged region is the sum of the comprehensive complexities corresponding to the coding tree units of the k-th merged region.

[0065] It should be understood that the kth merged area in the current frame image can be a key area or a non-key area. The comprehensive complexity of the key area is greater than that of the non-key area. Bits are dynamically allocated based on the comprehensive complexity to ensure that more bits can be allocated to the key areas, thereby improving the encoding accuracy of the key areas and significantly improving the clarity of the key areas and the overall monitoring efficiency.

[0066] S15, encoding the pixels in the kth merging area based on the number of allocated bits corresponding to the kth merging area.

[0067] S16, after all frame images in the image group are encoded, the encoded data is transmitted.

[0068] In the airport surveillance video transmission method provided in the embodiment of the present invention, the key areas and non-key areas can be accurately identified, and dynamic bit allocation is performed according to the comprehensive complexity of the key areas and non-key areas, thereby achieving accurate bit allocation of video frames when transmitting high-resolution video frames with low bandwidth, improving the encoding accuracy of key areas, and thus improving the overall quality of video frames. It can effectively solve the problem of image quality degradation caused by insufficient resource allocation under low bandwidth, and significantly improve the clarity of key areas and overall monitoring efficiency. It is suitable for complex scenes with low bandwidth and high-quality monitoring, such as airport runways, traffic sections, etc., to achieve image quality optimization based on existing monitoring equipment, and promote the further development of smart transportation and smart security.

[0069] Based on the above, regarding the content in S14, the embodiment of the present invention further provides an optional implementation, please refer to the following. S14, according to the comprehensive complexity of the k-th merged area in the current frame image and the total bit budget of the current frame image, the step of determining the number of allocated bits corresponding to the k-th merged area in the current frame image includes: S141, S142, S143 and S144, which are specifically described as follows.

[0070] S141, determining a region weight corresponding to the kth merged region according to the comprehensive complexity of the kth merged region and the sum of the complexities corresponding to the current frame image.

[0071] Optionally, the regional weight corresponding to the k-th merged region is calculated as follows:

[0072]

[0073]

[0074] Among them, C k represents the comprehensive complexity of the kth merged region, R k represents the kth merged region, C iRepresents the comprehensive complexity corresponding to the i-th coding tree unit, i∈R k Indicates that the i-th coding tree unit belongs to R k , W k represents the regional weight corresponding to the k-th merged region, Represents the total complexity corresponding to the current frame image, N R Indicates the total number of coding tree units in the current frame image, C j Represents the comprehensive complexity corresponding to the j-th coding tree unit.

[0075] S142, determining an initial number of bits corresponding to the kth merged region according to the region weight corresponding to the kth merged region and the total bit budget of the current frame image.

[0076] S143, when the kth merged area is a key area, adjusting the initial bits corresponding to the kth merged area in combination with the first signal-to-noise ratio target value and the peak signal-to-noise ratio corresponding to the kth merged area to obtain the number of allocated bits corresponding to the kth merged area.

[0077] S144, when the kth merged area is a non-key area, the initial bits corresponding to the kth merged area are adjusted in combination with the second signal-to-noise ratio target value and the peak signal-to-noise ratio corresponding to the kth merged area to obtain the number of allocated bits corresponding to the kth merged area.

[0078] Among them, the first signal-to-noise ratio target value is the average value or median of the peak signal-to-noise ratio of each key area in the encoded frame image in the image group, and the second signal-to-noise ratio target value is the average value or median of the peak signal-to-noise ratio of each non-key area in the encoded frame image in the image group.

[0079] Optionally, the formula for calculating the number of allocated bits corresponding to the k-th merged region is as follows:

[0080]

[0081] in, represents the initial number of bits corresponding to the kth merged region, W k represents the regional weight corresponding to the kth merged region, R total Indicates the total bit budget of the current frame image. represents the number of allocated bits corresponding to the kth merged region, q represents the first feedback strength coefficient, PSNR k Represents the peak signal-to-noise ratio corresponding to the kth merged area, PSNR target1 Represents the first signal-to-noise ratio target value, PSNR target2 Indicates the second signal-to-noise ratio target value.

[0082] q·(PSNR target1 -PSNRk ) and q·(PSNR target2 -PSNR k ) as feedback signals and regional dynamic features to optimize the bit allocation of the current frame. The bit allocation of the next frame is adjusted according to the feedback signal. The image quality continuity of the key area can be ensured through dynamic adjustment.

[0083] In an optional implementation, after the number of allocated bits corresponding to the k-th merged region is determined, S14, the step of determining the number of allocated bits corresponding to the k-th merged region in the current frame image according to the comprehensive complexity of the k-th merged region in the current frame image and the total bit budget of the current frame image, also includes: S145, S146 and S147, which are specifically described as follows.

[0084] S145, determining whether the number of allocated bits corresponding to the kth merged region is greater than the upper limit of the number of bits corresponding to the kth merged region. If yes, executing S146; if no, executing S147.

[0085] S146: The upper limit of the number of bits is determined to be the number of allocated bits corresponding to the kth merged region.

[0086] Among them, the upper limit of the number of bits corresponding to the kth merged area can be expressed as The upper limit of the number of bits corresponding to the k-th merged region can be dynamically adjusted according to the number of bits allocated to the key region during the encoding process of the previous frame of the image to adapt to regions of different complexity and prevent over-allocation.

[0087] S147: The number of allocated bits corresponding to the kth merged region remains unchanged.

[0088] Based on the above, regarding how to obtain the total bit budget corresponding to each frame of the image, the embodiment of the present invention also provides an optional implementation method, please refer to Figure 3 , after S11, the airport surveillance video transmission method further includes: S21, S22, S23 and S24, which are specifically described as follows.

[0089] S21, obtaining the sum of the complexities corresponding to each frame of the image according to the comprehensive complexity corresponding to the coding tree unit.

[0090] It should be understood that the sum of the complexities corresponding to the frame image is the sum of the comprehensive complexities corresponding to the coding tree units in the frame image.

[0091] S22, determining a frame weight corresponding to each frame of the image according to the sum of the complexity and the motion intensity corresponding to each frame of the image.

[0092] S23, determining an initial bit budget corresponding to each frame of image according to a frame weight corresponding to each frame of image and a sum of bit budgets corresponding to the image group.

[0093] S24, adjusting the initial bit budget corresponding to each frame of image in combination with the peak signal-to-noise ratio corresponding to each frame of object, so as to obtain a total bit budget corresponding to each frame of image.

[0094] Optionally, the total bit budget corresponding to the frame image is calculated as follows:

[0095]

[0096] R frame,j =R GOP ·W frame,j

[0097]

[0098] in, represents the total bit budget corresponding to the j-th frame image, R frame,j represents the initial bit budget corresponding to the j-th frame image, λ represents the second feedback strength coefficient, and PSNR target3 Represents the third signal-to-noise ratio target value, which is the average or median of the peak signal-to-noise ratio corresponding to all frame images, PSNR j represents the peak signal-to-noise ratio corresponding to the j-th frame image, R GOP represents the total bit budget corresponding to the image group, W frame,j represents the frame weight corresponding to the j-th frame image, C total,j represents the sum of the complexity corresponding to the j-th frame image, M frame,j represents the motion intensity corresponding to the j-th frame image, C total,k represents the sum of the complexity corresponding to the k-th frame image, M frame,k Indicates the motion intensity corresponding to the k-th frame image, N frames Indicates the number of frames in the picture group.

[0099] The frame-level bit budget is dynamically allocated within the GOP range according to the importance of the frame to ensure the balance of quality between frames. Through the inter-frame feedback mechanism, the bit budget and quality allocation of each frame are adjusted to ensure the stability of video quality in a longer time sequence.

[0100] See also Figure 4 , Figure 4 An airport surveillance video transmission device is provided in an embodiment of the present invention. Optionally, the airport surveillance video transmission device is applied to the electronic device described above.

[0101] The airport surveillance video transmission device includes: a first processing unit 301 and a second processing unit 302 .

[0102] The first processing unit 301 is used to obtain the comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, wherein the frame image in the image group is an image in the airport surveillance video;

[0103] The first processing unit 301 is further used to merge the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into a key area, and merge the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into a non-key area; wherein the first condition is that the comprehensive complexity corresponding to the coding tree unit is greater than the first complexity threshold, the second condition is that the two coding tree units to be merged are adjacent, the third condition is that the absolute value of the comprehensive complexity difference between the two coding tree units to be merged is less than the second complexity threshold, and the fourth condition is that the comprehensive complexity corresponding to the coding tree unit is less than or equal to the first complexity threshold;

[0104] The first processing unit 301 is further configured to determine the number of allocated bits corresponding to the kth merging region in the current frame image according to the comprehensive complexity of the kth merging region in the current frame image and the total bit budget of the current frame image, wherein the comprehensive complexity of the kth merging region is the sum of the comprehensive complexities corresponding to the coding tree units of the kth merging region;

[0105] The first processing unit 301 is further configured to encode the pixels in the kth merging area based on the number of allocated bits corresponding to the kth merging area;

[0106] The second processing unit 302 is used to transmit the encoded data after all frame images in the image group are encoded.

[0107] Optionally, the second processing unit 302 may execute the above S16, and the first processing unit 301 may execute other steps in the above method embodiment.

[0108] It should be noted that the airport surveillance video transmission device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effect. For the sake of brief description, for parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above embodiment.

[0109] The embodiment of the present invention further provides a storage medium, which stores computer instructions and programs, and when the computer instructions and programs are read and run, the airport surveillance video transmission method of the above embodiment is executed. The storage medium may include a memory, a flash memory, a register, or a combination thereof.

[0110] An electronic device is provided below, which may be a mobile phone, a computer, a server or the like. Figure 1As shown, the above-mentioned airport surveillance video transmission method can be implemented; specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 can be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the airport surveillance video transmission method of the above-mentioned embodiment is executed.

[0111] In summary, the airport surveillance video transmission method, device, storage medium and electronic device provided by the embodiment of the present invention obtain the comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, merge the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into the key area, and merge the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into the non-key area; determine the number of allocated bits corresponding to the kth merged area in the current frame image according to the comprehensive complexity of the kth merged area in the current frame image and the total bit budget of the current frame image; encode the pixels in the kth merged area based on the number of allocated bits corresponding to the kth merged area; after all the frame images in the image group are encoded, transmit the encoded data. The key areas and non-key areas can be accurately identified, and dynamic bit allocation is performed according to the comprehensive complexity of the key areas and non-key areas, so as to realize the accurate bit allocation of video frames in the case of low-bandwidth transmission of high-resolution video frames, improve the encoding accuracy of key areas, and thus improve the overall quality of video frames, which can effectively solve the problem of image quality degradation caused by insufficient resource allocation when the bandwidth is low, and significantly improve the clarity of key areas and the overall monitoring efficiency. It is suitable for complex scenarios with low bandwidth and requiring high-quality monitoring, such as airport runways, traffic sections, etc. It can optimize image quality based on existing monitoring equipment and promote the further development of smart transportation and smart security.

[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0113] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for transmitting airport surveillance video, characterized in that: The method comprises: Obtaining the comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, wherein the frame images in the image group are images in the airport surveillance video; Merge the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into a key area, and merge the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into a non-key area; wherein the first condition is that the comprehensive complexity corresponding to the coding tree unit is greater than the first complexity threshold, the second condition is that the two coding tree units to be merged are adjacent, the third condition is that the absolute value of the comprehensive complexity difference between the two coding tree units to be merged is less than the second complexity threshold, and the fourth condition is that the comprehensive complexity corresponding to the coding tree unit is less than or equal to the first complexity threshold; Determine the number of allocated bits corresponding to the kth merged region in the current frame image according to the comprehensive complexity of the kth merged region in the current frame image and the total bit budget of the current frame image, wherein the comprehensive complexity of the kth merged region is the sum of the comprehensive complexities corresponding to the coding tree units of the kth merged region; Encoding the pixels in the kth merged area based on the number of allocated bits corresponding to the kth merged area; After all frame images in the image group are encoded, the encoded data is transmitted.

2. The airport surveillance video transmission method according to claim 1, characterized in that: The step of determining the number of allocated bits corresponding to the kth merged region in the current frame image according to the comprehensive complexity of the kth merged region in the current frame image and the total bit budget of the current frame image comprises: Determine the region weight corresponding to the kth merged region according to the comprehensive complexity of the kth merged region and the sum of the complexity corresponding to the current frame image; Determine an initial number of bits corresponding to the kth merged region according to the region weight corresponding to the kth merged region and the total bit budget of the current frame image; When the kth merged area is a key area, the initial bits corresponding to the kth merged area are adjusted in combination with the first signal-to-noise ratio target value and the peak signal-to-noise ratio corresponding to the kth merged area to obtain the number of allocated bits corresponding to the kth merged area; When the kth merged area is a non-key area, the initial bits corresponding to the kth merged area are adjusted in combination with the second signal-to-noise ratio target value and the peak signal-to-noise ratio corresponding to the kth merged area to obtain the number of allocated bits corresponding to the kth merged area; Among them, the first signal-to-noise ratio target value is the average value or median of the peak signal-to-noise ratio of each key area in the encoded frame image in the image group, and the second signal-to-noise ratio target value is the average value or median of the peak signal-to-noise ratio of each non-key area in the encoded frame image in the image group.

3. The airport surveillance video transmission method according to claim 2, characterized in that: After the number of allocated bits corresponding to the k-th merged region is reached, the method further includes: Determine whether the number of allocated bits corresponding to the k-th merged region is greater than the upper limit of the number of bits corresponding to the k-th merged region; If yes, the upper limit of the number of bits is determined as the number of allocated bits corresponding to the kth merged region; If not, the number of allocated bits corresponding to the kth merged region remains unchanged.

4. The airport surveillance video transmission method according to claim 1, characterized in that: The step of obtaining the comprehensive complexity corresponding to each coding tree unit in each frame of the image group includes: Obtain the gradient complexity, texture complexity and motion complexity of the coding tree unit; A weighted operation is performed according to the gradient complexity, the texture complexity and the motion complexity to determine the comprehensive complexity.

5. The airport surveillance video transmission method according to claim 4, characterized in that: C i =w g ·G i +w t ·T i +w m ·M i Among them, C i represents the comprehensive complexity corresponding to the i-th coding tree unit, w g represents the gradient weight, w t represents the texture weight, w m represents the motion weight, G i represents the gradient complexity, T i Indicates the texture complexity, M i Represents motion complexity, N represents the number of pixels in the i-th coding tree unit, CTU i represents the i-th coding tree unit, I(x,y) represents the brightness value of the pixel (x,y), I(x+1,y) represents the brightness value of the pixel (x+1,y), I(x,y+1) represents the brightness value of the pixel (x,y+1), LBP(x,y) represents the texture value of the pixel (x,y), Indicates the motion value of the pixel (x, y).

6. The airport surveillance video transmission method according to claim 1, characterized in that: The method further comprises: According to the comprehensive complexity corresponding to the coding tree unit, obtaining the sum of the complexity corresponding to each frame of the image; Determine the frame weight corresponding to each frame of the image according to the sum of the complexity and the motion intensity corresponding to each frame of the image; Determine an initial bit budget corresponding to each frame of the image according to a frame weight corresponding to each frame of the image and a sum of bit budgets corresponding to the image group; Combined with the peak signal-to-noise ratio corresponding to each frame object, the initial bit budget corresponding to each frame image is adjusted to obtain the total bit budget corresponding to each frame image.

7. The airport surveillance video transmission method according to claim 6, characterized in that: R frame,j =R GOP ·W frame,j in, represents the total bit budget corresponding to the j-th frame image, R frame,j represents the initial bit budget corresponding to the j-th frame image, λ represents the second feedback strength coefficient, PSNR target3 Represents the third signal-to-noise ratio target value, PSNR j represents the peak signal-to-noise ratio corresponding to the j-th frame image, R GOP represents the sum of the bit budgets corresponding to the image group, W frame,j represents the frame weight corresponding to the j-th frame image, C total,j represents the sum of the complexity corresponding to the j-th frame image, M frame,j represents the motion intensity corresponding to the j-th frame image, C total,k represents the sum of the complexity corresponding to the k-th frame image, M frame,k Indicates the motion intensity corresponding to the k-th frame image, N frames Indicates the number of frames in the group of pictures.

8. An airport surveillance video transmission device, characterized in that: The device comprises: A first processing unit is used to obtain a comprehensive complexity corresponding to each coding tree unit in each frame image in the image group, wherein the frame image in the image group is an image in the airport surveillance video; The first processing unit is further used to merge the coding tree units that meet the first condition, the second condition and the third condition in the current frame image into a key area, and merge the coding tree units that meet the fourth condition, the second condition and the third condition in the current frame image into a non-key area; wherein the first condition is that the comprehensive complexity corresponding to the coding tree unit is greater than the first complexity threshold, the second condition is that the two coding tree units to be merged are adjacent, the third condition is that the absolute value of the comprehensive complexity difference between the two coding tree units to be merged is less than the second complexity threshold, and the fourth condition is that the comprehensive complexity corresponding to the coding tree unit is less than or equal to the first complexity threshold; The first processing unit is further configured to determine the number of allocated bits corresponding to the kth merging region in the current frame image according to the comprehensive complexity of the kth merging region in the current frame image and the total bit budget of the current frame image, wherein the comprehensive complexity of the kth merging region is the sum of the comprehensive complexities corresponding to the coding tree units of the kth merging region; The first processing unit is further configured to encode the pixels in the kth merging area based on the number of allocated bits corresponding to the kth merging area; The second processing unit is used to transmit the encoded data after all frame images in the image group are encoded.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images