Method, system and electronic device for low-light image object detection hardware implementation

Abstract

The application discloses a method, system and electronic equipment for hardware implementation of dark light image object detection, and the method comprises the following steps: acquiring input image data, N sets of enhancement weights and M sets of identification weights; calculating the first pixel data according to the N sets of enhancement weights in sequence to obtain second pixel data; calculating the second pixel data according to the M sets of identification weights in sequence to obtain third pixel data; and determining a dark light image object detection result according to the third pixel data. The application reduces the power consumption of an algorithm deployment system by implementing dark light image object detection in hardware, and the calculation efficiency is improved compared with a traditional computer system, and the application range is improved. The dark light enhancement algorithm and the target identification algorithm are combined, and the dark light image object detection and identification rate and the algorithm calculation efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system and electronic equipment for hardware implementation of dark light image object detection. Background Art

[0002] With the advent of the information age, society's demand for hardware capabilities is increasing. The definition of hardware devices has also expanded from traditional computing devices to include new electronic devices that can meet users' needs in all aspects of life. Machine vision is a key technology for realizing artificial intelligence, widely applicable in fields such as industrial control and automation, and has a wide range of use cases. Object detection algorithms have been proposed and studied precisely to meet this demand, and convolutional neural networks are one of the key methods for implementing them.

[0003] Previously, most object detection algorithms were deployed on traditional von Neumann architecture PCs. While many mature convolutional neural networks can now detect and classify objects, object recognition algorithms often experience reduced efficiency in certain lighting conditions, such as low light. Furthermore, currently, all algorithms are deployed on computers, and traditional PC computing devices are increasingly unable to handle the rapidly increasing structure and parameters of artificial neural networks.

[0004] In summary, there is an urgent need for a hardware-implemented method for low-light image object detection to solve the problems existing in the above-mentioned existing technologies. Summary of the Invention

[0005] Since the existing methods have the above-mentioned problems, the present invention proposes a method, system and electronic device for hardware implementation of dark light image object detection.

[0006] In a first aspect, the present invention provides a hardware-implemented method for detecting objects in dark-light images, comprising:

[0007] Obtain input image data, N enhancement weight sets, and M recognition weight sets; the input image data includes first pixel data; the N enhancement weight sets correspond one-to-one to the N convolutional layers of the low-light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolutional layers of the target recognition algorithm; N and M are positive integers;

[0008] Calculating the first pixel data in sequence according to the N enhancement weight sets to obtain second pixel data;

[0009] Calculate the second pixel data in sequence according to the M recognition weight sets to obtain third pixel data;

[0010] A dark-light image object detection result is determined according to the third pixel data.

[0011] Furthermore, the dark light enhancement algorithm is a zero-reference depth curve estimation algorithm; and the target recognition algorithm is a Yolo algorithm.

[0012] Furthermore, before sequentially calculating the first pixel data according to the N enhancement weight sets, the method further includes:

[0013] Denoising the input image data using an estimation method;

[0014] The input image data is filtered using a Gaussian filter.

[0015] Furthermore, the calculating the first pixel data in sequence according to the N enhancement weight sets includes:

[0016] Obtaining a first enhancement weight set from the N enhancement weight sets;

[0017] Matching the first enhancement weight set with the first pixel data by a row-column matcher and calculating by a calculation unit to obtain a first intermediate result;

[0018] Obtaining a second enhancement weight set from the N enhancement weight sets;

[0019] Matching the second enhancement weight set with the first intermediate result and calculating to obtain a second intermediate result;

[0020] The above steps are repeated until the Nth enhancement weight set is matched and calculated with the N-1th intermediate result to obtain the second pixel data.

[0021] Furthermore, the computing unit includes L-level computing subunits, and the step of using the computing unit to calculate and obtain the first intermediate result includes:

[0022] The first sub-result is obtained by calculating using the first-level calculation sub-unit;

[0023] Determine a second sub-result of a second-level calculation sub-unit according to the first sub-result;

[0024] Repeat the above steps until the Lth sub-result of the Lth level calculation sub-unit is determined according to the L-1th sub-result; L is a positive integer;

[0025] The first intermediate result is determined according to the Lth sub-result.

[0026] In a second aspect, the present invention provides a hardware implementation system for dark light image object detection, comprising: a central processing unit, a data transmission control module, a calculation module, and a memory data interface;

[0027] The memory data interface is used to obtain input image data, N enhancement weight sets, and M recognition weight sets; the input image data includes first pixel data; the N enhancement weight sets correspond one-to-one to the N convolutional layers of the low-light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolutional layers of the target recognition algorithm; N and M are positive integers;

[0028] The central processing unit is connected to the memory data interface, and is used to drive the data transmission control module so that the data transmission control module transmits the input image data, the N enhancement weight sets, and the M recognition weight sets to the calculation module;

[0029] The calculation module is used to calculate the first pixel data in sequence according to the N enhancement weight sets to obtain second pixel data; calculate the second pixel data in sequence according to the M recognition weight sets to obtain third pixel data; and determine the dark light image object detection result based on the third pixel data.

[0030] Furthermore, the calculation bit width of the calculation module is determined by the bit width of each enhancement weight in the enhancement weight set and the bit width of each recognition weight in the recognition weight set.

[0031] Furthermore, the calculation module includes: an input image data buffer, a weight data buffer, an accumulation output buffer, an internal buffer, a column accumulation memory, a calculation unit, and a row-column matcher;

[0032] The input image data buffer is used to cache the input image data;

[0033] The weight data buffer is used to cache the enhanced weight set and the identified weight set;

[0034] The cumulative output buffer is used to cache the output results of the computing unit;

[0035] The internal buffer is used to buffer the input and output of the computing module;

[0036] The column accumulation memory is used to cache the intermediate results of the computing unit;

[0037] The row and column matcher is used to match the weight set with the input image data;

[0038] The calculation unit is used to calculate the matching result.

[0039] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for hardware-implemented detection of objects in dark-light images as described in the first aspect is implemented.

[0040] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the hardware-implemented method for dark-light image object detection as described in the first aspect.

[0041] As can be seen from the above technical solutions, the hardware implementation method, system, and electronic device for low-light image object detection provided by the embodiments of the present invention reduce the power consumption of the algorithm deployment system by implementing low-light image object detection hardware. This also improves computational efficiency compared to traditional computer systems, expanding its scope of application. The combination of a low-light enhancement algorithm and a target recognition algorithm improves the recognition rate and computational efficiency of low-light image object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 The system framework of the hardware implementation system for dark-light image object detection provided by the present invention;

[0044] Figure 2 A schematic diagram of a computing module provided by the present invention;

[0045] Figure 3 A flowchart of the hardware implementation method for dark-light image object detection provided by the present invention;

[0046] Figure 4 A flowchart of the hardware implementation method for dark-light image object detection provided by the present invention;

[0047] Figure 5 A schematic diagram of the image data transmission method provided by the present invention;

[0048] Figure 6 A schematic diagram of the hardware resources occupied by the hardware implementation system for low-light image object detection provided by the present invention;

[0049] Figure 7 This is a schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] The following embodiments of the present invention are further described in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0051] Figure 1 Schematic diagram of a hardware implementation system for dark-light image object detection provided by an embodiment of the present invention. The system includes: a memory data interface 100, a central processing unit 200, a data transmission control module 300, and a calculation module 400.

[0052] The memory data interface 100 is used to obtain input image data, N enhancement weight sets, and M recognition weight sets.

[0053] It should be noted that the input image data includes first pixel data; the N enhancement weight sets correspond one-to-one to the N convolution layers of the dark light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolution layers of the target recognition algorithm; N and M are positive integers.

[0054] In the embodiment of the present invention, the memory data interface is a connection circuit between the external memory and the central processing unit.

[0055] In a possible implementation, a DDR3 memory interface is used to connect to a DDR3 external memory.

[0056] The central processing unit 200 is connected to the memory data interface 100 and is used to drive the data transmission control module 300 so that the data transmission control module 300 transmits the input image data, N enhancement weight sets and M recognition weight sets to the calculation module.

[0057] In the embodiment of the present invention, the central processing unit controls the data transmission control module and also transmits data to the external memory.

[0058] The calculation module 400 is used to calculate the first pixel data in sequence according to N enhancement weight sets to obtain second pixel data; calculate the second pixel data in sequence according to M recognition weight sets to obtain third pixel data; and determine the dark light image object detection result based on the third pixel data.

[0059] This solution reduces the power consumption of the algorithm deployment system by implementing low-light image object detection hardware. It also improves computational efficiency compared to traditional computer systems, broadening its scope of application. Combining a low-light enhancement algorithm with a target recognition algorithm improves both the recognition rate and computational efficiency of low-light image object detection.

[0060] In the embodiment of the present invention, the calculation bit width of the calculation module is determined by the bit width of each enhancement weight in the enhancement weight set and the bit width of each recognition weight in the recognition weight set.

[0061] Specifically, the data types of the dark light enhancement algorithm and the target recognition algorithm are analyzed and reduced to the minimum number of bits, so that the calculation bit width of the calculation module is minimized.

[0062] The above solution, by modifying the algorithm structure, narrows the calculation bit width of the computing module and thus reduces the resources of the hardware circuit, making it smaller and more suitable for low-power fields. It improves the system's computational parallelism without losing too much precision, thereby solving the current problems of excessive weights and complex network structures in neural network algorithms.

[0063] Furthermore, if Figure 2 As shown, the calculation module includes: an input image data buffer, a weight data buffer, an accumulation output buffer, an internal buffer, a column accumulation memory, a calculation unit and a row and column matcher;

[0064] The input image data buffer is used to cache the input image data;

[0065] It should be noted that, since the number of pixels in the input image is large and the maximum number of computing units used for calculation each time is limited, the input image needs to be segmented multiple times and stored in the input image buffer for calculation.

[0066] The weight data buffer is used to cache the enhanced weight set and the identified weight set;

[0067] In the embodiment of the present invention, the weight data buffer is used to store weight data corresponding to the input image that needs to be calculated by the current calculation array.

[0068] The cumulative output buffer is used to cache the output results of the calculation unit;

[0069] It should be noted that, due to the large pixel size of the input image during the calculation process,

[0070] For example, there are 614,400 pixels in total, but the input image data buffer is often limited by resource constraints and can only store a small number of rows of image data at a time. For example, the number of computing units is 12*14, that is, 12 rows of computing units. Therefore, only 12 rows of input image data can be calculated at a time. Since the input image has 480 rows of data, 40 calculations are required. The algorithm's sliding window needs to traverse the entire image and add the values ​​of all columns after the input image buffer is refreshed. The cumulative output buffer is used to store the column sums from the previous calculation, and the column sums are continued to be iteratively added to obtain the final calculation result for the entire image.

[0071] The internal buffer is used to buffer the input and output of the computing module;

[0072] In a possible implementation, the internal buffer is an FPGA internal buffer.

[0073] It should be noted that the FPGA internal buffer is a part of the circuit that is separated from the FPGA internal circuit. Its function is to serve as a cache inside the FPGA to temporarily store the data to be calculated.

[0074] The column accumulation memory is used to cache the intermediate results of the computing unit;

[0075] The row and column matcher is used to match the weight set with the input image data;

[0076] Specifically, it is determined whether the current data matches the row where the matching circuit is located, that is, whether the data currently transmitted to the data bus is the data required by the current row or column. If the data is the calculation data required by the current row or column, the data passes through the row and column matcher; otherwise, the data is blocked outside the row and column matcher, and the calculation unit of the current row or column is shut down by gating the clock.

[0077] The calculation unit is used to calculate the matching results.

[0078] The above solution decomposes complex neural network calculations into a computing system composed of a computing array, and designs the computing array into a format of hardware computing units, thereby reducing the power consumption of the entire system by shielding the computing units of the computing array.

[0079] In the embodiment of the present invention, the computing module may adopt a field programmable gate array (FPGA) or an image signal processing circuit (ISP), which is not specifically limited in the embodiment of the present invention.

[0080] For example, the development board uses Xilinx's PYNQ-Z7020 and uses Vivado as the integrated development environment (IDE).

[0081] In this embodiment of the present invention, the development board's RAM memory and the FPGA's internal buffer BRAM are configured to facilitate the FPGA reading image data. After setting the initial address and pixel number of the input image, the address of the FPGA read / write interface is configured to correspond to the corresponding on-board memory address.

[0082] In an embodiment of the present invention, a high-dimensional register group involved in calculations in an algorithm is divided and grouped according to calculation blocks, thereby improving the efficiency of accessing high-dimensional registers during calculations and reading and writing processes, and reducing power consumption of data transmission.

[0083] The above scheme makes the algorithm structures of the two algorithms more suitable for hardware implementation and more efficient through hardware optimization.

[0084] Furthermore, the design process of the hardware implementation system for low-light image object detection is as follows:

[0085] S1: Perform image preprocessing on the input image.

[0086] S2: Use the HLS tool to perform hardware optimization on the dark light enhancement algorithm and target recognition algorithm.

[0087] S3: Perform front-end simulation of the circuit's functionality.

[0088] S4: Use the IDE tool to burn the circuit layout and routing results to the FPGA, and perform testing and verification on the FPGA.

[0089] S5: Use the RAM interface of the FPGA to read the image in the memory and perform hardware acceleration within the FPGA. After the calculation is completed, the processed image data is stored in the RAM.

[0090] S6: Use IDE tools to read the processing results from the FPGA RAM and compare and test them with the expected results.

[0091] Specifically, the image to be tested is transferred to the configured memory of the development board using the Ethernet interface, and the relevant read and write addresses are configured.

[0092] The working state of the entire circuit is specified. The circuit starts working by setting the two registers GIE and AP_CTRL in the FPGA. The circuit working end bit is judged by AP_CTRL in the FPGA.

[0093] In a possible implementation, the circuit start setting is specifically: GIE is set low, AP_CTRL is set high; the circuit end judgment is specifically: judging whether the second highest bit of the AP_CTRL register is high level, if it is high, the circuit operation is ended, if it is low, it is not ended.

[0094] Furthermore, the image data after FPGA accelerated calculation is transmitted to a PC via Ethernet to record the results and compare them with the expected results.

[0095] The above solution deploys the system on an FPGA platform, which reduces computing power consumption compared to a PC and increases the system's scope of use. It can be widely used in object recognition tasks in environments with poor lighting conditions, such as security monitoring, face recognition, industrial inspection, and other fields.

[0096] Based on the above-mentioned dark light image object detection hardware implementation system, Figure 3 The following is an exemplary flowchart of a method for hardware-implemented low-light image object detection provided by an embodiment of the present invention, which can be executed by the hardware-implemented system for low-light image object detection.

[0097] like Figure 3 As shown in the figure, the process specifically includes:

[0098] Step 301: Obtain input image data, N enhancement weight sets, and M recognition weight sets.

[0099] It should be noted that the input image data includes first pixel data; the N enhancement weight sets correspond one-to-one to the N convolution layers of the dark light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolution layers of the target recognition algorithm; N and M are positive integers.

[0100] In the embodiment of the present invention, a multi-channel simultaneous reading and writing method is used to read the input image data or feature data in parallel.

[0101] For example, when the input image is an RGB image, a three-channel simultaneous reading and writing method is adopted.

[0102] In a possible implementation, the dark light enhancement algorithm is a zero-reference depth curve estimation algorithm; and the target recognition algorithm is a Yolo algorithm.

[0103] It should be noted that the dark light enhancement algorithm can also be a multi-exposure image fusion MEF algorithm, an EnlightenGAN algorithm, etc., and the target recognition algorithm can also be AlexNet, GoogleNet, etc., which is not specifically limited in this embodiment of the present invention.

[0104] The above solution uses a zero-reference depth curve estimation algorithm to make the enhancement result more uniform, avoiding problems such as overexposure after image illumination enhancement.

[0105] For example, write a Python script to extract the neural network structure, weights and other parameters of the zero-reference depth curve estimation algorithm and Yolo algorithm.

[0106] It should be noted that the zero-reference depth curve estimation algorithm has 7 layers of parameters, and the Yolo algorithm has 28 layers of parameters. That is, the embodiment of the present invention includes 7 enhancement weight sets and 28 recognition weight sets.

[0107] Furthermore, the extracted parameters are stored in a document, which makes it easier to extract the parameters and generate the hardware algorithm of the corresponding structure in the Jupyter tool of the development board pynq-z7020.

[0108] In the embodiment of the present invention, language simulation software ModelSim and compiled Verilog simulator VCS are used to write Verilog codes and comprehensively generate data flow abstract RTL level circuits.

[0109] Step 302: Calculate the first pixel data in sequence according to N enhancement weight sets to obtain second pixel data.

[0110] Step 303 : Calculate the second pixel data in sequence according to the M recognition weight sets to obtain third pixel data.

[0111] Step 304 : Determine a dark-light image object detection result based on the third pixel data.

[0112] This solution reduces the power consumption of the algorithm deployment system by implementing low-light image object detection hardware. It also improves computational efficiency compared to traditional computer systems, broadening its scope of application. Combining a low-light enhancement algorithm with a target recognition algorithm improves both the recognition rate and computational efficiency of low-light image object detection.

[0113] In the embodiment of the present invention, before step 302, an estimation method is used to denoise the input image data; and a Gaussian filter is used to filter the input image data.

[0114] In a possible implementation, column fixed pattern noise (FPN) noise removal is performed on the input image data.

[0115] Specifically, a relationship curve between the column mean vector and the column number of each image sequence, that is, a column mean vector curve, is obtained.

[0116] Furthermore, we need to find the relationship curve between the average column mean vector Y and the column number of K images. The specific calculation formula is as follows:

[0117] Y(Te)=1 / k(Y1(Te)+Y2(Te)+…+Y k (Te))

[0118] Furthermore, D of K images k The average value obtained by adding them together is the estimated CFPN. The specific calculation formula is as follows:

[0119] D(C,Te)=ceil{1 / K[D1(C,Te)+D2(C,Te)+…+D k (C,Te)]}

[0120] Where ceil is the ceiling function in MATLAB, and Te is the exposure time.

[0121] In the embodiment of the present invention, the obtained CFPN estimation value is stored in a lookup table, and real-time correction can be performed.

[0122] The above solution improves the processing efficiency of subsequent algorithms by preprocessing the image.

[0123] Furthermore, in step 302 of the embodiment of the present invention, the specific steps are as follows: Figure 4 As shown below:

[0124] Step 401: Obtain a first enhancement weight set among N enhancement weight sets.

[0125] Step 402: Match the first enhancement weight set with the first pixel data through a row-column matcher and calculate using a calculation unit to obtain a first intermediate result.

[0126] It should be noted that the computing unit includes L-level computing sub-units, where L is a positive integer.

[0127] Specifically, the first sub-result is obtained by calculation using the first-level calculation sub-unit;

[0128] Determine a second sub-result of the second-level calculation sub-unit according to the first sub-result;

[0129] Repeat the above steps until the Lth sub-result of the Lth level calculation sub-unit is determined based on the L-1th sub-result;

[0130] A first intermediate result is determined according to the Lth subresult.

[0131] The above solution adopts pipeline technology to improve throughput and reduce latency by designing pipeline stages, namely computing sub-unit technology.

[0132] Step 403: Obtain a second enhancement weight set from the N enhancement weight sets.

[0133] Step 404: Match the second enhancement weight set with the first intermediate result and calculate to obtain a second intermediate result.

[0134] Step 405 : Repeat the above steps until the Nth enhancement weight set is matched and calculated with the N-1th intermediate result to obtain the second pixel data.

[0135] The above scheme combines the dark light enhancement algorithm and the target recognition algorithm to improve the recognition rate of objects detected in dark light images and the algorithm calculation efficiency.

[0136] Furthermore, taking a 3x3 convolution kernel and 64 channels of the input image as an example, the schematic diagram of the image data transmission method is as follows Figure 5 shown.

[0137] As can be seen from the figure, channel0, channel1, ..., channel63 are the number of channels of the input image, and the three dotted boxes represent three parallel computing units, that is, the calculations in the three boxes are performed synchronously.

[0138] Furthermore, since the input image has 64 channels, there are also 64 corresponding convolution kernels for calculation. The short boxes represent these 64 convolution windows, each corresponding to the data of the 64 channels, and the results of the calculations for each channel are accumulated. Since the calculations of the three boxes are performed simultaneously, when their respective calculation windows slide to the end of the row of image data, that is, when the convolution calculation of the row of image data has ended, the calculations of the other two rows are also completed. The final calculation result only requires adding the data in the three boxes to obtain the calculated image.

[0139] Furthermore, the hardware resources occupied by the algorithm hardware optimization of the embodiment of the present invention are as follows: Figure 6 As shown, it can be seen that the object recognition algorithm is combined with the dark light enhancement algorithm, and the architecture of the two algorithms is modified in hardware, which achieves low power consumption and reduces the hardware resources occupied by the algorithm.

[0140] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, see Figure 7 , the electronic device specifically includes the following contents: a processor 701, a memory 702, a communication interface 703 and a communication bus 704;

[0141] The processor 701, memory 702, and communication interface 703 communicate with each other via the communication bus 704; the communication interface 703 is used to implement information transmission between devices;

[0142] The processor 701 is used to call the computer program in the memory 702. When the processor executes the computer program, all steps of the above-mentioned method for hardware implementation of low-light image object detection are implemented. For example, when the processor executes the computer program, the following steps are implemented: obtaining input image data, N enhancement weight sets and M recognition weight sets; the input image data includes first pixel data; the N enhancement weight sets correspond one-to-one to the N convolution layers of the low-light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolution layers of the target recognition algorithm; N and M are positive integers; the first pixel data is calculated in sequence according to the N enhancement weight sets to obtain second pixel data; the second pixel data is calculated in sequence according to the M recognition weight sets to obtain third pixel data; and the low-light image object detection result is determined according to the third pixel data.

[0143] Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements all the steps of the above-mentioned method for hardware implementation of low-light image object detection. For example, when the processor executes the computer program, it implements the following steps: obtaining input image data, N enhancement weight sets and M recognition weight sets; the input image data contains first pixel data; the N enhancement weight sets correspond one-to-one to the N convolution layers of the low-light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolution layers of the target recognition algorithm; N and M are positive integers; the first pixel data is calculated in sequence according to the N enhancement weight sets to obtain second pixel data; the second pixel data is calculated in sequence according to the M recognition weight sets to obtain third pixel data; and the low-light image object detection result is determined according to the third pixel data.

[0144] In addition, the logical instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a device implemented by hardware for dark light image object detection, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0145] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the embodiments of the present invention. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0146] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a device for hardware-implemented dark-light image object detection, or a network device, etc.) to execute the hardware-implemented method for dark-light image object detection described in each embodiment or certain parts of the embodiment.

[0147] Furthermore, in the present invention, terms such as "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0148] In addition, in the present invention, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0149] In addition, in the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A hardware-implemented method for detecting objects in dark-light images, characterized in that: include: Obtain input image data, N enhancement weight sets, and M recognition weight sets; The input image data includes first pixel data; The N enhancement weight sets correspond one-to-one to the N convolution layers of the dark light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolution layers of the target recognition algorithm; N and M are positive integers; Calculating the first pixel data in sequence according to the N enhancement weight sets to obtain second pixel data; Calculate the second pixel data in sequence according to the M recognition weight sets to obtain third pixel data; determining a dark-light image object detection result according to the third pixel data; The calculating the first pixel data in sequence according to the N enhancement weight sets includes: Obtaining a first enhancement weight set from the N enhancement weight sets; Matching the first enhancement weight set with the first pixel data by a row-column matcher and calculating by a calculation unit to obtain a first intermediate result; Obtaining a second enhancement weight set from the N enhancement weight sets; Matching the second enhancement weight set with the first intermediate result and calculating to obtain a second intermediate result; The above steps are repeated until the Nth enhancement weight set is matched and calculated with the N-1th intermediate result to obtain the second pixel data.

2. The method for hardware-implemented object detection in dark-light images according to claim 1, characterized in that: The dark light enhancement algorithm is a zero-reference depth curve estimation algorithm; the target recognition algorithm is a Yolo algorithm.

3. The method for hardware-implemented object detection in dark-light images according to claim 1, characterized in that: Before sequentially calculating the first pixel data according to the N enhancement weight sets, the method further includes: Denoising the input image data using an estimation method; The input image data is filtered using a Gaussian filter.

4. The method for hardware-implemented object detection in dark-light images according to claim 1, characterized in that: The computing unit includes L-level computing subunits, and the step of using the computing unit to calculate and obtain the first intermediate result includes: The first sub-result is obtained by calculating using the first-level calculation sub-unit; Determine a second sub-result of a second-level calculation sub-unit according to the first sub-result; Repeat the above steps until the Lth sub-result of the Lth level calculation sub-unit is determined according to the L-1th sub-result; L is a positive integer; The first intermediate result is determined according to the Lth sub-result.

5. A hardware implementation system for dark light image object detection, characterized in that: include: Central processing unit, data transmission control module, computing module and memory data interface; The memory data interface is used to obtain input image data, N enhancement weight sets, and M recognition weight sets; the input image data includes first pixel data; the N enhancement weight sets correspond one-to-one to the N convolutional layers of the low-light enhancement algorithm; the M recognition weight sets correspond one-to-one to the M convolutional layers of the target recognition algorithm; N and M are positive integers; The central processing unit is connected to the memory data interface, and is used to drive the data transmission control module so that the data transmission control module transmits the input image data, the N enhancement weight sets, and the M recognition weight sets to the calculation module; The calculation module is used to calculate the first pixel data in sequence according to the N enhancement weight sets to obtain second pixel data; and calculate the second pixel data in sequence according to the M recognition weight sets to obtain third pixel data; determining a dark-light image object detection result according to the third pixel data; The calculating the first pixel data in sequence according to the N enhancement weight sets includes: Obtaining a first enhancement weight set from the N enhancement weight sets; Matching the first enhancement weight set with the first pixel data by a row-column matcher and calculating by a calculation unit to obtain a first intermediate result; Obtaining a second enhancement weight set from the N enhancement weight sets; Matching the second enhancement weight set with the first intermediate result and calculating to obtain a second intermediate result; The above steps are repeated until the Nth enhancement weight set is matched and calculated with the N-1th intermediate result to obtain the second pixel data.

6. The hardware implementation system for low-light image object detection according to claim 5, characterized in that: The calculation bit width of the calculation module is determined by the bit width of each enhancement weight in the enhancement weight set and the bit width of each recognition weight in the recognition weight set.

7. The hardware implementation system for low-light image object detection according to claim 5, characterized in that: The calculation module includes: an input image data buffer, a weight data buffer, an accumulation output buffer, an internal buffer, a column accumulation memory, a calculation unit and a row and column matcher; The input image data buffer is used to cache the input image data; The weight data buffer is used to cache the enhanced weight set and the identified weight set; The cumulative output buffer is used to cache the output results of the computing unit; The internal buffer is used to buffer the input and output of the computing module; The column accumulation memory is used to cache the intermediate results of the computing unit; The row and column matcher is used to match the weight set with the input image data; The calculation unit is used to calculate the matching result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images