Image Recognition System and Method

Abstract

The present invention provides an image recognition system and method, relating to the technical field of image recognition. The system includes: an image preprocessing module, a dynamic dehazing and enhancement module, a neural network acceleration module, an intelligent data scheduling module, and an energy consumption management module; the image preprocessing module is used to preprocess the image to be recognized; the dynamic dehazing and enhancement module is used to perform dehazing and enhancement processing on the preprocessed image; the neural network acceleration module is used to select a target neural network model suitable for the current image from multiple neural network models, and recognize the current image through the target neural network model; the target neural network model recognizes the current image based on a parallel block convolution acceleration algorithm; the intelligent data scheduling module is used to dynamically manage the data streams of other modules; the energy consumption management module is used to manage the energy consumption of other modules. The present invention can improve the recognition efficiency and adaptability of image recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly relates to an image recognition system and method. Background Art

[0002] Image recognition is an important branch in the field of computer vision, which aims to automatically identify and classify objects, scenes, and features in images through computer algorithms. However, current image recognition technologies have problems of low recognition efficiency and low adaptability.

[0003] Based on this, how to improve the recognition efficiency and adaptability of image recognition has become a technical problem to be solved urgently. Summary of the Invention

[0004] In view of this, to solve the above technical problems, the present invention provides an image recognition system and method.

[0005] The present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides an image recognition system, including: an image preprocessing module, a dynamic haze removal and enhancement module, a neural network acceleration module, an intelligent data scheduling module, and an energy consumption management module;

[0007] The image preprocessing module is used to preprocess the image to be recognized and send the preprocessed image to the dynamic haze removal and enhancement module;

[0008] The dynamic haze removal and enhancement module is used to determine whether the preprocessed image needs to be subjected to haze removal and enhancement processing. When the preprocessed image needs to be subjected to haze removal and enhancement processing, it performs haze removal and enhancement processing on the preprocessed image and sends the haze-removed and enhanced image to the neural network acceleration module. When the preprocessed image does not need to be subjected to haze removal and enhancement processing, it sends the preprocessed image to the neural network acceleration module;

[0009] The neural network acceleration module is used to select a target neural network model applicable to the current image from multiple neural network models and identify the current image through the target neural network model; the target neural network model identifies the current image based on a parallel block convolution acceleration algorithm;

[0010] The intelligent data scheduling module is used to dynamically manage the data streams of other modules;

[0011] The energy consumption management module is used to manage the energy consumption of other modules.

[0012] Optionally, the image preprocessing module is specifically used to perform at least one of the following preprocessing operations on the image to be recognized:

[0013] Color space conversion, edge detection and enhancement, bilateral filtering, erosion and dilation, corner detection, image registration, and infinite zooming.

[0014] Optionally, the dynamic haze removal and enhancement module specifically performs haze removal and enhancement processing on the preprocessed image based on the dark channel dehazing technique and the light adaptation adjustment strategy.

[0015] Optionally, after selecting the target neural network model, the neural network acceleration module is also used to allocate computing resources to each layer of the target neural network model according to the load situation of each layer, so that the heavier the load of a layer, the more computing resources it obtains.

[0016] Optionally, the intelligent data scheduling module is specifically used for:

[0017] For each first module among the image preprocessing module, the dynamic haze removal and enhancement module, the neural network acceleration module, and the energy consumption management module, calculate the load weight of the first module according to the current data requirements, computing power, and average response time of the first module;

[0018] Calculate the buffer size of the first module according to the load weight;

[0019] Allocate a buffer for the first module according to the buffer size.

[0020] Optionally, the intelligent data scheduling module is specifically further used for:

[0021] After calculating the load weights of each first module, calculate the resource allocation ratio of each first module according to the load weights;

[0022] Allocate computing and storage resources to each first module according to the resource allocation ratio.

[0023] Optionally, the intelligent data scheduling module is specifically further used for:

[0024] For any second module among the image preprocessing module, the dynamic haze removal and enhancement module, the neural network acceleration module, and the energy consumption management module, predict the next moment data demand of the second module according to the historical data correlation coefficient of the second module and the input data stream at the previous moment;

[0025] Based on the next moment data demand, preload the target data required by the second module at the next moment to complete the loading of the target data before the data demand of the target data is generated.

[0026] Optionally, the energy consumption management module is specifically used for:

[0027] For any third module among the image preprocessing module, the dynamic haze removal and enhancement module, the neural network acceleration module, and the intelligent data scheduling module, calculate the energy consumption weight of the third module according to the instantaneous power consumption, load level, and execution time of the current task of the third module;

[0028] According to the energy consumption weight and the current power state of the third module, determine whether it is necessary to switch the power state of the third module; the power state includes a high power state, a medium power state, and a low power state;

[0029] When it is necessary to switch the power state of the third module, perform a power state switch on the third module.

[0030] Optionally, the energy consumption management module is further specifically configured to:

[0031] Calculate the power distribution ratio of the third module according to the priority of the current task of the third module and the energy consumption weight;

[0032] Allocate electric energy to the third module according to the power distribution ratio.

[0033] In a second aspect, the present invention provides an image recognition method, which is applied to the image recognition system as described above. The image recognition method includes:

[0034] After receiving the image to be recognized, the image preprocessing module preprocesses the image to be recognized and sends the preprocessed image to the dynamic haze removal and enhancement module;

[0035] The dynamic haze removal and enhancement module determines whether the preprocessed image needs to be subjected to haze removal and enhancement processing;

[0036] When the preprocessed image needs to be subjected to haze removal and enhancement processing, the dynamic haze removal and enhancement module performs haze removal and enhancement processing on the preprocessed image and sends the haze-removed and enhanced image to the neural network acceleration module; when the preprocessed image does not need to be subjected to haze removal and enhancement processing, the dynamic haze removal and enhancement module sends the preprocessed image to the neural network acceleration module;

[0037] The neural network acceleration module selects a target neural network model applicable to the current image from multiple neural network models and recognizes the current image through the target neural network model; the target neural network model recognizes the current image based on the parallel block convolution acceleration algorithm.

[0038] The present invention adopts the above technical solutions. An image recognition system includes: an image preprocessing module, a dynamic haze removal and enhancement module, a neural network acceleration module, an intelligent data scheduling module, and an energy consumption management module. The image preprocessing module is used to preprocess the image to be recognized and send the preprocessed image to the dynamic haze removal and enhancement module. The dynamic haze removal and enhancement module is used to determine whether the preprocessed image needs to be subjected to haze removal and enhancement processing. When the preprocessed image needs to be subjected to haze removal and enhancement processing, it performs haze removal and enhancement processing on the preprocessed image and sends the haze-removed and enhanced image to the neural network acceleration module. When the preprocessed image does not need to be subjected to haze removal and enhancement processing, it sends the preprocessed image to the neural network acceleration module. The neural network acceleration module is used to select a target neural network model applicable to the current image from multiple neural network models and recognize the current image through the target neural network model. The target neural network model recognizes the current image based on the parallel block convolution acceleration algorithm. The intelligent data scheduling module is used to dynamically manage the data streams of other modules. The energy consumption management module is used to manage the energy consumption of other modules.

[0039] Based on this, the present invention pre-sets multiple neural network models. Subsequently, the neural network acceleration module can select a target neural network model applicable to the current image from multiple neural network models, enabling the present invention to quickly switch between different models according to task requirements. This flexible adaptability makes it show higher generality and adaptability in multi-task processing and different application scenarios. Moreover, the present invention recognizes the current image through the target neural network model based on the parallel block convolution acceleration algorithm, that is, the parallel computing units in the system circuit simultaneously process multiple sub-block convolution operations, thereby improving the image recognition efficiency of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 It is a schematic structural diagram of an image recognition system provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic flowchart of an image recognition method provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other implementation manners obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope protected by the present invention.

[0044] Image recognition is an important branch in the field of computer vision, which aims to automatically identify and classify objects, scenes, and features in images through computer algorithms. Current image recognition technologies usually use a single neural network model. When it is needed for different tasks, manual adjustment of model parameters is required to adapt to different task requirements, resulting in the problems of low recognition efficiency and low adaptability in existing image recognition technologies.

[0045] Based on this, in order to improve the recognition efficiency and adaptability of image recognition, the present invention provides an image recognition system and method.

[0046] The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Figure 1 It is a schematic structural diagram of an image recognition system provided by an embodiment of the present invention. It should be noted that this image recognition system can be applied to existing integrated circuits such as FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), GPU (Graphics Processing Unit), and ARM (Advanced RISC Machines). In this embodiment of the present invention, taking the application to FPGA as an example for illustration, those skilled in the art can easily think of the specific implementation manners for applying to other integrated circuits.

[0048] As Figure 1 shown, this image recognition system includes: an image preprocessing module 11, a dynamic haze removal and enhancement module 12, a neural network acceleration module 13, an intelligent data scheduling module 14, and an energy consumption management module 15.

[0049] The image preprocessing module 11 is used to preprocess the image to be recognized and send the preprocessed image to the dynamic haze removal and enhancement module 12. The image preprocessing module 11 will be specifically introduced below.

[0050] The image preprocessing module 11 integrates image preprocessing operations such as color space conversion, edge detection and enhancement, bilateral filtering, and erosion and dilation, and further adds advanced preprocessing functions such as corner detection, image registration, and infinite zoom, thus significantly improving the usability of images in complex environments and providing enhanced image input for recognition.

[0051] In practical applications, the image preprocessing module 11 is specifically used to perform at least one preprocessing operation on the image to be recognized, such as color space conversion, edge detection and enhancement, bilateral filtering, erosion and dilation, corner detection, image registration, and infinite zoom, according to the actual situation.

[0052] The image preprocessing module 11 can specifically perform preprocessing on the image to be recognized in a parallel computing manner to improve its preprocessing efficiency. For example, when the image preprocessing module 11 performs color space conversion on its input image, by converting the image from the RGB color space to the YC b C r color space, the separation of luminance information and chrominance information is achieved, facilitating focusing on the structural details of the image in subsequent processing. Specifically, let the pixel matrix of the input image be , where represents the spatial coordinates of the image, , , are the pixel values of the red, green, and blue channels respectively and are converted. This conversion process is completed by the parallel multiplier-accumulator circuit in the FPGA, and the component calculation of each pixel is performed in an independent data stream, ensuring efficient conversion without increasing the system latency.

[0053] After color space conversion, the image luminance channel can be input into the multi-scale edge enhancement algorithm. Through multi-scale edge detection and enhancement, image details at different scales are captured to ensure clear edge information in complex backgrounds. Specifically, the system first calculates the horizontal and vertical gradient values of the luminance channel through the Sobel operator (Sobel operator). Assuming the image gradient is , its horizontal and vertical components are and respectively, and are obtained through the Sobel convolution kernel:

[0054] ......(1)

[0055] ......(2)

[0056] where and are the Sobel kernel matrices. The calculated gradient magnitude is as follows:

[0057] ...... (3)

[0058] On this basis, the gradient image can be further smoothed by multi-scale Gaussian filtering to extract detail information at different scales. Let the Gaussian filtering parameters at different scales be , and the gradient image corresponding to the scale is denoted as . The final edge image is obtained by the weighted sum of the gradient maps at each scale:

[0059] ...... (4)

[0060] where are the weight coefficients at different scales. This multi-scale mechanism allows the system to enhance edge information at different levels of detail to ensure that the outlines of key objects remain clearly distinguishable even in complex backgrounds. This multi-scale edge enhancement process is completed by parallel convolution and addition circuits in the FPGA, enabling the multi-scale edge enhancement to be efficiently completed and significantly improving the real-time processing ability of the system.

[0061] After edge enhancement is completed, the image preprocessing module can use an adaptive bilateral denoising algorithm to denoise the image. This algorithm combines the weights of spatial distance and color similarity, retaining edge information while removing noise. Specifically, let the output of bilateral filtering be , then its calculation formula is:

[0062] ...... (5)

[0063] where is the spatial distance weight function, is the color similarity weight function, is the neighborhood range of pixel point , and is the normalization coefficient used to ensure that the sum of the weights is 1. This algorithm is implemented by parallel accumulation circuits in the FPGA, enabling the denoising process to effectively suppress noise without losing image details and ensuring the clarity of the final image output.

[0064] The dynamic defogging and enhancement module 12 is used to determine whether the preprocessed image needs to be defogged and enhanced. When the preprocessed image needs to be defogged and enhanced, it performs defogging and enhancement on the preprocessed image and sends the defogged and enhanced image to the neural network acceleration module 13. When the preprocessed image does not need to be defogged and enhanced, it sends the preprocessed image to the neural network acceleration module 13. The following specifically introduces the dynamic defogging and enhancement module 12.

[0065] The core algorithm of the dynamic dehazing enhancement module 12 is the adaptive dark channel dehazing enhancement algorithm. This algorithm combines the dark channel dehazing technology and the light-adaptive adjustment strategy, effectively improving the image clarity in low-light and high-haze environments. Moreover, the dynamic dehazing enhancement module 12 can also perform dehazing enhancement processing in a parallel computing manner to improve the dehazing enhancement processing efficiency and achieve real-time dynamic dehazing processing.

[0066] Specifically, the algorithm initially estimates the influence of haze by calculating the dark channel image of the image. Let the pixel value of the input image be , where represents the spatial coordinates of the image, and the values of the RGB three channels are respectively . The dark channel image is defined as:

[0067] ...... (6)

[0068] Among them, represents the neighborhood range centered on , and the size of this neighborhood is adaptively adjusted according to the image resolution and haze thickness. On the FPGA, the minimum value search of the dark channel is realized through parallel computing, and the minimum value calculation of each channel is quickly completed through pixel comparison within the neighborhood, ensuring the real-time performance of dehazing.

[0069] Next, the transmission rate is estimated using the dark channel image, that is, the proportion of effective information retained in the dehazed image. The estimation formula for the transmission rate is:

[0070] ...... (7)

[0071] Among them, is the adjustment coefficient, which is used to control the intensity of the transmission rate and make the dehazing effect more natural. This coefficient is adaptively adjusted according to the environmental light conditions to adapt to changes in different lighting and haze concentrations.

[0072] Based on the above transmission rate, the system estimates the global atmospheric light component A to simulate the light scattering effect in the natural environment. The atmospheric light A is one of the brightest pixel values in the input image. Select some pixels with the highest dark channel values in the image and take the color mean to estimate this value. This process is also completed in parallel on the FPGA to ensure the high efficiency of the calculation.

[0073] Then, the dehazed image is restored through the following formula:

[0074] ...... (8)

[0075] Among them, is a relatively small constant to prevent division by zero. This restoration process is implemented on an FPGA through multiplication and division circuits, achieving the output of real-time dehazed images and ensuring the natural transition and visual effect of the images.

[0076] To enhance the adaptability of the system in a dynamic environment, the algorithm further incorporates a light adaptive adjustment module. Specifically, this module dynamically adjusts and by real-time monitoring the average brightness of the input image, enabling the dehazing effect to maintain high clarity under different lighting conditions. In low-brightness scenarios, the system increases and the size of the neighborhood window to obtain a stronger dehazing effect; in high-brightness environments, these parameters are appropriately reduced to avoid detail loss caused by excessive dehazing. This adaptive adjustment strategy is implemented through control logic in the FPGA, updating the algorithm parameters in real time to ensure the stability of the system in complex environments.

[0077] The neural network acceleration module 13 is used to select a target neural network model suitable for the current image from multiple neural network models and perform recognition on the current image through the target neural network model; the target neural network model performs recognition on the current image based on the parallel block convolution acceleration algorithm. The neural network acceleration module 13 is introduced in detail below.

[0078] The neural network acceleration module 13 can support multiple neural network models such as convolutional neural networks and multi-layer perceptrons through the deep integration of the FPGA and the neural network accelerator. The core of this module is the adaptive model selection and dynamic acceleration algorithm, which selects the optimal neural network model and adjusts the allocation of computing resources in real time to ensure the balance between recognition speed and accuracy of the system in complex environments and provide support for subsequent recognition. In addition, the neural network acceleration module 13 can also adopt a parallel computing method to improve its computing efficiency.

[0079] Specifically, the system loads multiple neural network models with different depths and structures, denoted as , and each model has different complexities and applicable scenarios. During actual use, the neural network acceleration module 13 first selects the neural network model corresponding to the target scenario from multiple neural network models according to the target scenario selected by the user, and then, by analyzing the feature distribution of the current input image, calculates the selection weights of each neural network model corresponding to the target scenario:

[0080] ...... (9)

[0081] Among them, represents the computational complexity of model , is a tuning parameter for controlling model complexity. Select the weight reflects the applicability of the model under the current task conditions. Thus, according to the selection weights of each neural network model corresponding to the target scenario, the optimal model (i.e., the target neural network model) is automatically selected among each neural network model corresponding to the target scenario. In the FPGA architecture, this selection process is completed by parallel computing of weights, and hardware resources are dynamically allocated to ensure the efficient operation of the optimal model.

[0082] Furthermore, through an adaptive dynamic acceleration strategy, an appropriate amount of computing resources is allocated to the selected model to optimize the processing speed and power consumption during recognition. Specifically, for the convolutional calculation of each layer of the network, the present invention proposes a parallel block convolution acceleration algorithm. Let the input feature map of the current convolutional layer be , and the convolutional kernel weight be , where , and are the height, width, and number of channels of the input feature map respectively, is the convolutional kernel size, is the number of output channels. The algorithm divides the input feature map into multiple sub-blocks , and each sub-block performs convolutional calculation independently. The formula is as follows:

[0083] ......(10)

[0084] where, is the convolutional result of the sub-block . The parallel computing units in the FPGA process multiple sub-block convolution operations simultaneously, accelerating each layer of convolutional calculation. In addition, to further improve the computing efficiency, on-chip memory can be used on the FPGA to store the convolutional kernel weights, avoiding frequent memory access, thereby reducing the latency of data transmission.

[0085] The neural network acceleration module 13 is also used to allocate computing resources to each layer according to the load situation of each layer of the target neural network model after the target neural network model is selected, so that the heavier the load of a layer, the more computing resources it obtains. Specifically, the neural network acceleration module 13 also adds an adaptive computing load allocation algorithm during the neural network inference process to dynamically schedule the hardware resources of the FPGA between different computing tasks, further optimizing the computing performance. The algorithm dynamically adjusts the resource allocation ratio of each computing task by monitoring the load situation of each layer of convolutional and pooling operations. Let the load weight of the current task be , then the resource allocation ratio of each layer is:

[0086] ......(11)

[0087] Among them, is the number of layers of the neural network, represents the computational complexity of the l-th layer. The FPGA can allocate more resources to the layers with higher computational loads and reduce the resource allocation to the low-load layers, thereby balancing the overall computational speed and energy consumption. This algorithm is implemented in the FPGA through a hardware scheduler, and the computational resource allocation for each layer is adaptively optimized.

[0088] In summary, the neural network acceleration module 13 combines model selection, parallel acceleration, and dynamic resource scheduling to achieve high efficiency and low power consumption for deep learning inference on the FPGA. By real-time selecting an adapted model and dynamically adjusting computational resources, the neural network acceleration module can still operate at high speed under the conditions of high system load and complex computations, providing high-precision and low-latency recognition results for image recognition tasks in complex environments. The organic combination of the parallel structure of the FPGA and the algorithm design enables this module to not only have high computational performance but also be flexibly adjustable according to environmental requirements, greatly enhancing the adaptability and stability of the image recognition system.

[0089] The intelligent data scheduling module 14 is used to dynamically manage the data streams of other modules, and through a DDR (Double Data Rate SDRAM) multi-channel arbitration controller, it realizes efficient scheduling of data caching and sending, ensuring that the data throughput of each module reaches the optimal level. At the same time, it balances the data stream under high-load conditions to prevent system bottlenecks.

[0090] Specifically, the intelligent data scheduling module 14 can be used for:

[0091] (1) For each first module in the image preprocessing module 11, the dynamic defogging and enhancement module 12, the neural network acceleration module 13, and the energy consumption management module 15, calculate the load weight of the first module according to the current data requirements, computing power, and average response time of the first module.

[0092] (2) Calculate the buffer size of the first module according to the load weight;

[0093] (3) Allocate a buffer for the first module according to the buffer size.

[0094] Specifically, the intelligent data scheduling module 14 can also be used for:

[0095] After calculating the load weights of each first module, calculate the resource allocation ratio of each first module according to the load weights; allocate computing and storage resources for each first module according to the resource allocation ratio.

[0096] Specifically, the intelligent data scheduling module 14 can also be used for:

[0097] (1) For any second module among the image preprocessing module 11, the dynamic dehazing enhancement module 12, the neural network acceleration module 13, and the energy consumption management module 15, predict the data demand of the second module at the next moment according to the historical data correlation coefficient of the second module and the input data stream at the previous moment.

[0098] (2) Based on the data demand at the next moment, preload the target data required by the second module at the next moment to complete the loading of the target data before the data demand of the target data is generated.

[0099] The intelligent data scheduling module 14 is specifically introduced below.

[0100] First, the algorithm dynamically adjusts the priority of data allocation by calculating the load weights of each module. Let the load weights of each module be , then Adaptive adjustment is made according to the real-time data demand, computing power, and response speed of the module:

[0101] ...... (12)

[0102] Among them, is the current data demand of module , is the computing power of the module, is the average response time of the module. The load weight reflects the demand priority of each module for data resources under the current task conditions. The scheduling control unit in the FPGA ensures the preferential allocation of data resources among high-load modules by parallel computing the value of each module.

[0103] During the data sending process, the DDR multi-channel arbitration controller plays a key role. Its dynamic buffer management strategy based on priority is used to avoid data sending bottlenecks. Specifically, the module allocates a buffer for each data channel, and the size of the buffer is adjusted according to the load weights of each module. Let the current buffer size be , then:

[0104] ...... (13)

[0105] Among them, is the basic buffer size, is the adjustment coefficient. This adaptive buffering strategy combines the scheduling function of the DDR multi-channel arbitration controller, ensuring the stability of the data stream under bursty high-load conditions, effectively avoiding transmission blockages, and guaranteeing the efficient flow of data. At the same time, the FPGA manages the buffers of multiple data channels in parallel, enabling fast writing and reading of data through on-chip high-speed caches, ensuring that the system can maintain a stable transmission efficiency even under high data traffic.

[0106] Moreover, the algorithm introduces a real-time data prefetching algorithm to further improve the efficiency of data scheduling. By analyzing the characteristics of the data stream at the previous moment, it predicts the next data requirements of each module and preloads the required data from memory in advance. Assume that the input data stream at the previous moment is , then the predicted data requirements for the next moment are , and the formula is:

[0107] ...... (14)

[0108] where is the historical data correlation coefficient, which is used to control the amount of data prefetching. Through the high-speed data channels of the FPGA, the system can complete data loading before the data requirements occur, reducing access latency and improving processing speed. The prediction mechanism effectively reduces the data waiting time between modules, ensuring that the system can still respond quickly when the task volume changes.

[0109] In addition, the real-time data prefetching mechanism of the present invention can be replaced with a cache optimization-based strategy. Specifically, the cache controller preloads key data in the data stream in advance to reduce the waiting time for data transmission. For example, a hierarchical high-speed cache strategy is adopted to load data into different levels of caches (each level corresponds to a layer of the target neural network model, and the cache at any level will be placed in the corresponding neural network model layer) to achieve a similar prefetching effect.

[0110] Furthermore, to further optimize data transmission, the intelligent data scheduling module adopts a resource allocation and arbitration algorithm. By adjusting the resource allocation ratio inside the FPGA, high-load modules can obtain more computing and storage resource support. Let the current resource allocation ratio be , then is calculated through the load weight as follows:

[0111] ...... (15)

[0112] where Nis the total number of modules in the system. When the DDR multi-channel arbitration controller executes, it combines the priority arbitration mechanism to ensure that high-load modules obtain the required resources, avoid resource waste, achieve dynamic optimization and allocation of resources, and ensure the stable operation of the system under high-load conditions. This process is implemented in the resource scheduler of the FPGA, and the resource usage of each module is optimized and dynamically allocated, ensuring the stable operation of the system under high load.

[0113] The energy consumption management module 15 is used to manage the energy consumption of other modules.

[0114] Specifically, the energy consumption management module 15 can be used for:

[0115] (1) For any third module among the image preprocessing module 11, the dynamic haze removal and enhancement module 12, the neural network acceleration module 13, and the intelligent data scheduling module 14, calculate the energy consumption weight of the third module according to the instantaneous power consumption, load level, and execution time of the current task of the third module.

[0116] (2) According to the energy consumption weight and the current power state of the third module, determine whether to switch the power state of the third module; the power states include high power state, medium power state, and low power state.

[0117] (3) When it is necessary to switch the power state of the third module, perform a power state switch on the third module.

[0118] Specifically, the energy consumption management module 15 can also be used for:

[0119] Calculate the power distribution ratio of the third module according to the priority of the current task and the energy consumption weight of the third module; allocate electric energy to the third module according to the power distribution ratio.

[0120] The above solutions will be specifically described below.

[0121] The energy consumption management module 15 adjusts its power state by monitoring the power consumption requirements and task execution conditions of each module. First, the energy consumption management module 15 monitors the real-time load status of each module and calculates its energy consumption weight according to the workload and current power consumption of each module :

[0122] ......(16)

[0123] Among them, represents the instantaneous power consumption of module , is the load level of module , is module 's execution time for the current task. The energy consumption weight It reflects the energy consumption priorities of each module under the current task conditions, providing a basis for subsequent energy consumption allocation and power management. The monitoring unit in the FPGA calculates the of each module in real time to ensure that energy consumption resources are preferentially allocated to critical modules.

[0124] In terms of power consumption control, the energy consumption management module 15 adopts the power state adjustment mechanism proposed in the present invention to dynamically reduce the energy consumption of the system by quickly switching the power state. This mechanism divides the power state of the FPGA into three levels: high power state, medium power state, and low power state, and switches according to the of each module and the overall energy consumption demand. Let the switching threshold of the power state be , then the power state of the system switches when the following conditions are met:

[0125] ......(17)

[0126] Among them, and are the switching thresholds of the high power state and the low power state respectively. The power management unit of the FPGA calculates the power state of each module through this formula and triggers a state switch when the conditions are met, reducing the overall power consumption of the system.

[0127] In addition, the energy consumption management module 15 further reduces the total energy consumption of the system by recovering unused electrical energy. Based on the electrical energy distribution and reuse mechanism of the FPGA, the energy consumption management module 15 monitors the power consumption fluctuations of each module during low load or standby, recovers the uncompletely used electrical energy, and uses it to support the high load tasks of other modules. Let the unused electrical energy be , then the energy recovery ratio is:

[0128] ......(18)

[0129] Among them, is the total power consumption of the system. The electrical energy distributor in the FPGA uses this recovery ratio to redistribute the recovered electrical energy to high load modules, thereby achieving efficient utilization and dynamic balance of electrical energy, enabling the system to maintain a stable power consumption during load fluctuations.

[0130] In addition, the energy consumption management module 15 also combines a power management mechanism based on task priorities to ensure that high priority tasks obtain stable power consumption support. Let the priority of the task be , then the electrical energy distribution ratio of each module and the energy consumption weight is dynamically adjusted according to the priority of its current task. The electrical energy distribution ratio is:

[0131] ...... (19)

[0132] Wherein, N is the total number of modules. This power management mechanism ensures that high-priority tasks can still be executed efficiently with limited power consumption by dynamically adjusting the power distribution ratio, thus meeting the power requirements of critical tasks while ensuring the overall low power consumption of the system.

[0133] It can be seen that the energy consumption management module improves the adaptability and energy consumption efficiency of the system under different load conditions through refined power management and power recovery strategies, providing reliable support for the application of the image recognition system in embedded and edge computing scenarios.

[0134] In summary, in the embodiments of the present invention, the above technical solutions are adopted. The energy consumption management module automatically adjusts its power state according to the energy consumption weights of each module, reducing or even avoiding energy waste while meeting the energy consumption requirements of the modules; the energy consumption management module also redistributes the unused electrical energy to other modules through the energy recovery mechanism, further avoiding energy waste; the energy consumption management module also allocates electrical energy according to the priority of each module for the current task and its energy consumption weight, so as to meet the power requirements of critical tasks while ensuring the overall low power consumption of the system.

[0135] In addition, the intelligent data scheduling module determines the buffer size and resource allocation ratio of each module according to the load weights of each module, enabling the system to dynamically allocate data transmission resources according to the load weights of each module in high-load scenarios, avoiding data congestion, and improving the stability of the image recognition speed; the intelligent data scheduling module also predicts the data requirements of each module according to the task load and pre-loads data through the real-time data prefetch mechanism, further reducing the data waiting time, enabling the system to still achieve fast response and data fluency even in a high-concurrency task environment, and this advantage significantly improves the performance of the system in real-time applications.

[0136] The multi-scale edge enhancement algorithm and adaptive bilateral denoising algorithm in the image preprocessing module enable the system to dynamically adjust the edge and denoising effects during image preprocessing according to different environments. For example, in a complex background, multi-scale edge detection can enhance the clarity of key details, while the adaptive bilateral denoising algorithm retains important image edge information while removing noise. This adaptability of image preprocessing ensures that the system can still maintain high recognition accuracy in low-light, uneven illumination, or complex background environments, improving the reliability and environmental adaptability of recognition.

[0137] The neural network acceleration module can automatically select a target neural network model suitable for the current task from multiple neural network models, enabling the system to quickly switch between different models according to the task requirements, and making the system show higher generality and adaptability in multi-task processing and different application scenarios; the neural network acceleration module also adopts a parallel block convolution acceleration algorithm, which improves the recognition speed of the target neural network model; the neural network acceleration module also allocates computing resources to each layer according to the load situation of each layer of the target neural network model, so that the heavier the load of a layer, the more computing resources it gets, which improves the balance of computing resource allocation.

[0138] Based on a general inventive concept, the present invention also provides an image recognition method. This image recognition method is implemented by the above-mentioned image recognition system. Figure 2 It is a schematic flowchart of an image recognition method provided by an embodiment of the present invention. As Figure 2 shown, this process includes:

[0139] Step 201: After receiving the image to be recognized, the image preprocessing module preprocesses the image to be recognized and sends the preprocessed image to the dynamic defogging and enhancement module.

[0140] Step 202: The dynamic defogging and enhancement module determines whether the preprocessed image needs to be defogged and enhanced; when the preprocessed image needs to be defogged and enhanced, step 203 is executed, and when the preprocessed image does not need to be defogged and enhanced, step 204 is executed.

[0141] Step 203: The dynamic defogging and enhancement module performs defogging and enhancement processing on the preprocessed image and sends the defogged and enhanced image to the neural network acceleration module.

[0142] Step 204: The dynamic defogging and enhancement module sends the preprocessed image to the neural network acceleration module. That is, when the preprocessed image does not need to be defogged and enhanced, the dynamic defogging and enhancement module directly sends the image it receives to the neural network acceleration module.

[0143] Step 205: The neural network acceleration module selects a target neural network model suitable for the current image from multiple neural network models and recognizes the current image through the target neural network model; the target neural network model recognizes the current image based on the parallel block convolution acceleration algorithm.

[0144] Optionally, the image recognition method of the embodiment of the present invention further includes:

[0145] Before the neural network acceleration module recognizes the current image through the target neural network model, the intelligent data scheduling module uses a real-time data prefetching algorithm to preload in advance the data required by the target neural network model when recognizing the current image, and sends the required data to the target neural network model, so that the target neural network model can recognize the current image according to the required data, improving the image recognition efficiency.

[0146] It should be noted that the present image recognition method and the foregoing image recognition system are based on a general inventive concept and have the same or corresponding implementation processes. Therefore, the specific solutions of the present image recognition method can be referred to the foregoing embodiments and will not be elaborated herein.

[0147] It can be understood that the same or similar parts in the above embodiments can be referred to each other, and the content not detailed in some embodiments can be referred to the same or similar content in other embodiments.

[0148] It should be noted that in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "a plurality of" refers to at least two.

[0149] Any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process. And the scope of the preferred embodiments of the present invention includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the technical field of the embodiments of the present invention.

[0150] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following well-known technologies in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0151] Those of ordinary skill in the art of the present technology can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0152] In addition, each functional unit in various embodiments of the present invention may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0153] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, or the like.

[0154] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0155] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. An image recognition system, characterized in that, Including: An image preprocessing module, a dynamic haze removal and enhancement module, a neural network acceleration module, an intelligent data scheduling module, and an energy consumption management module; The image preprocessing module is used to preprocess the image to be recognized and send the preprocessed image to the dynamic haze removal and enhancement module; The dynamic haze removal and enhancement module is used to determine whether the preprocessed image needs to be haze-removed and enhanced. When the preprocessed image needs to be haze-removed and enhanced, it performs haze-removal and enhancement processing on the preprocessed image and sends the haze-removed and enhanced image to the neural network acceleration module. When the preprocessed image does not need to be haze-removed and enhanced, it sends the preprocessed image to the neural network acceleration module; The neural network acceleration module is used to select a target neural network model suitable for the current image from multiple neural network models and recognize the current image through the target neural network model; the target neural network model recognizes the current image based on the parallel block convolution acceleration algorithm; The intelligent data scheduling module is used to dynamically manage the data streams of other modules; The energy consumption management module is used to manage the energy consumption of other modules; The intelligent data scheduling module specifically is used for: For any one of the image preprocessing module, the dynamic haze removal and enhancement module, the neural network acceleration module, and the energy consumption management module, calculate the load weight of the module according to the current data demand, computing power, and average response time of the module; Calculate the buffer size of the module according to the load weight; Allocate a buffer for the module according to the buffer size.

2. The image recognition system according to claim 1, characterized in that, The image preprocessing module specifically is used to perform at least one of the following preprocessing operations on the image to be recognized: Color space conversion, edge detection and enhancement, bilateral filtering, erosion and dilation, corner detection, image registration, and seamless zooming.

3. The image recognition system according to claim 1, characterized in that, The dynamic haze removal and enhancement module specifically performs haze-removal and enhancement processing on the preprocessed image based on the dark channel dehazing technology and the light adaptation adjustment strategy.

4. The image recognition system according to claim 1, wherein After selecting the target neural network model, the neural network acceleration module is also used to allocate computing resources to each layer of the target neural network model according to the load situation of each layer of the target neural network model, so that the heavier the load of a layer, the more computing resources it obtains.

5. The image recognition system according to claim 1, wherein The intelligent data scheduling module specifically is also used for: After calculating the load weights of each module, calculate the resource allocation ratio of each module according to the load weights; Allocate computing and storage resources to each module according to the resource allocation ratio.

6. The image recognition system according to any one of claims 1 to 5, characterized in that The intelligent data scheduling module specifically is also used for: For any one of the image preprocessing module, the dynamic haze removal and enhancement module, the neural network acceleration module, and the energy consumption management module, predict the next moment data demand of the module according to the historical data correlation coefficient of the module and the input data stream at the previous moment; Based on the next moment data demand, preload the target data required by the module at the next moment to complete the loading of the target data before the data demand of the target data is generated.

7. The image recognition system according to claim 1, wherein The energy consumption management module is specifically configured to: For any one of the image preprocessing module, the dynamic haze removal and enhancement module, the neural network acceleration module, and the intelligent data scheduling module, calculate the energy consumption weight of the module according to the instantaneous power consumption, the load level, and the execution time of the current task of the module; According to the energy consumption weight and the current power state of the module, determine whether to switch the power state of the module; the power state includes a high power state, a medium power state, and a low power state; When it is necessary to switch the power state of the module, perform a power state switch on the module.

8. The image recognition system according to claim 7, wherein The energy consumption management module is specifically further configured to: Calculate the power distribution ratio of the module according to the priority of the current task of the module and the energy consumption weight; Allocate electric energy to the module according to the power distribution ratio.

9. An image recognition method, characterized in that, Applied to the image recognition system according to any one of claims 1 to 8, the image recognition method includes: After receiving the image to be recognized, the image preprocessing module preprocesses the image to be recognized and sends the preprocessed image to the dynamic haze removal and enhancement module; The dynamic haze removal and enhancement module determines whether the preprocessed image needs to be subjected to haze removal and enhancement processing; When the preprocessed image needs to be subjected to haze removal and enhancement processing, the dynamic haze removal and enhancement module performs haze removal and enhancement processing on the preprocessed image and sends the haze-removed and enhanced image to the neural network acceleration module; when the preprocessed image does not need to be subjected to haze removal and enhancement processing, the dynamic haze removal and enhancement module sends the preprocessed image to the neural network acceleration module; The neural network acceleration module selects a target neural network model applicable to the current image from multiple neural network models and recognizes the current image through the target neural network model; the target neural network model recognizes the current image based on the parallel block convolution acceleration algorithm.

Images