An image processing method and system based on an FPGA chip

Abstract

This invention relates to an image processing method and system based on an FPGA chip, belonging to the field of image processing technology. The method includes: acquiring image processing requirements; selecting a target image processing algorithm based on the requirements; configuring image data transmission parameters according to the target algorithm; synchronously sending the target algorithm and image data transmission parameters to a preset FPGA chip to complete function initialization; receiving raw image data transmitted from a preset external image acquisition device; sending the raw image data to the preset FPGA chip for processing, generating processed image data, and forwarding it for output. This invention improves the timeliness of image processing for large image data.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method and system based on an FPGA chip. Background Technology

[0002] Image processing refers to a series of operations such as acquisition, conversion, analysis, and optimization of image signals using computers or dedicated hardware devices. By processing data such as pixel information, color features, and spatial structure of images, it can achieve functions such as image enhancement, noise reduction, segmentation, recognition, and compression to meet the needs of image quality or information extraction in different scenarios. It is widely used in fields such as semiconductor equipment, industrial vision, medical imaging, and security monitoring.

[0003] Currently, image processing algorithms are generally developed using GPU-based solutions. These algorithms are broken down into multiple thread blocks and distributed to different computing cores for parallel execution, thereby enabling image data processing.

[0004] When there are data dependencies between threads (such as edge detection requiring reference to neighboring pixels), data needs to be synchronized through shared memory, causing some threads to wait and reducing parallel efficiency. Therefore, when using GPUs for large image data processing, the timeliness of image processing is easily reduced. Summary of the Invention

[0005] To improve the timeliness of image processing for large image data, this invention provides an image processing method and system based on an FPGA chip.

[0006] In a first aspect, the present invention provides an image processing method based on an FPGA chip, which adopts the following technical solution:

[0007] An image processing method based on an FPGA chip includes:

[0008] S1: Obtain image processing requirements;

[0009] S2: Select the target image processing algorithm according to the image processing requirements;

[0010] S3: Configure image data transmission parameters according to the target image processing algorithm;

[0011] S4: The target image processing algorithm and the image data transmission parameters are synchronously sent to the preset FPGA chip to complete the function initialization;

[0012] S5: Receive raw image data transmitted from a preset external image acquisition device;

[0013] S6: Send the original image data to a preset FPGA chip for processing, generate processed image data, and forward it for output.

[0014] Optionally, the method for selecting the target image processing algorithm includes:

[0015] S21: Retrieve the processing object and processing location from the image processing requirements;

[0016] S22: Determine the required environment and initial algorithm according to the processing object;

[0017] S23: Retrieve the location environment from the processing location;

[0018] S24: Compare the required environment with the location environment to obtain environmental deviation information;

[0019] S25: Combining the deviation information between the processing object and the environment, select an environment adaptation algorithm from the initial algorithm and use the environment adaptation algorithm as the target image processing algorithm.

[0020] Optionally, the method for selecting the environment adaptation algorithm includes:

[0021] S251: Retrieve the illumination deviation value and electromagnetic interference deviation value from the environmental deviation information;

[0022] S252: Determine the unit value of light and the unit value of electromagnetic interference according to the object being processed;

[0023] S253: Calculate the product of the illumination deviation value and the unit value of the light ray, and use it as the illumination reference value;

[0024] S254: Calculate the product of the electromagnetic interference deviation value and the electromagnetic interference unit value, and use it as the electromagnetic interference reference value;

[0025] S255: Determine a comprehensive reference value based on the illumination reference value and the electromagnetic interference reference value;

[0026] S256: Retrieve the reference interval according to the initial algorithm;

[0027] S257: Based on the case where the comprehensive reference value falls within the reference benchmark interval, select the corresponding initial algorithm as the fall-in selection algorithm, and use the fall-in selection algorithm as the environment adaptation algorithm.

[0028] Optionally, after using the fall-in selection algorithm as the environment adaptation algorithm, the algorithm further includes:

[0029] S2571: Retrieve the number of falls based on the fall selection algorithm;

[0030] S2572: When the number of occurrences exceeds the preset baseline number, the corresponding reference baseline interval is used as the occurrence baseline interval;

[0031] S2573: Determine the intermediate value, maximum value, and minimum value according to the reference interval;

[0032] S2574: Calculate the deviation between the comprehensive reference value and the median, maximum, and minimum values ​​to obtain the median deviation value, maximum deviation value, and minimum deviation value;

[0033] S2575: Generate a deviation reference value by combining the maximum deviation value, the minimum deviation value, and the intermediate deviation value;

[0034] S2576: Sort the deviation reference values ​​from largest to smallest, and select the first-to-last value corresponding to the fall-in selection algorithm to replace the environment adaptation algorithm.

[0035] Optionally, the method for generating the deviation reference value includes:

[0036] S25751: Compare the maximum deviation value with the minimum deviation value, and take the smaller value as the end selection deviation value;

[0037] S25752: Calculate the difference between the intermediate deviation value and the selected end deviation value, and use it as the mid-end offset value;

[0038] S25753: Determine the offset ratio value based on the midpoint offset value and the falling into the reference interval;

[0039] S25754: Determine the intermediate deviation ratio based on the intermediate deviation value and the falling within the reference range;

[0040] S25755: The offset ratio value and the intermediate deviation ratio value are weighted and calculated to obtain the deviation reference value.

[0041] Optionally, the method for generating the processed image data includes:

[0042] S61: Retrieve the interface transmission mode, data format parameters, and cache scheduling parameters from the image data transmission parameters;

[0043] S62: Transmit the original image data to a preset FPGA chip via the interface transmission method;

[0044] S63: Adapt the original image data to the data format parameters to generate processed format image data;

[0045] S64: Cache the processed format image data according to the cache scheduling parameters to form cached image data;

[0046] S65: The cached image data is parsed and processed using the target image processing algorithm to obtain parsed image data, and the parsed image data is used as the processed image data.

[0047] Optionally, after using the parsed image data as the processed image data, the method further includes:

[0048] S71: Retrieve the image clarity and processing time from the processed image data;

[0049] S72: Retrieve the required image clarity and required processing time from the image processing requirements;

[0050] S73: Calculate the difference between the image sharpness and the required sharpness, as the sharpness deviation value;

[0051] S74: Calculate the difference between the time consumed by the demand and the time consumed by the processing, and use it as the time deviation value;

[0052] S75: Generate an image processing adjustment value by combining the sharpness deviation value and the time consumption deviation value;

[0053] S76: Correct the deviation reference value with the image processing adjustment value, and reselect the environment adaptation algorithm.

[0054] Optionally, after using the parsed image data as the processed image data, the method further includes:

[0055] S81: Data reception rate, processing frame rate, and algorithm running status;

[0056] S82: Retrieve the receiving reference rate and processing reference frame rate from the target image processing algorithm;

[0057] S83: Calculate the difference between the data receiving rate and the receiving reference rate as the rate deviation value;

[0058] S84: Calculate the difference between the processed frame rate and the processed baseline frame rate as the frame rate deviation value;

[0059] S85: Combine the rate deviation value, the frame rate deviation value, and the algorithm running status to generate processing display information and send it to the terminal held by the administrator.

[0060] Optionally, the method for generating the processing display information includes:

[0061] S851: Determine the status display information, rate reference deviation range, and frame rate reference deviation range according to the algorithm's running status;

[0062] S852: Determine the rate anomaly value based on the rate deviation value and the rate reference deviation range;

[0063] S853: Determine the frame rate anomaly value based on the frame rate deviation value and the frame rate reference deviation range;

[0064] S854: Combine the rate anomaly value and the frame rate anomaly value to generate synchronization anomaly display information;

[0065] S855: Merge the synchronization error display information and the status display information and use them as processing display information.

[0066] Secondly, this invention provides an image processing system based on an FPGA chip, employing the following technical solution:

[0067] An image processing system based on an FPGA chip includes:

[0068] The acquisition module is used to acquire image processing requirements, raw image data, data receiving rate, processing frame rate, and algorithm running status.

[0069] The memory stores a program for implementing an image processing method based on an FPGA chip as described in any one of the first aspects;

[0070] The processor loads and executes programs stored in memory.

[0071] In summary, the present invention has at least one of the following beneficial technical effects:

[0072] 1. By acquiring image processing requirements and selecting target image processing algorithms to configure image data transmission parameters, the preset FPGA chip is first initialized. Then, the raw image data acquired by the external image acquisition device is processed to generate processed image data and forwarded for output. Thus, the FPGA chip is used to process the image, improving the timeliness of image processing for large image data.

[0073] 2. By retrieving the processing object and processing location from the image processing requirements, the required environment and initial algorithm are determined by the processing object, and the location environment is retrieved by the processing location. Then, environmental deviation information is obtained through analysis. After combining the processing object and environmental deviation information, an environment-adaptive algorithm is selected from the initial algorithm and used as the target image processing algorithm to improve the accuracy of the obtained target image processing algorithm.

[0074] 3. By collecting the data reception rate and processing frame rate during the processing, processing display information is generated and sent to the administrator, which makes it easier for the administrator to monitor the system's operating status in real time and promptly identify and resolve potential performance bottlenecks, algorithm malfunctions, and other issues. Attached Figure Description

[0075] Figure 1 This is a flowchart of an image processing method based on an FPGA chip.

[0076] Figure 2 This is a flowchart illustrating the method for selecting the target image processing algorithm;

[0077] Figure 3 This is a flowchart of the method for generating processed image data. Detailed Implementation

[0078] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0079] An image processing method based on an FPGA chip acquires information such as image processing requirements, raw image data, data reception rate, processing frame rate, and algorithm running status to extract the processing object and location information. It then compares the required environment with the actual location environment to obtain environmental deviations such as illumination and electromagnetic interference, calculates a comprehensive reference value, and selects a suitable target image processing algorithm from the initial algorithms. Next, based on the target image processing algorithm, it configures image data transmission parameters such as interface transmission mode, data format, and buffer scheduling, and synchronously sends the algorithm and parameters to the FPGA chip to complete functional initialization. Finally, it receives raw image data from an external image acquisition device. After completing interface transmission, format adaptation, and caching according to the transmission parameters, the FPGA chip calls the target image processing algorithm to perform hardware-based calculations to generate processed image data and forward it for output. At the same time, by comparing the clarity / time consumption of the processed image with the required standards, adjustment values ​​are generated to correct the algorithm selection logic to optimize the algorithm. It also collects running data such as data receiving rate and processing frame rate, and combines the algorithm running status to generate processing display information containing abnormal information and push it to the administrator terminal, thereby improving the timeliness of image processing for large image data and providing high-precision, high-real-time image processing solutions for scenarios such as industrial inspection and semiconductor defect detection.

[0080] Reference Figure 1 This invention discloses an image processing method based on an FPGA chip, comprising:

[0081] S1: Obtain image processing requirements.

[0082] Image processing requirements refer to the specific set of requirements from users or application scenarios regarding the final image processing effect, processing efficiency, and suitability for the scenario. Image processing requirements include the processing object, processing location, required image clarity, and required processing time.

[0083] Image processing requirements are provided to users through a visual interactive interface on the host computer, which provides a requirement input template for users to input and obtain the results.

[0084] S2: Select the target image processing algorithm based on the image processing requirements.

[0085] Among them, the target image processing algorithm refers to the algorithm that can meet specific image processing requirements.

[0086] By selecting the target image processing algorithm based on the image processing requirements, it becomes easier to use it subsequently.

[0087] The target image processing algorithm can be directly selected by the operator based on the image processing requirements and practical experience, or it can be selected by matching from a preset algorithm library based on the image processing requirements.

[0088] For example, when the image processing requirement is for noisy industrial images, the target image processing algorithm can be a noise reduction algorithm. When the image processing requirement is for semiconductor wafer images, the target image processing algorithm can be a defect detection algorithm.

[0089] To further ensure the rationality of the target image processing algorithm, it is necessary to perform further separate analysis and calculation on the target image processing algorithm, which will be explained in detail through the following steps.

[0090] Reference Figure 2 The method for selecting the target image processing algorithm includes the following steps:

[0091] S21: Retrieve the processing object and processing location from the image processing requirements.

[0092] The processing object refers to the specific entity or scene type corresponding to the image that requires technical processing in the image processing requirements. Processing objects include semiconductor wafers, industrial product parts, and specific organs in medical images.

[0093] The processing location refers to the actual physical environment location specified in the image processing requirements, upon which image acquisition and subsequent processing are based. Processing locations include locations such as industrial production lines, semiconductor cleanrooms, and medical image acquisition rooms.

[0094] By retrieving the processing object and processing location from the image processing requirements, it is convenient for subsequent use.

[0095] S22: Determine the required environment and initial algorithm based on the processing object.

[0096] The "demand environment" refers to the standard parameters corresponding to the environmental conditions that image acquisition and processing must meet, based on the characteristics of the processing object and the desired processing effect. The "initial algorithm" refers to a set of image processing algorithms with adaptability potential, initially selected based on the type of processing object and core processing requirements.

[0097] By inputting the processing object into a preset processing object characteristic database, the required environment and initial algorithm are obtained through matching, which facilitates subsequent use.

[0098] The processing object characteristic database pre-stores a mapping table of different processing objects, their corresponding required environments, and initial algorithms. The processing object characteristic database is retrieved by the operator after selecting and pre-storing the required environments and initial algorithms based on the core attributes of different processing objects.

[0099] For example, when the object being processed is a semiconductor wafer, there are requirements such as micron-level defect detection, sensitivity to light uniformity, and high resistance to electromagnetic interference. Therefore, the required environment is set with "light intensity of 500 to 800 lux, electromagnetic interference ≤10dB, and temperature of 23±2℃" in accordance with semiconductor testing industry standards. The semiconductor wafer defect detection algorithm and high-precision edge detection algorithm are used as the initial algorithms, and then the corresponding lookup table is formed and stored in the processing object characteristic database.

[0100] S23: Retrieve the location environment from the processing location.

[0101] Among them, the location environment refers to the set of detection parameters of the objective environmental conditions of the processing location, including real-time light intensity, electromagnetic interference intensity, ambient temperature, ambient humidity, etc.

[0102] The location environment is obtained by retrieving sensors preset at that location through location processing, facilitating subsequent use. Sensors include light sensors, electromagnetic interference detectors, and temperature and humidity sensors. These sensors are deployed synchronously by the operator when setting up external image acquisition equipment. External image acquisition equipment refers to devices that acquire images that need to be processed; such equipment can be cameras or video cameras.

[0103] S24: Compare the required environment with the location environment to obtain environmental deviation information.

[0104] Among them, environmental deviation information refers to the set of difference data between the demand environment and the location environment obtained by comparing the two. Environmental deviation information includes key parameters such as illumination deviation value and electromagnetic interference deviation value.

[0105] By extracting different types of environmental parameters from the demand environment and the location environment, and comparing the same type of environmental parameters in the demand environment and the location environment to obtain the deviation value, and then integrating the deviation values ​​of each type to form environmental deviation information, which is convenient for subsequent use.

[0106] S25: Combining the deviation information between the processing object and the environment, select the environment adaptation algorithm from the initial algorithm and use the environment adaptation algorithm as the target image processing algorithm.

[0107] Among them, the environment adaptation algorithm refers to the algorithm selected from the initial algorithm set that can adapt to the actual environment of the processing location and match the characteristics of the processing object.

[0108] By combining the processing object with environmental deviation information for analysis, and selecting an initial algorithm to obtain an environment-adapted algorithm, and then using the environment-adapted algorithm as the target image processing algorithm, the accuracy of the obtained target image processing algorithm is improved.

[0109] To further ensure the rationality of the environment adaptation algorithm, it is necessary to perform further separate analysis and calculation on the environment adaptation algorithm, which will be explained in detail through the following steps.

[0110] The selection method for environment adaptation algorithms includes the following steps:

[0111] S251: Retrieve illumination deviation value and electromagnetic interference deviation value from environmental deviation information.

[0112] Among them, the illumination deviation value refers to the deviation value corresponding to the existence of deviation in the ambient illumination, and the electromagnetic interference deviation value refers to the deviation value corresponding to the existence of deviation in the intensity of electromagnetic interference in the environment.

[0113] By retrieving the illumination deviation value and electromagnetic interference deviation value from the environmental deviation information, it is convenient for subsequent use.

[0114] S252: Determine the unit values ​​for light and electromagnetic interference based on the object being processed.

[0115] The light unit value is a baseline value set based on the sensitivity of the processed object to changes in illumination, used to quantify the weight of the impact of illumination deviation on image processing results. The electromagnetic interference unit value is a baseline value set based on the tolerance of the processed object to electromagnetic interference, used to quantify the weight of the impact of electromagnetic interference deviation.

[0116] Different processing objects correspond to different light unit values ​​and electromagnetic interference unit values.

[0117] By inputting the object to be processed into a preset database of object characteristics for matching, the unit values ​​of light and electromagnetic interference are obtained for convenient subsequent use.

[0118] The object characteristic database pre-stores a lookup table of different objects and their corresponding light and electromagnetic interference unit values. The object characteristic database is retrieved by the operator after setting and pre-storing the light and electromagnetic interference unit values ​​based on the core attributes of different objects.

[0119] For example, when processing semiconductor wafers, the light unit value can be set to 0.7, and the electromagnetic interference unit value can be set to 1.3. When processing ordinary plastic parts, the light unit value can be set to 0.3, and the electromagnetic interference unit value can be set to 0.5.

[0120] S253: Calculate the product of the illumination deviation value and the unit value of the light ray, and use it as the illumination reference value.

[0121] Among them, the illumination reference value refers to the reference value after quantifying the actual impact of illumination deviation on the image processing effect of a specific object.

[0122] The product of the illumination deviation value and the unit value of light is calculated, and the calculation result is used as the illumination reference value for convenient subsequent use.

[0123] S254: Calculate the product of the electromagnetic interference deviation value and the electromagnetic interference unit value, and use it as the electromagnetic interference reference value.

[0124] Among them, the electromagnetic interference reference value refers to the reference value after quantifying the actual impact of electromagnetic interference deviation on the image processing effect of a specific object.

[0125] The product of the electromagnetic interference deviation value and the magnetic interference unit value is calculated, and the calculation result is used as the electromagnetic interference reference value for convenient subsequent use.

[0126] S255: Determine the comprehensive reference value based on the illumination reference value and the electromagnetic interference reference value.

[0127] Among them, the comprehensive reference value refers to the reference value after quantifying the actual impact of illumination deviation and electromagnetic interference deviation on the image processing effect of a specific processing object.

[0128] The sum of the illumination reference value and the electromagnetic interference reference value is calculated, and the calculation result is used as a comprehensive reference value to facilitate subsequent use.

[0129] S256: Retrieve the reference range according to the initial algorithm.

[0130] The reference range refers to the allowable reference range when the initial algorithm is used normally. That is, the reference range that corresponds to when the actual impact of illumination deviation and electromagnetic interference deviation on the image processing effect of a specific processing object will not affect the normal use of the initial algorithm.

[0131] The initial algorithm is input into a preset reference database to obtain a reference reference range, which facilitates subsequent use.

[0132] The reference reference database pre-stores a table of different initial algorithms and their corresponding reference reference intervals. The reference reference database is obtained by the operator conducting experiments on different initial algorithms under different lighting and electromagnetic interference conditions to obtain the lighting reference interval and the electromagnetic interference reference interval. The reference reference interval is then obtained by combining the lighting reference interval and the electromagnetic interference reference interval.

[0133] For example, when the initial algorithm is a semiconductor wafer defect detection algorithm, the light reference range obtained from the experiment is [1.5, 6.8] and the electromagnetic interference reference range is [0.8, 5.2]. The sum of the endpoints of the two ranges is calculated as the endpoint of the reference range, thus obtaining the reference range as [2.3, 12].

[0134] S257: Based on the situation where the comprehensive reference value falls within the reference baseline range, select the corresponding initial algorithm as the fall-in selection algorithm, and use the fall-in selection algorithm as the environment adaptation algorithm.

[0135] Among them, the falling-in selection algorithm refers to the initial algorithm selected when falling into the interval.

[0136] By analyzing the cases where the comprehensive reference value falls within the reference baseline range, and using the initial algorithm corresponding to the comprehensive reference value falling within the reference baseline range as the fall-in selection algorithm, and using the fall-in selection algorithm as the environment adaptation algorithm, the accuracy of the obtained environment adaptation algorithm is improved.

[0137] To further ensure the rationality of the environment adaptation algorithm, it is necessary to perform further separate analysis and calculation on the environment adaptation algorithm, which will be explained in detail through the following steps.

[0138] Following the fall-in selection algorithm as the environment adaptation algorithm, the following steps are also included:

[0139] S2571: Retrieve the number of falls based on the fall selection algorithm.

[0140] The number of occurrences refers to the number of occurrences corresponding to the selection algorithm.

[0141] By counting the number of times the selection algorithm is applied, and using the count as the number of times the selection algorithm is applied, it is convenient to use in subsequent applications.

[0142] S2572: When the number of occurrences exceeds the preset baseline number, the corresponding reference baseline interval will be used as the occurrence baseline interval.

[0143] The baseline quantity refers to the quantity that does not require further filtering. The baseline quantity is preset by the operator according to needs.

[0144] When the number of occurrences exceeds the preset baseline number, it indicates that further filtering is required. Therefore, the reference baseline interval corresponding to the occurrence selection algorithm is defined as the occurrence baseline interval to facilitate subsequent use.

[0145] S2573: Determine the median, maximum, and minimum values ​​based on whether they fall within the reference range.

[0146] The maximum value refers to the upper limit of the baseline interval, the minimum value refers to the lower limit of the baseline interval, and the median value refers to the average value between the upper and lower limits of the baseline interval.

[0147] By retrieving the two endpoints that fall within the baseline interval and using them as the maximum and minimum values ​​respectively, and then calculating the average value between the maximum and minimum values ​​and using it as the median value, it is convenient for subsequent use.

[0148] S2574: Calculate the deviation between the comprehensive reference value and the median, maximum, and minimum values ​​to obtain the median deviation value, maximum deviation value, and minimum deviation value.

[0149] The median deviation value refers to the deviation between the comprehensive reference value and the median value. The maximum deviation value refers to the deviation between the comprehensive reference value and the maximum value. The minimum deviation value refers to the deviation between the comprehensive reference value and the minimum value.

[0150] By calculating the differences between the comprehensive reference value and the median, maximum, and minimum values, the results are used as the median deviation value, maximum deviation value, and minimum deviation value, respectively, for convenient subsequent use.

[0151] S2575: Generate a deviation reference value by combining the maximum deviation value, minimum deviation value, and intermediate deviation value.

[0152] Among them, the deviation reference value refers to a single quantitative indicator obtained by standardizing and integrating the maximum deviation value, minimum deviation value, and intermediate deviation value.

[0153] By combining and analyzing the maximum deviation value, minimum deviation value, and intermediate deviation value, a deviation reference value is obtained for convenient subsequent use.

[0154] To further ensure the reasonableness of the deviation reference value, it is necessary to perform a further separate analysis and calculation on the deviation reference value, which will be explained in detail through the steps shown below.

[0155] The method for generating deviation reference values ​​includes the following steps:

[0156] S25751: Compare the maximum deviation value with the minimum deviation value, and take the smaller value as the end selection deviation value.

[0157] Among them, the end selection deviation value refers to the deviation value corresponding to the selection of the maximum deviation value and the minimum deviation value.

[0158] By comparing the maximum and minimum deviation values, and selecting the smaller value as the endpoint deviation value, it is convenient for subsequent use.

[0159] S25752: Calculate the difference between the intermediate deviation value and the selected deviation value at the end, and use it as the intermediate offset value.

[0160] The midpoint offset value refers to the deviation value used to indicate the degree of proximity to the center of the interval.

[0161] The difference between the intermediate deviation value and the selected deviation value at the end is calculated, and the calculation result is used as the intermediate offset value for convenient subsequent use.

[0162] S25753: Determine the offset ratio based on the midpoint offset value and the value falling within the reference range.

[0163] The offset ratio refers to the ratio between the midpoint offset value and the interval span.

[0164] The difference between the maximum and minimum values ​​corresponding to the points falling within the baseline interval is calculated and used as the interval span value. Then, the ratio between the midpoint offset value and the interval span value is calculated and used as the offset ratio value, which facilitates subsequent use.

[0165] S25754: Determine the intermediate deviation ratio based on the intermediate deviation value and the value falling within the reference range.

[0166] The intermediate deviation ratio refers to the ratio between the intermediate deviation value and the interval span.

[0167] By using the ratio between the intermediate deviation value and the interval span value as the intermediate deviation ratio, it is easier to use in subsequent applications.

[0168] S25755: The offset ratio value and the intermediate deviation ratio value are weighted and calculated to obtain the deviation reference value.

[0169] Specifically, the corresponding reference weight value is retrieved by processing the object, and then the offset ratio value and the intermediate deviation ratio value are weighted and calculated. The calculation result is used as the deviation reference value, thereby improving the accuracy of the obtained deviation reference value.

[0170] The reference weight value refers to the weight parameter based on the offset ratio and the intermediate deviation ratio of the processed object.

[0171] The reference weight value is obtained by matching the processed object with a preset processed object characteristic database. The processed object characteristic database pre-stores a lookup table of different processed objects and their corresponding reference weight values. The processed object characteristic database is preset and stored by the operator according to the requirements of the processed object.

[0172] When the processing object requires high precision and is sensitive to environmental fluctuations, the reference weight value corresponding to the intermediate deviation ratio is greater than the reference weight value corresponding to the offset ratio. When the processing object requires frequent environmental fluctuations and high boundary fault tolerance, the reference weight value corresponding to the intermediate deviation ratio is less than the reference weight value corresponding to the offset ratio.

[0173] S2576: Sort the deviation reference values ​​from largest to smallest, and select the first one corresponding to the fall-in selection algorithm to replace the environment adaptation algorithm.

[0174] Specifically, by sorting the deviation reference values ​​from largest to smallest, selecting the first deviation reference value in the sorted results and using the corresponding selection algorithm, and then replacing the environment adaptation algorithm, the accuracy of the obtained environment adaptation algorithm is improved.

[0175] S3: Configure image data transmission parameters according to the target image processing algorithm.

[0176] Image data transmission parameters refer to the set of core parameters configured to ensure efficient and accurate reception of raw image data. These parameters include interface transmission mode, data format parameters, and buffer scheduling parameters. Interface transmission mode refers to the hardware interface protocol and link type used for transmission between the acquisition end, processing end, and storage end. Data format parameters are a set of parameters describing the encoding form of the image data itself. Buffer scheduling parameters are strategic parameters used to manage data buffering during transmission.

[0177] The target image processing algorithm extracts the image data type, input resolution requirements, minimum input frame rate, maximum allowable transmission delay, and tolerance for data errors required for transmission processing. Then, by retrieving the capability boundaries of each hardware component in the preset FPGA chip, the algorithm's requirements are matched with the hardware's capability boundaries. Finally, the matched interface transmission method, data format parameters, and buffer scheduling parameters are combined to form the image data transmission parameters for convenient subsequent use.

[0178] S4: Synchronously send the target image processing algorithm and image data transmission parameters to the preset FPGA chip to complete the function initialization.

[0179] Specifically, by synchronously sending the target image processing algorithm and image data transmission parameters to a preset FPGA chip, and loading the target image processing algorithm and image data transmission parameters through the preset FPGA chip, the preset FPGA chip completes the hardware deployment of the algorithm and the configuration of the transmission link, and switches the preset FPGA chip from a hardware idle state to an image processing ready state, ultimately enabling it to receive image data and execute algorithm processing.

[0180] S5: Receive raw image data transmitted from a preset external image acquisition device.

[0181] Raw image data refers to the original image information captured directly by external image acquisition equipment without any compression or algorithmic processing. External image acquisition equipment refers to devices that acquire images; such devices can be industrial cameras, medical imaging detectors, security cameras, etc.

[0182] By receiving the raw image data, it is convenient for the subsequent pre-programmed FPGA chip to process it.

[0183] S6: Send the raw image data to the preset FPGA chip for processing, generate processed image data, and forward it for output.

[0184] Among them, the processed image data refers to the result data obtained after the original image data is processed by a preset FPGA chip.

[0185] By sending the raw image data to a preset FPGA chip, the raw image data is processed to generate processed image data, and then the processed image data is forwarded and output, thereby improving the timeliness of image processing for large image data.

[0186] To further ensure the rationality of the processed image data, it is necessary to perform further separate analysis and calculation on the processed image data, which will be explained in detail through the steps shown below.

[0187] Reference Figure 3 The method for generating processed image data includes the following steps:

[0188] S61: Retrieve the interface transmission mode, data format parameters, and buffer scheduling parameters from the image data transmission parameters.

[0189] Specifically, the interface transmission method, data format parameters, and cache scheduling parameters are retrieved from the image data transmission parameters to facilitate subsequent use.

[0190] S62: Transmits the original image data to the preset FPGA chip via interface transmission.

[0191] The original image data is transmitted to a preset FPGA chip via an interface transmission method, which facilitates subsequent processing.

[0192] S63: Adapt the original image data to the data format parameters to generate processed image data.

[0193] Among them, the processed format image data refers to the image data generated after the original image data has been converted according to the data format parameters, which meets the processing requirements of the FPGA chip and the input requirements of the target algorithm.

[0194] The hardware format conversion module pre-installed on the FPGA chip verifies and converts the format of the original image data according to the data format parameters, thereby generating processed format image data for convenient subsequent use.

[0195] The hardware format conversion module is a module used to convert formats. The hardware format conversion module is pre-configured on the FPGA chip.

[0196] S64: Based on the cache scheduling parameters, cache the formatted image data to form cached image data.

[0197] Cached image data refers to temporary data that can be directly read by the target image processing algorithm, formed after the processed image data is stored in a cache module pre-installed on the FPGA chip according to the rules of the cache scheduling parameters. The cache module is pre-installed on the FPGA chip and is used for temporary data storage. The cache scheduling parameters include parameters such as cache type, pre-cache trigger threshold, overflow protection threshold, and cache cleanup strategy.

[0198] By storing the processed image data into a cache module pre-set in the FPGA chip according to the cache scheduling parameters, cached image data is obtained for convenient subsequent use.

[0199] S65: Use the target image processing algorithm to parse and process the cached image data to obtain the parsed image data, and use the parsed image data as the processed image data.

[0200] Among them, parsed image data refers to the result data obtained after the cached image data is processed by the target image processing algorithm.

[0201] By employing a target image processing algorithm to parse and process the cached image data, parsed image data is obtained, and this parsed image data is used as the processed image data, thereby improving the accuracy of the obtained processed image data.

[0202] To further ensure the rationality of the processed image data, it is necessary to perform further separate analysis and calculation on the processed image data, which will be explained in detail through the steps shown below.

[0203] After parsing the image data into processed image data, the following steps are also included:

[0204] S71: Retrieve image sharpness and processing time from the processed image data.

[0205] Image sharpness refers to the degree to which details in the processed image data are discernible. Processing time refers to the total time taken from the initial reading of cached image data from the preset FPGA chip to the generation of parsed image data.

[0206] The image sharpness is obtained by retrieving the corresponding sharpness of the processed image data and using it as the image sharpness. The start time and generation time are also retrieved, and the time difference is calculated as the processing time, which is convenient for subsequent use.

[0207] S72: Retrieve the required sharpness and processing time from the image processing requirements.

[0208] Among them, the required clarity refers to the minimum level of detail that the processed image must reach, as specified in the image processing requirements. The required time refers to the maximum allowable time from the input of image data to the preset FPGA chip to the generation of the processed image data, as specified in the image processing requirements.

[0209] The image processing requirements are retrieved based on the required image clarity and processing time, making it convenient for subsequent use.

[0210] S73: Calculate the difference between the image sharpness and the required sharpness, and use it as the sharpness deviation value.

[0211] Among them, the sharpness deviation value refers to the deviation value corresponding to the existence of a sharpness deviation.

[0212] The difference between the image sharpness and the required sharpness is calculated, and the calculation result is used as the sharpness deviation value for convenient subsequent use.

[0213] S74: Calculate the difference between the required time and the processing time, and use it as the time deviation value.

[0214] Among them, the time consumption deviation value refers to the deviation value corresponding to the time consumption deviation.

[0215] The difference between the time taken to fulfill the request and the time taken to process it is calculated, and the calculation result is used as the time deviation value for convenient subsequent use.

[0216] S75: Combines sharpness deviation value and time consumption deviation value to generate image processing adjustment value.

[0217] Among them, the image processing adjustment value refers to the adjustment value corresponding to the deviation reference value when it is necessary to adjust it.

[0218] By weighting the sharpness deviation value and the time consumption deviation value, and using the calculation result as the image processing adjustment value, it is convenient for subsequent use.

[0219] By retrieving the corresponding clarity weight value and time consumption weight value from the processed object, and then weighting the offset ratio value and the intermediate deviation ratio value based on the clarity weight value and time consumption weight value, the calculation result is used as the deviation reference value, thereby improving the accuracy of the obtained deviation reference value.

[0220] Sharpness weight value refers to the weighting parameter based on the sharpness deviation value of the processed object. Time consumption weight value refers to the weighting parameter based on the time consumption deviation value of the processed object.

[0221] By inputting the processing objects into a pre-defined processing object characteristic database for matching, a clear weight value and a time consumption weight value are obtained. The processing object characteristic database pre-stores a mapping table of different processing objects and their corresponding clear weight values ​​and time consumption weight values. The processing object characteristic database is pre-set and stored by the operator according to the requirements of the processing objects.

[0222] For example, when processing medical impact diagnosis, the clarity weight value can be set to 0.6 and the time consumption weight value can be set to 0.4. When processing industrial real-time detection, the clarity weight value can be set to 0.4 and the time consumption weight value can be set to 0.6. When processing civilian security monitoring, the clarity weight value can be set to 0.5 and the time consumption weight value can be set to 0.5.

[0223] S76: Correct the deviation reference value with image processing adjustment value and reselect the environment adaptation algorithm.

[0224] Specifically, the accuracy of the acquired environment adaptation algorithm is improved by calculating the sum between the image processing adjustment value and the deviation reference value, replacing the deviation reference value with the calculation result, and then reselecting the environment adaptation algorithm based on the replaced deviation reference value.

[0225] To further ensure the rationality of the processed image data, it is necessary to perform further separate analysis and calculation on the processed image data, which will be explained in detail through the steps shown below.

[0226] After parsing the image data into processed image data, the following steps are also included:

[0227] S81: Data reception rate, processing frame rate, and algorithm running status.

[0228] Here, data reception rate refers to the amount of raw image data transmitted from an external image acquisition device to a preset FPGA chip per unit time. Processing frame rate refers to the number of image frames processed per second by the preset FPGA chip through the target image processing algorithm. Algorithm running status refers to the real-time working status of the target image processing algorithm on the FPGA chip, including normal operation, abnormal lag, and functional failure.

[0229] The data reception rate is obtained by transmitting real-time statistics on the received data through an interface monitoring module pre-installed on the FPGA chip. The processing frame rate is obtained by statistically analyzing the number of effective image frames processed per second by the FPGA chip. The algorithm's running status is determined by collecting and comprehensively evaluating the FPGA chip's resource usage, data flow, and result validity. The specific determination method is existing technology and will not be elaborated upon here.

[0230] S82: Retrieve the receiving reference rate and processing reference frame rate from the target image processing algorithm.

[0231] The receiving baseline rate refers to the standard value of the raw image data receiving rate that matches the processing capability of the target image processing algorithm. The processing baseline frame rate refers to the theoretical processing frame rate at which the target image processing algorithm achieves optimal performance on a preset FPGA chip. The target image processing algorithm includes both the receiving baseline rate and the processing baseline frame rate.

[0232] The target image processing algorithm retrieves the receiving reference rate and processing reference frame rate for convenient subsequent use.

[0233] S83: Calculate the difference between the data receiving rate and the receiving reference rate, and use it as the rate deviation value.

[0234] Among them, the speed deviation value refers to the deviation value corresponding to the existence of a speed deviation.

[0235] The difference between the data receiving rate and the receiving reference rate is calculated, and the calculation result is used as the rate deviation value for convenient subsequent use.

[0236] S84: Calculate the difference between the processed frame rate and the processed baseline frame rate as the frame rate deviation value.

[0237] Among them, the frame rate deviation value refers to the deviation value corresponding to the frame rate deviation.

[0238] The difference between the processed frame rate and the processed baseline frame rate is calculated, and the calculation result is used as the frame rate deviation value for convenient subsequent use.

[0239] S85: Combines the rate deviation value, frame rate deviation value, and algorithm running status to generate processing display information and sends it to the terminal held by the administrator.

[0240] Here, "processing and displaying information" refers to the information displayed to show the processing status. "Administrator-held terminal" refers to the device used to monitor the system status. The administrator-held terminal can be a host monitoring computer, a mobile management app, or an industrial control panel.

[0241] By comprehensively analyzing the rate deviation value, frame rate deviation value, and algorithm running status, processing display information is generated and sent to the terminal held by the administrator, so that the administrator can understand the processing status in a timely manner.

[0242] To further ensure the rationality of the processing and display of information, it is necessary to perform further separate analysis and calculation on the processing and display of information, which will be explained in detail through the steps shown below.

[0243] The method for generating display information includes the following steps:

[0244] S851: Determine the status display information, rate reference deviation range, and frame rate reference deviation range according to the algorithm's running status.

[0245] The status display information refers to descriptive information generated based on the algorithm's running status, used to intuitively reflect the system's current working condition. The status display information includes elements such as status level, core anomaly characteristics, and risk warnings. The rate reference deviation range refers to the range, dynamically set based on the algorithm's running status, that allows the actual data receiving rate to deviate from the receiving reference rate. The frame rate reference deviation range refers to the range, set based on the algorithm's running status, that allows the actual processed frame rate to deviate from the processed reference frame rate.

[0246] Different algorithm operating states correspond to different status display information, rate reference deviation ranges, and frame rate reference deviation ranges. The more stable the algorithm's operating state, the more stringent the rate reference deviation ranges and frame rate reference deviation ranges; conversely, the more unstable the algorithm's operating state, the more lenient the rate reference deviation ranges and frame rate reference deviation ranges.

[0247] For example, when the algorithm is in a slightly abnormal running state, the status display information can be a set of display elements indicating that resource usage is 85%, data is slightly delayed, and the risk is low.

[0248] S852: Determine the rate anomaly value based on the rate deviation value and the rate reference deviation range.

[0249] Among them, the abnormal value of the rate refers to the deviation value corresponding to the abnormal deviation of the rate.

[0250] The system determines whether the rate deviation value falls within the rate reference deviation range. When it falls within the range, the output is 0, indicating a rate anomaly. When it does not fall within the range, the system calculates the difference between the rate deviation value and the endpoint of the rate reference deviation range, and uses the absolute value of the difference as the rate anomaly for subsequent use.

[0251] S853: Determine abnormal frame rate values ​​based on the frame rate deviation value and the frame rate reference deviation range.

[0252] Among them, frame rate outliers refer to the deviation values ​​corresponding to abnormal deviations in frame rate.

[0253] The frame rate deviation value is determined by whether it falls within the frame rate reference deviation range. When it falls within the range, the output is 0, which is a frame rate anomaly. When it does not fall within the range, the difference between the frame rate deviation value and the end value of the frame rate reference deviation range is calculated, and the absolute value of the difference is taken as the frame rate anomaly for subsequent use.

[0254] S854: Combines rate anomalies with frame rate anomalies to generate synchronization anomaly display information.

[0255] Among them, the synchronization anomaly display information refers to the display information shown when there is a synchronization anomaly in the rate and frame rate.

[0256] By matching the time corresponding to the abnormal rate value with the time corresponding to the abnormal frame rate value, and combining the abnormal rate value and the abnormal frame rate value corresponding to the matching, a synchronization abnormality display information is formed for convenient subsequent use.

[0257] S855: Merge synchronization error display information and status display information and display them as processing information.

[0258] In this process, by merging the synchronization error display information and the status display information, a set of display information is formed to serve as the processing display information, thereby improving the accuracy of the obtained processing display information.

[0259] Based on the same inventive concept, embodiments of the present invention provide an image processing system based on an FPGA chip, comprising:

[0260] The acquisition module is used to acquire image processing requirements, raw image data, data receiving rate, processing frame rate, and algorithm running status.

[0261] The memory stores a program for implementing an image processing method based on an FPGA chip as described above;

[0262] The processor loads and executes programs stored in memory.

[0263] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0264] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An image processing method based on an FPGA chip, characterized in that, include: S1: Obtain image processing requirements; S2: Select the target image processing algorithm according to the image processing requirements; S3: Configure image data transmission parameters according to the target image processing algorithm; S4: The target image processing algorithm and the image data transmission parameters are synchronously sent to the preset FPGA chip to complete the function initialization; S5: Receive raw image data transmitted from a preset external image acquisition device; S6: Send the original image data to a preset FPGA chip for processing, generate processed image data, and forward and output it; The method for selecting the target image processing algorithm includes: S21: Retrieve the processing object and processing location from the image processing requirements; S22: Determine the required environment and initial algorithm according to the processing object; S23: Retrieve the location environment from the processing location; S24: Compare the required environment with the location environment to obtain environmental deviation information; S25: Combining the processing object with the environmental deviation information, select an environment adaptation algorithm from the initial algorithm, and use the environment adaptation algorithm as the target image processing algorithm; The method for selecting the environment adaptation algorithm includes: S251: Retrieve the illumination deviation value and electromagnetic interference deviation value from the environmental deviation information; S252: Determine the unit value of light and the unit value of electromagnetic interference according to the object being processed; S253: Calculate the product of the illumination deviation value and the unit value of the light ray, and use it as the illumination reference value; S254: Calculate the product of the electromagnetic interference deviation value and the electromagnetic interference unit value, and use it as the electromagnetic interference reference value; S255: Determine a comprehensive reference value based on the illumination reference value and the electromagnetic interference reference value; S256: Retrieve the reference interval according to the initial algorithm; S257: Based on the situation where the comprehensive reference value falls within the reference benchmark interval, select the corresponding initial algorithm as the falling-in selection algorithm, and use the falling-in selection algorithm as the environment adaptation algorithm; Following the fall-in selection algorithm as the environment adaptation algorithm, the following is also included: S2571: Retrieve the number of falls based on the fall selection algorithm; S2572: When the number of occurrences exceeds the preset baseline number, the corresponding reference baseline interval is used as the occurrence baseline interval; S2573: Determine the intermediate value, maximum value, and minimum value according to the reference interval; S2574: Calculate the deviation between the comprehensive reference value and the median, maximum, and minimum values ​​to obtain the median deviation value, maximum deviation value, and minimum deviation value; S2575: Generate a deviation reference value by combining the maximum deviation value, the minimum deviation value, and the intermediate deviation value; S2576: Sort the deviation reference values ​​from largest to smallest, and select the first-to-last value corresponding to the fall-in selection algorithm to replace the environment adaptation algorithm.

2. The image processing method based on an FPGA chip according to claim 1, characterized in that, The method for generating the deviation reference value includes: S25751: Compare the maximum deviation value with the minimum deviation value, and take the smaller value as the end selection deviation value; S25752: Calculate the difference between the intermediate deviation value and the selected end deviation value, and use it as the mid-end offset value; S25753: Determine the offset ratio value based on the midpoint offset value and the falling into the reference interval; S25754: Determine the intermediate deviation ratio based on the intermediate deviation value and the falling within the reference range; S25755: The offset ratio value and the intermediate deviation ratio value are weighted and calculated to obtain the deviation reference value.

3. The image processing method based on an FPGA chip according to claim 1, characterized in that, The method for generating the processed image data includes: S61: Retrieve the interface transmission mode, data format parameters, and cache scheduling parameters from the image data transmission parameters; S62: Transmit the original image data to a preset FPGA chip via the interface transmission method; S63: Adapt the original image data to the data format parameters to generate processed format image data; S64: Cache the processed format image data according to the cache scheduling parameters to form cached image data; S65: The cached image data is parsed and processed using the target image processing algorithm to obtain parsed image data, and the parsed image data is used as the processed image data.

4. The image processing method based on an FPGA chip according to claim 3, characterized in that, The process of using the parsed image data as the processed image data further includes: S71: Retrieve the image clarity and processing time from the processed image data; S72: Retrieve the required image clarity and required processing time from the image processing requirements; S73: Calculate the difference between the image sharpness and the required sharpness, as the sharpness deviation value; S74: Calculate the difference between the time consumed by the demand and the time consumed by the processing, and use it as the time deviation value; S75: Generate an image processing adjustment value by combining the sharpness deviation value and the time consumption deviation value; S76: Correct the deviation reference value with the image processing adjustment value, and reselect the environment adaptation algorithm.

5. The image processing method based on an FPGA chip according to claim 3, characterized in that, The process of using the parsed image data as the processed image data further includes: S81: Data reception rate, processing frame rate, and algorithm running status; S82: Retrieve the receiving reference rate and processing reference frame rate from the target image processing algorithm; S83: Calculate the difference between the data receiving rate and the receiving reference rate as the rate deviation value; S84: Calculate the difference between the processed frame rate and the processed baseline frame rate as the frame rate deviation value; S85: Combine the rate deviation value, the frame rate deviation value, and the algorithm running status to generate processing display information and send it to the terminal held by the administrator.

6. The image processing method based on an FPGA chip according to claim 5, characterized in that, The method for generating the processing and display information includes: S851: Determine the status display information, rate reference deviation range, and frame rate reference deviation range according to the algorithm's running status; S852: Determine the rate anomaly value based on the rate deviation value and the rate reference deviation range; S853: Determine the frame rate anomaly value based on the frame rate deviation value and the frame rate reference deviation range; S854: Combine the rate anomaly value and the frame rate anomaly value to generate synchronization anomaly display information; S855: Merge the synchronization error display information and the status display information and use them as processing display information.

7. An image processing system based on an FPGA chip, characterized in that, include: The acquisition module is used to acquire image processing requirements, raw image data, data receiving rate, processing frame rate, and algorithm running status. The memory stores a program for implementing an image processing method based on an FPGA chip as described in any one of claims 1 to 6; The processor loads and executes programs stored in memory.

Images