An FPGA implementation method for sound source identification and visualization image processing

Abstract

This invention discloses an FPGA implementation method for visual image processing of sound source identification, comprising: 1. Normalizing the sound intensity values ​​obtained from sound source identification and converting them into grayscale values; 2. Obtaining corresponding color pixel values ​​by performing three independent mappings on the grayscale values; 3. Upsampling the low-resolution color acoustic cloud map using an interpolation algorithm to improve the image resolution and form a visual acoustic cloud map that represents sound intensity in color. This invention enables the visual image processing of sound source identification to be implemented on an FPGA in the form of digital circuits, thereby improving the real-time performance of the entire sound source identification process, reducing the hardware complexity of the sound source identification platform, and saving time and hardware costs.

Description

Technical Field

[0001] This invention belongs to the field of noise source identification and localization, and more specifically relates to an FPGA implementation method for visual image processing of sound source identification. Background Technology

[0002] Noise pollution is a significant form of environmental pollution, disrupting people's lives in various ways. Unlike other toxic and harmful pollutants, noise pollution does not accumulate in its impact on the environment; once the sound source stops emitting sound, the noise pollution disappears immediately. Therefore, controlling noise at its source is the most direct and effective method for preventing and controlling noise pollution. The prerequisite for controlling noise at its source is accurately identifying the location of the sound source.

[0003] The sound source identification process can be divided into three parts: sound signal acquisition, signal processing, and recognition result display. The result display involves converting the sound intensity values ​​calculated by the signal processing algorithm into pixel values ​​and performing image processing to obtain a visual acoustic cloud map that represents sound intensity using color. The acoustic cloud map can transform auditory information into more sensitive visual information, intuitively displaying the location and intensity of the sound source through the warmth and intensity of colors.

[0004] Currently, most sound source recognition systems developed based on embedded platforms use hardware such as FPGAs for sound signal acquisition and processing, and software such as PCs or ARM processors for displaying the sound source recognition results. These solutions place the sound source recognition result display process on an additional soft-core processor platform, which not only complicates the system hardware structure but also compromises the system's portability and real-time performance, while increasing the system cost. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by proposing an FPGA implementation method for visual image processing of sound source identification. The aim is to enable the visual image processing of sound source identification to be implemented on an FPGA platform, thereby improving the real-time performance of the entire sound source identification process, reducing the hardware complexity of the sound source identification platform, and saving time and hardware costs.

[0006] The present invention adopts the following technical solution to solve the technical problem:

[0007] The FPGA implementation method for sound source identification and visualization image processing of this invention is characterized by the following steps:

[0008] Step 1: Set a virtual grid surface on the surface of the object to be measured, and divide the virtual grid surface evenly into N grid points. Use a microphone array to collect the sound pressure signal of the object to be measured, and use a sound source identification algorithm to identify the sound source of the sound pressure signal, and obtain the sound intensity value at each grid point, denoted as {d}. i |i=1,2,...,N};where, d i Let W represent the sound intensity value at the i-th grid point; let W be the data bit width of the sound intensity value at each grid point; N represents the number of grid points, and N = N r ·N c N r N represents the number of rows in the virtual mesh. c The number of columns for the virtual grid surface;

[0009] Step 2: Configure a RAM memory with a storage depth of N and a data bit width of W in the FPGA, and set {d i After |i=1,2,...,N} is converted into binary two's complement, it is stored sequentially into the RAM memory;

[0010] Step 3: The FPGA calculates the most significant bit m of the maximum value among the N sound intensity values, and m ≤ (W-1);

[0011] Step 4: The FPGA sequentially reads the i-th sound intensity value d from the RAM memory. i And extract d i The most significant bit m and the following 7 bits are used to obtain the grayscale value g with a data bit width of 8. i This converts N sound intensity values ​​into grayscale values.

[0012] Step 5: The FPGA uses equations (1)-(3) to process the i-th grayscale value g. i Perform three independent pseudo-color mappings to obtain the corresponding R channel color component values ​​R. i G channel color component value G i B channel color component value B i :

[0013]

[0014]

[0015]

[0016] In equations (1)-(3), R i G i B i The data bit width is 8, and i = 1, 2, ..., N;

[0017] Step 6: The FPGA will R i Extract the high 5 bits and use G i Extract the high 6 bits, and then extract B. i After extracting the high 5 bits, the pixels are concatenated in RGB order to obtain the pixel value P of a color acoustic cloud with a bit depth of 16. i ;

[0018] Or R i G i B i By directly concatenating the bits in RGB order, the pixel values ​​Q of the color acoustic cloud map with a bit depth of 24 are obtained. i ;

[0019] Step 7: The FPGA will convert the pixel values ​​P of the color acoustic cloud map i Or Q i The pixel values ​​{P} of the N color acoustic cloud maps stored in the RAM are sequentially stored into another RAM memory with a storage depth of N. i |i=1,2,...,N} or {Q i |i=1,2,...,N} constitutes a frame with resolution N. r ·N c Low-resolution color acoustic cloud map;

[0020] Step 8: Upsample the low-resolution color acoustic cloud image using nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation, thereby upsampling the N resolution color acoustic cloud image. r ·N c The image is magnified k times to obtain a resolution of kN. r ·kN c High-resolution color acoustic cloud image.

[0021] The FPGA implementation method for sound source identification and visualization image processing described in this invention is also characterized in that step 3 is performed according to the following process:

[0022] Step 3.1: Set up a register with a data width of W, store a value D in the register, and initialize D to a binary number of all zeros;

[0023] Step 3.2: After each input sound intensity value, perform a logical OR operation with D to obtain the updated value D = d1||d2||...||d N ;

[0024] Step 3.3: Set up a subtraction counter and initialize the count value to W-1;

[0025] Step 3.4: Perform a cyclic left shift operation on the value D stored in the register. Each time the register is shifted left, check the most significant bit of the value stored in the register. If the value of the most significant bit is 0, the counter value is decremented by 1. If the value of the most significant bit is 1, stop counting and use the current counter value as the most significant bit m of the maximum value among the N sound intensity values.

[0026] In step 5, the pseudo-color mapping from grayscale values ​​to R, G, and B color component values ​​is achieved through the following process:

[0027] Step 5.1: Set up three ROM memories with a data bit width of 8 and a storage depth of 256;

[0028] Step 5.2: Calculate the 256 sets of R, G, and B color component values ​​corresponding to gray values ​​from 0 to 255 according to equations (1) to (3);

[0029] Step 5.3: Store the 256 groups of R, G, and B color component values ​​into the three ROM memories in order of grayscale values ​​from 0 to 255;

[0030] Step 5.4: Using grayscale value g i As addresses, read R from the three ROM memories respectively. i B i G i This enables pseudo-color mapping from grayscale values ​​to R, G, and B color component values.

[0031] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program that supports the processor in executing the FPGA implementation method, and the processor is configured to execute the program stored in the memory.

[0032] The present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when run by a processor, executes the steps of the FPGA implementation method.

[0033] Compared with the prior art, the beneficial effects of the present invention are reflected in:

[0034] 1. This invention fully utilizes the logic and storage resources of the FPGA platform, taking advantage of its parallel execution characteristics. It replaces complex arithmetic operations such as multiplication and division, which are not suitable for FPGA implementation, with logical operations, shifting, and lookup tables. This solves the problem that sound source recognition visualization image processing is not suitable for implementation on the FPGA platform. Compared with traditional implementation methods, it significantly reduces the complexity and resource consumption of sound source recognition visualization image processing on the FPGA platform, saves hardware resources, and has a simple implementation method with practical application value.

[0035] 2. This invention proposes a novel segmented pseudo-color mapping method tailored to the characteristics of acoustic cloud images. This method is computationally simple, produces good visual results, maintains a clear and clean background, highlights the location of the sound source in the image, and facilitates the subsequent overlay of optical images.

[0036] 3. The method of the present invention makes full use of the storage resources of the hardware circuit, and uses a lookup table method to replace on-site calculation. The result of pseudo-color mapping is pre-calculated and stored in ROM memory, saving the resource-consuming multiplication and division operations and simplifying the complexity of FPGA implementation. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the sound source identification layout of the method of the present invention;

[0038] Figure 2 This is a schematic diagram illustrating the effect of low-resolution acoustic cloud maps using the method of the present invention.

[0039] Figure 3 This is a schematic diagram illustrating the effect of visual image processing for sound source identification using the method of the present invention. Detailed Implementation

[0040] In this embodiment, an FPGA implementation method for sound source identification and visualization image processing is performed according to the following steps:

[0041] Step 1: As Figure 1 As shown, the object to be measured contains a sound source that excites and radiates a sound field, and the plane where the sound source is located is the sound source plane.

[0042] In the radiated sound field of the surface of the object to be measured, a virtual grid surface parallel to the sound source surface is set up. The virtual grid surface is uniformly divided into N grid points. In specific implementation, the size of the grid surface is the spatial range in which the acoustic cloud map can display sound intensity information. The number of grid points is related to the accuracy of sound source identification.

[0043] A sound array is constructed using a set of microphones. The plane in which the microphone array is located should be parallel to the virtual grid surface and the sound source surface. The sound pressure signal in the radiated sound field of the object to be measured is collected in parallel using the microphone array.

[0044] In practice, a beamforming algorithm is used to identify the sound source of the sound pressure signal. A beamforming calculation is performed for each grid point, and the sound intensity value at each grid point is denoted as {d}. i |i=1,2,...,N};where, d i Let W represent the sound intensity value at the i-th grid point; let W be the data bit width of the sound intensity value at each grid point; N represents the number of grid points, and N = N r ·N c N rN represents the number of rows in the virtual mesh. c This represents the number of columns in the virtual grid.

[0045] Step 2: Set up a RAM memory with a storage depth of N and a data bit width of W in the FPGA. For each result calculated by the beamforming algorithm in Step 1, the write enable signal of the RAM memory is pulled high for one clock cycle, the write address is incremented by one from zero, and the calculated data is written. This process is repeated N times to obtain {d}. i After |i=1,2,...,N} is converted into binary two's complement, it is stored sequentially into RAM memory.

[0046] Step 3: The FPGA calculates the most significant bit m of the maximum value among N sound intensity values, where m ≤ (W-1). The sound intensity values ​​in the FPGA are represented as positive integers in two's complement binary representation, with invalid high-order bits all being zero. The most significant bit is the position of the first non-zero number to the right of the most significant bit (W-1) in each value. In specific implementation, m is calculated as follows:

[0047] Step 3.1: Set up a register with a data width of W in the FPGA and store a value D in the register. When the FPGA is powered on or reset, D will be initialized to a binary number with all zeros.

[0048] Step 3.2: Synchronized with the write enable signal in Step 2, after each input sound intensity value, perform a logical OR operation between that value and D. While Step 2 stores all N sound intensity values, Step 3.2 can obtain the updated value D = d1||d2||...||d N ;

[0049] Step 3.3: Set up a subtraction counter, and initialize the counter value to W-1 when the FPGA is powered on or reset;

[0050] Step 3.4: After completing step 3.2, perform a cyclic left shift operation on the value D stored in the register, shifting left once every clock cycle, and simultaneously checking the most significant bit of the value stored in the register; if the value of the most significant bit is 0, the counter value is decremented by 1; if the value of the most significant bit is 1, stop counting, and use the current counter value as the most significant bit m of the maximum value among N sound intensity values; set a calculation flag signal flag in the FPGA, which is low after power-on or reset, and pulls the signal high after calculating m.

[0051] Step 4: After the flag signal goes high, the read enable signal of the RAM memory is continuously pulled high for N clock cycles. The read address starts from zero and increases by one each clock cycle. Each clock cycle, one sound intensity value can be read. The FPGA then reads the i-th sound intensity value d from the RAM memory sequentially. i ;

[0052] While each data point is read, a continuous assignment statement is used to extract d. i The most significant bit m and the following 7 bits are used to obtain the grayscale value g with a data bit width of 8. i An 8-bit grayscale value can represent 256 grayscale levels from 0 to 255, thus converting N sound intensity values ​​into grayscale values ​​that represent 256 grayscale levels.

[0053] Step 5: The FPGA uses equations (1)-(3) to convert the i-th grayscale value g. i Perform three independent pseudo-color mappings to obtain the corresponding R channel color component values ​​R. i G channel color component value G i B channel color component value B i :

[0054]

[0055]

[0056]

[0057] In equations (1)-(3), R i G i B i The data bit width is 8, and i = 1, 2, ..., N;

[0058] In practice, the pseudo-color mapping from grayscale values ​​to R, G, and B color component values ​​is achieved through the following process:

[0059] Step 5.1: Configure three ROM memories in the FPGA with a data bit width of 8 and a storage depth of 256;

[0060] Step 5.2: Use MATLAB to pre-calculate the 256 sets of R, G, and B color component values ​​corresponding to gray values ​​from 0 to 255 according to equations (1) to (3);

[0061] Step 5.3: Write the results calculated by MATLAB into the ROM module through the initialization statement, so that when the FPGA is powered on, it can store 256 groups of R, G, and B color component values ​​in the order of grayscale values ​​from 0 to 255 into the three ROM memories as ROM storage values, and once the storage values ​​are written, they will not change during the operation of the FPGA.

[0062] Step 5.4: Step 4 yields the grayscale value g. i At the same time, with g i As addresses, read R from the three ROM memories respectively. i B i G i This enables pseudo-color mapping from grayscale values ​​to R, G, and B color component values.

[0063] Step 6: Simultaneously with data reading in step 5.4, the FPGA uses a continuous assignment statement to assign values ​​to R. i Extract the high 5 bits and use G i Extract the high 6 bits, and then extract B. i After extracting the high 5 bits, the pixels are concatenated in RGB order to obtain the pixel value P of a color acoustic cloud with a bit depth of 16. i ;

[0064] Or R i G i B i By directly concatenating the bits in RGB order, the pixel values ​​Q of the color acoustic cloud map with a bit depth of 24 are obtained. i .

[0065] Step 7: The FPGA converts the pixel values ​​P of the color acoustic cloud obtained in Step 6. i Or Q i The pixel values ​​{P} of the N color acoustic cloud maps stored in the RAM are sequentially stored into another RAM memory with a storage depth of N. i |i=1,2,...,N} or {Q i |i=1,2,...,N} constitutes a frame as follows Figure 2 The resolution shown is N r ·N c Low-resolution color acoustic cloud map.

[0066] Step 8: Upsample the low-resolution color acoustic cloud image using nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation, thereby upsampling the N resolution color acoustic cloud image. r ·N c The image is magnified k times to obtain the following: Figure 3 The resolution shown is kN r ·kNc High-resolution color acoustic cloud image.

[0067] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor in executing the FPGA implementation method described above, and the processor is configured to execute the program stored in the memory.

[0068] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above-described FPGA implementation method.

Claims

1. A method for FPGA implementation of a sound source identification visualized image processing, characterized in that, The following steps are taken: Step 1: a virtual grid surface is arranged on the surface of an object to be measured, and the virtual grid surface is evenly divided into N grid points; sound pressure signals of the object to be measured are collected by using a microphone array, and a sound source identification algorithm is used to identify the sound pressure signals to obtain sound intensity values at each grid point, denoted as {d i |i=1,2,...,N};wherein d i represents the sound intensity value at the i-th grid point; the data bit width of the sound intensity value at each grid point is W; N represents the number of grid points, and N=N r ·N c , N r is the number of rows of the virtual grid surface, and N c is the number of columns of the virtual grid surface; Step 2: Set a RAM memory with a storage depth of N and a data bit width of W in the FPGA, and sequentially store {d i i = 1, 2,..., N} converted to binary complement in the RAM memory. Step 3: The FPGA calculates the most significant bit m of the maximum value of the N sound intensity values, and m≤(W-1); Step 4: the FPGA reads out the i-th sound intensity value d from the RAM memory in sequence i and intercepts the most significant bit m and the following 7 bits of d i , thus obtaining the gray value g with a data bit width of 8 i , thus converting N sound intensity values into gray values; Step 5: The FPGA uses the formula (1) - (3) to calculate the i-th gray value g i Three independent pseudo-color mappings are performed respectively to obtain the corresponding R channel color component value R i , G channel color component value G i , and B channel color component value B i : In formula (1) - formula (3), R i , G i , B i The data bit width is 8, i = 1, 2,..., N; Step 6: the FPGA will R i Take high 5 bits, and G i Take high 6 bits, and B i After taking high 5 bits, perform bit splicing in the order of RGB to obtain the pixel value P of the color acoustic cloud image with a bit depth of 16 i ; Or R i , G i , B i are directly bit-spliced in the order of RGB to obtain the pixel value Q i of the color acoustic cloud image with a bit depth of 24. Step 7: The FPGA stores the pixel values P i or Q i in another RAM memory with depth N in order to form a frame of a low resolution colorized acoustic cloud with resolution N i · N c by the N pixel values {P i |i = 1, 2,..., N} or {Q r |i = 1, 2,..., N} stored in the RAM memory. Step 8: upsample the low resolution color acoustic cloud map using nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation to increase the resolution of the image by a factor of k, resulting in a high resolution color acoustic cloud map with resolution kN r • N c • kN r • kN c • kN 2. The FPGA implementation method of sound source identification visualized image processing according to claim 1, wherein, The step 3 is taken by the following process: Step 3.1: Set a register with a data bit width of W, and store a value D in the register, and initialize D as a binary number of all zeros; Step 3.2: After inputting each sound intensity value, do logical OR operation with D to get updated value D = dl || d2 ||... || dn N ; Step 3.3: Set a subtraction counter, and let the initial value of the counter be W-1; Step 3.4: Perform a cyclic left shift operation on the value D stored in the register, and check the most significant bit of the register value every time the left shift is performed; if the value of the most significant bit is 0, then the counter value is reduced by 1; if the value of the most significant bit is 1, then stop counting, and take the current counter value as the most significant bit m of the maximum value of the N sound intensity values.

3. The FPGA implementation method of sound source identification visualized image processing according to claim 1, wherein, The step 5 is taken by the following process to realize the pseudo-color mapping of the gray value to the R, G, and B color component values: Step 5.1: Set three blocks of ROM memories with a data bit width of 8 and a storage depth of 256; Step 5.2: Calculate 256 sets of R, G, and B color component values corresponding to the gray values from 0 to 255 according to the formulas (1)-(3); Step 5.3: Store the 256 sets of R, G, and B color component values in the three blocks of ROM memories in the order of the gray values from 0 to 255; Step 5.4: in gray value g i R is read out from the three ROM memories as addresses i B i G i , so as to realize the pseudo-color mapping of the gray value to the R, G, B color component values.

4. An electronic device comprising a memory and a processor, characterized in that The memory is used to store a program supporting the processor to execute the FPGA implementation method of any one of claims 1-3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to perform the steps of the FPGA implementation method of any one of claims 1-3.

Images