Method and system for processing video images acquired by multiple lenses

Abstract

The invention discloses a multi-lens acquisition video image processing method and system, and relates to the technical field of image processing, and the system comprises an image acquisition module, an image adjustment module, an audio acquisition module, an audio adjustment module, and a video detection module. According to the method and the system for processing the video images acquired by the multiple lenses, through an image acquisition module and an image adjustment module, when the video images acquired by the multiple lenses are identified, the range of the view field angle shot by the corresponding lens is identified, and the pictures acquired by the multiple lenses are sorted through the view field angle; the integrity of multiple pictures can be guaranteed, ghosting during synthesis of multiple images can be avoided through feature segmentation when the pictures are spliced, and sound effect positions of audios in the video pictures are adjusted by setting an audio acquisition module and an audio adjustment module and adjusting audio waveforms. And the audio sound effect position is matched with the video picture when the video is played.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for processing video images acquired by multiple lenses. Background Art

[0002] Multi-lens refers to a camera device with multiple cameras. Multi-image fusion technology refers to the natural fusion of the overlapping areas captured by two cameras. Multi-lens multi-image stitching technology is the technical prerequisite for manufacturing multi-lens cameras. Multi-lens cameras are camera devices with wide-angle, low distortion and panoramic photos and videos.

[0003] Multi-lens cameras can provide complete environmental information and three-dimensional sense, allowing viewers to have an immersive experience. Multi-lens cameras have a wide range of applications, such as virtual reality, smart office, smart home, tourism, security and other fields.

[0004] At present, due to the shooting angle problem of the multi-lens camera, the multiple pictures taken cannot be perfectly integrated during use, resulting in ghosting when the multiple images are integrated. At the same time, the recording effect of the camera cannot be considered during the multi-image fusion. This will cause the audio sound position of the video captured by the multi-lens camera to not match the video screen when it is played. Summary of the invention

[0005] The purpose of the present invention is to provide a video image processing method and system for multi-lens acquisition to solve the above-mentioned deficiencies in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a video image processing method acquired by multiple lenses, comprising the following steps: S1, collecting pictures of different video images captured by multiple lenses at the same time, and sorting the multiple different pictures; S2, based on the sorting result in step S1, stitching two adjacent pictures; S3, continuously processing the video images captured by adjacent lenses based on the splicing scheme of the two adjacent screen images in step S2; S4, collecting the picture audio collected when multiple lenses capture video images, and performing audio processing on the picture audio, wherein the audio processing includes audio denoising, audio keyword recognition, and audio volume adjustment; S5, performing picture and audio matching on the video image processed in step S3 and the multiple pictures and sounds pre-processed in step S4 based on a data matching algorithm, synthesizing the pictures and audios with matching results, and performing audio adjustment on the pictures and audios without matching results; S6, performing audio adjustment on the picture and sound for which no matching result exists in step S5 based on the audio waveform adjustment algorithm, and synthesizing the adjusted audio with the video image, wherein the specific standard of the audio waveform adjustment is to adjust the sound effect position of the audio in the video picture by adjusting the audio waveform; S7, based on the video detection algorithm, simultaneously detects the lens image and the audio playback effect to ensure that the audio sound effect position matches the video image when the video is played.

[0007] Furthermore, the step S1 of sorting the multiple different video images specifically includes the following steps: A1, taking panoramic images by a panoramic camera; A2, dividing the panoramic video captured by the panoramic camera according to the horizontal relative angle and the vertical relative angle of the panoramic camera by a panoramic video angle division module, to obtain the field of view angle corresponding to each position of the panoramic video; A3, using a picture recognition algorithm to respectively identify the surrounding image information of different pictures collected by multiple lenses; A4, performing image matching on the surrounding image information of the picture identified in step A3 and the panoramic image captured in step A1 based on an image matching algorithm, and numbering the matched picture according to the field of view angle obtained in step A2; A5, in step A4, the pictures with different horizontal viewing angles are numbered from left to right in the first level, and the pictures with the same horizontal viewing angle but different vertical viewing angles are numbered from top to bottom in the second level.

[0008] Furthermore, the stitching of two adjacent pictures in step S2 specifically includes the following steps: B1, performing adjacent end feature segmentation based on the two pictures with adjacent primary labels in step A5 and the two pictures with the same primary label but adjacent secondary labels in step A5; B2, performing cross feature optimization on two adjacent pictures after feature segmentation based on a graph optimization algorithm, wherein the cross feature is a shot picture in which both adjacent pictures exist; B3, based on the image synthesis algorithm, synthesize the picture in which the adjacent end features are segmented and there is no cross feature in step B1 and the cross picture optimized in step B2.

[0009] Furthermore, the feature segmentation in step B1 specifically includes the following steps: C1, in order to obtain the detailed features of different spatial dimensions of the picture to improve the accuracy of feature recognition, the feature map is split along the vertical and horizontal dimensions, and the two independent feature maps output by the backbone network are named and , namely the longitudinal branch and the transverse branch, and then divide them into several sub-areas in the longitudinal and transverse directions, which are specifically expressed as shown in the following formula: , Where N represents the number of samples in the data set; , Respectively represent the block feature maps of the i-th image in the vertical and horizontal directions; and Respectively represent the number of divisions in the vertical and horizontal directions; C2, assuming that the feature is C, adopts a multi-granularity segmentation method, and the specific expression is shown in the following formula: , Where N represents the number of samples in the feature dataset, Represents the overall characteristics, Represents the features of the i-th segmented image; C3, cross-feature optimization is performed on the pictures with the same features as the i-th segmented image in step C2, and the pictures with different features as the i-th segmented image in step C2 are stored for subsequent image synthesis.

[0010] Furthermore, the audio processing of the picture audio in step S4 specifically includes the following steps: D1, adjust the audio volume of the collected picture and audio to ensure that the processed picture and audio are of uniform volume; D2, performing audio denoising based on the picture audio whose volume has been adjusted in step D1, specifically comprising the following steps: D21, for the noisy frequency signal, a section containing only the noise signal is reserved; D22, after the noisy frequency signal is preliminarily denoised by the improved multi-window spectrum subtraction method, it is also necessary to adjust the noise signal collected according to the spectrum subtraction and gain parameters transmitted in the first stage of denoising, such as reducing the signal amplitude; D23, then the adjusted noise signal is used as the actual reference signal and input into the filter together with the audio signal after the first stage of denoising, so as to complete the overall denoising process and obtain a relatively pure audio signal; D24, finally, the audio signal after denoising can be intercepted and processed according to the time interval size of the interval containing only noise signals reserved in the previous stage to obtain a pure audio signal; D3, extracting audio keywords from the denoised pure audio signal, specifically including the following steps: D31, feature extraction is to extract the speech features of the collected audio signal. This step can effectively enhance the key features of the audio signal and reduce the computational dimension of keyword recognition; D32, the neural network part is to classify the key information obtained after feature extraction, so as to obtain the result after keyword recognition in the audio signal; D4, performing subsequent processing based on the noise-free frequency keywords generated in steps D3 and D2, wherein the subsequent processing includes synthesizing the audio and video images and adjusting the audio based on an audio waveform adjustment algorithm.

[0011] A video image processing system for multi-lens acquisition includes an image acquisition module, an image adjustment module, an audio acquisition module, an audio adjustment module, and a video detection module: The image acquisition module is used to collect different video images acquired by multiple lenses, and according to the time when the multiple lenses acquired the images, the different video images acquired by the multiple lenses at the same time are uniformly processed, and the screen images are sorted; The image adjustment module is used to receive different video images collected by multiple lenses at the same time and sent by the image acquisition module, and to splice two adjacent pictures; The audio acquisition module is used to collect the audio of the picture collected when multiple lenses capture video images, and perform audio processing on the audio of the picture, wherein the audio processing includes audio denoising, audio keyword recognition and audio volume adjustment; The audio adjustment module is used to match the processed video image with the processed multiple pictures and sounds by using a data matching algorithm, and synthesize the pictures and audios for which matching results exist; for pictures and sounds for which matching results do not exist, perform audio adjustment on the sounds based on an audio waveform adjustment algorithm, and synthesize the adjusted audio with the video image; The video detection module simultaneously detects the lens image and the audio playback effect based on the video detection algorithm.

[0012] Compared with the prior art, the present invention provides a method and system for processing video images acquired by multiple lenses. When identifying video images acquired by multiple lenses, the image acquisition module and the image adjustment module are used to identify the range of the field of view angles shot by the corresponding lenses, and the pictures acquired by multiple lenses are sorted according to the field of view angles, thereby ensuring the integrity of the multiple pictures. At the same time, feature segmentation can be used to avoid ghosting when multiple images are synthesized during picture splicing. At the same time, the audio acquisition module and the audio adjustment module are set to adjust the sound effect position of the audio in the video picture by adjusting the audio waveform, thereby ensuring that the audio sound effect position matches the video picture during video playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0014] Figure 1 A system structure block diagram provided for an embodiment of the present invention; Figure 2 A flowchart of a method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0016] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0017] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0018] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0019] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0020] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0021] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0022] The embodiments described herein may be described with reference to plan views and / or cross-sectional views by means of idealized schematic diagrams of the present disclosure. Therefore, the example illustrations may be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to the embodiments shown in the accompanying drawings, but include modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the accompanying drawings have schematic properties, and the shapes of the regions shown in the figures illustrate the specific shapes of the regions of the elements, but are not intended to be limiting.

[0023] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0024] See also Figure 1-2 , a video image processing method and system for multi-lens acquisition, comprising the following steps: S1, collecting pictures of different video images captured by multiple lenses at the same time, and sorting the multiple different pictures; S2, based on the sorting result in step S1, stitching two adjacent pictures; S3, continuously processing the video images captured by adjacent lenses based on the splicing scheme of the two adjacent screen images in step S2; S4, collecting the audio of the pictures collected when multiple lenses capture video images, and performing audio processing on the audio of the pictures, the audio processing including audio denoising, audio keyword recognition, and audio volume adjustment; S5, performing picture and audio matching on the video image processed in step S3 and the multiple pictures and sounds pre-processed in step S4 based on a data matching algorithm, synthesizing the pictures and audios with matching results, and performing audio adjustment on the pictures and audios without matching results; S6, performing audio adjustment on the picture and sound for which no matching result exists in step S5 based on the audio waveform adjustment algorithm, and synthesizing the adjusted audio with the video image, wherein the specific standard of the audio waveform adjustment is to adjust the sound effect position of the audio in the video picture by adjusting the audio waveform; S7, based on the video detection algorithm, simultaneously detects the lens image and the audio playback effect to ensure that the audio sound effect position matches the video image when the video is played.

[0025] By setting up the image acquisition module and the image adjustment module in this way, when identifying video images captured by multiple lenses, the range of the field of view angles shot by the corresponding lenses is identified, and the pictures captured by multiple lenses are sorted according to the field of view angles, thereby ensuring the integrity of the multiple pictures. At the same time, feature segmentation can be used to avoid ghosting when multiple images are synthesized during picture stitching. At the same time, the audio acquisition module and the audio adjustment module are set to adjust the sound effect position of the audio in the video screen by adjusting the audio waveform, thereby ensuring that the audio sound effect position matches the video screen during video playback.

[0026] Sorting the multiple different video images in step S1 specifically includes the following steps: A1, taking panoramic images by a panoramic camera; A2, the panoramic video captured by the panoramic camera is divided according to the horizontal relative angle and the vertical relative angle with the panoramic camera by the panoramic video angle division module, and the field of view angle corresponding to each position of the panoramic video is obtained, and the field of view angle is 0 degrees to 360 degrees (not included), which is used to mark the angle between each position of the panoramic video and the panoramic camera module. The angle between each position of the panoramic video and the panoramic camera module can be expressed by setting the positive shooting direction of the panoramic camera module and calculating the angle between each position of the panoramic video and the positive shooting direction. The angle includes the angle in the horizontal direction and the angle in the vertical direction. A3, using a picture recognition algorithm to respectively identify the surrounding image information of different pictures collected by multiple lenses; A4, performing image matching on the surrounding image information of the picture identified in step A3 and the panoramic image captured in step A1 based on an image matching algorithm, and numbering the matched picture according to the field of view angle obtained in step A2; A5, in step A4, the screen images with different horizontal viewing angles are numbered from left to right in the first level, and the screen images with the same horizontal viewing angle but different vertical viewing angles are numbered from top to bottom in the second level.

[0027] The step S2 of stitching two adjacent pictures comprises the following steps: B1, performing adjacent end feature segmentation based on the two pictures with adjacent primary labels in step A5 and the two pictures with the same primary label but adjacent secondary labels in step A5; B2, based on the graph optimization algorithm, cross feature optimization is performed on the two adjacent pictures after feature segmentation, and the cross feature is the shooting picture that exists in the two adjacent pictures; B3, based on the image synthesis algorithm, synthesize the picture in which the adjacent end features are segmented and there is no cross feature in step B1 and the cross picture optimized in step B2.

[0028] The feature segmentation in step B1 specifically includes the following steps: C1, in order to obtain the detailed features of different spatial dimensions of the picture to improve the accuracy of feature recognition, the feature map is split along the vertical and horizontal dimensions, and the two independent feature maps output by the backbone network are named and , namely the longitudinal branch and the transverse branch, and then divide them into several sub-areas in the longitudinal and transverse directions, which are specifically expressed as follows: , Where N represents the number of samples in the data set; , Respectively represent the block feature maps of the i-th image in the vertical and horizontal directions; and Respectively represent the number of divisions in the vertical and horizontal directions; C2, assuming that the feature is C, adopts a multi-granularity segmentation method, and the specific expression is shown in the following formula: , Where N represents the number of samples in the feature dataset, Represents the overall characteristics, Represents the features of the i-th segmented image; C3, cross-feature optimization is performed on the pictures with the same features as the i-th segmented image in step C2, and the pictures with different features as the i-th segmented image in step C2 are stored for subsequent image synthesis.

[0029] The audio processing of the picture audio in step S4 specifically includes the following steps: D1, adjust the audio volume of the collected picture and audio to ensure that the processed picture and audio are of uniform volume; D2, performing audio denoising based on the picture audio whose volume has been adjusted in step D1, specifically comprising the following steps: D21, for the noisy frequency signal, a section containing only the noise signal is reserved; D22, after the noisy frequency signal is preliminarily denoised by the improved multi-window spectrum subtraction method, it is also necessary to adjust the noise signal collected according to the spectrum subtraction and gain parameters transmitted in the first stage of denoising, such as reducing the signal amplitude; D23, then the adjusted noise signal is used as the actual reference signal and input into the filter together with the audio signal after the first stage of denoising, so as to complete the overall denoising process and obtain a relatively pure audio signal; D24, finally, the audio signal after denoising can be intercepted and processed according to the time interval size of the interval containing only noise signals reserved in the previous stage to obtain a pure audio signal; D3, extracting audio keywords from the denoised pure audio signal, specifically including the following steps: D31, feature extraction is to extract the speech features of the collected audio signal. This step can effectively enhance the key features of the audio signal and reduce the computational dimension of keyword recognition; D32, the neural network part is to classify the key information obtained after feature extraction, so as to obtain the result after keyword recognition in the audio signal; D4, performing subsequent processing based on the noise-free frequency keywords generated in steps D3 and D2, the subsequent processing including synthesizing the audio and video images and adjusting the audio based on an audio waveform adjustment algorithm.

[0030] A video image processing system for multi-lens acquisition includes an image acquisition module, an image adjustment module, an audio acquisition module, an audio adjustment module, and a video detection module: The image acquisition module is used to collect different video images collected by multiple lenses, and according to the time when the multiple lenses collected the images, the different video images collected by multiple lenses at the same time are uniformly processed and the pictures are sorted; The image adjustment module is used to receive different video images collected by multiple lenses at the same time and sent by the image acquisition module, and to stitch two adjacent pictures; The audio acquisition module is used to collect the audio of the picture collected when multiple lenses capture video images, and perform audio processing on the picture audio. The audio processing includes audio denoising, audio keyword recognition and audio volume adjustment. The audio adjustment module is used for performing picture and audio matching between the processed video image and the processed multiple pictures and sounds by using a data matching algorithm, and synthesizing the pictures and audios for which matching results exist; for pictures and sounds for which matching results do not exist, performing audio adjustment on the sounds based on an audio waveform adjustment algorithm, and synthesizing the adjusted audio with the video image; The video detection module detects the lens image and audio playback effects simultaneously based on the video detection algorithm.

[0031] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A video image processing method for multi-lens acquisition, characterized in that: The following steps are involved: S1, collecting pictures of different video images captured by multiple lenses at the same time, and sorting the multiple different pictures; S2, based on the sorting result in step S1, stitching two adjacent pictures; S3, continuously processing the video images captured by adjacent lenses based on the splicing scheme of the two adjacent screen images in step S2; S4, collecting the picture audio collected when multiple lenses capture video images, and performing audio processing on the picture audio, wherein the audio processing includes audio denoising, audio keyword recognition, and audio volume adjustment; S5, performing picture and audio matching on the video image processed in step S3 and the multiple pictures and sounds pre-processed in step S4 based on a data matching algorithm, synthesizing the pictures and audios with matching results, and performing audio adjustment on the pictures and audios without matching results; S6, performing audio adjustment on the picture and sound for which no matching result exists in step S5 based on the audio waveform adjustment algorithm, and synthesizing the adjusted audio with the video image, wherein the specific standard of the audio waveform adjustment is to adjust the sound effect position of the audio in the video picture by adjusting the audio waveform; S7, based on the video detection algorithm, simultaneously detects the lens image and the audio playback effect to ensure that the audio sound effect position matches the video image when the video is played.

2. The method for processing video images acquired by multiple lenses according to claim 1, characterized in that: The step S1 of sorting the multiple different video images specifically includes the following steps: A1, taking panoramic images by a panoramic camera; A2, dividing the panoramic video captured by the panoramic camera according to the horizontal relative angle and the vertical relative angle of the panoramic camera by a panoramic video angle division module, to obtain the field of view angle corresponding to each position of the panoramic video; A3, using a picture recognition algorithm to respectively identify the surrounding image information of different pictures collected by multiple lenses; A4, performing image matching on the surrounding image information of the picture identified in step A3 and the panoramic image captured in step A1 based on an image matching algorithm, and numbering the matched picture according to the field of view angle obtained in step A2; A5, in step A4, the pictures with different horizontal viewing angles are numbered from left to right in the first level, and the pictures with the same horizontal viewing angle but different vertical viewing angles are numbered from top to bottom in the second level.

3. The video image processing method for multi-lens acquisition according to claim 2, characterized in that: The step S2 of stitching two adjacent pictures specifically includes the following steps: B1, performing adjacent end feature segmentation based on the two pictures with adjacent primary labels in step A5 and the two pictures with the same primary label but adjacent secondary labels in step A5; B2, performing cross feature optimization on two adjacent pictures after feature segmentation based on a graph optimization algorithm, wherein the cross feature is a shot picture in which both adjacent pictures exist; B3, based on the image synthesis algorithm, synthesize the picture in which the adjacent end features are segmented and there is no cross feature in step B1 and the cross picture optimized in step B2.

4. The method for processing video images acquired by multiple lenses according to claim 3, characterized in that: The feature segmentation in step B1 specifically includes the following steps: C1, in order to obtain the detailed features of different spatial dimensions of the picture to improve the accuracy of feature recognition, the feature map is split along the vertical and horizontal dimensions, and the two independent feature maps output by the backbone network are named and , namely the longitudinal branch and the transverse branch, and then divide them into several sub-areas in the longitudinal and transverse directions, which are specifically expressed as shown in the following formula: ， Where N represents the number of samples in the data set; , Respectively represent the block feature maps of the i-th image in the vertical and horizontal directions; and Respectively represent the number of divisions in the vertical and horizontal directions; C2, assuming that the feature is C, adopts a multi-granularity segmentation method, and the specific expression is shown in the following formula: ， Where N represents the number of samples in the feature dataset, Represents the overall characteristics, Represents the features of the i-th segmented image; C3, cross-feature optimization is performed on the pictures with the same features as the i-th segmented image in step C2, and the pictures with different features as the i-th segmented image in step C2 are stored for subsequent image synthesis.

5. The method for processing video images acquired by multiple lenses according to claim 4, characterized in that: The audio processing of the picture audio in step S4 specifically includes the following steps: D1, adjust the audio volume of the collected picture and audio to ensure that the processed picture and audio are of uniform volume; D2, performing audio denoising based on the picture audio whose volume has been adjusted in step D1, specifically comprising the following steps: D21, for the noisy frequency signal, a section containing only the noise signal is reserved; D22, after the noisy frequency signal is preliminarily denoised by the improved multi-window spectrum subtraction method, it is also necessary to adjust the noise signal collected according to the spectrum subtraction and gain parameters transmitted in the first stage of denoising, such as reducing the signal amplitude; D23, then the adjusted noise signal is used as the actual reference signal and input into the filter together with the audio signal after the first stage of denoising, so as to complete the overall denoising process and obtain a relatively pure audio signal; D24, finally, the audio signal after denoising can be intercepted and processed according to the time interval size of the interval containing only noise signals reserved in the previous stage to obtain a pure audio signal; D3, extracting audio keywords from the denoised pure audio signal, specifically including the following steps: D31, feature extraction is to extract the speech features of the collected audio signal. This step can effectively enhance the key features of the audio signal and reduce the computational dimension of keyword recognition; D32, the neural network part is to classify the key information obtained after feature extraction, so as to obtain the result after keyword recognition in the audio signal; D4, performing subsequent processing based on the noise-free frequency keywords generated in steps D3 and D2, wherein the subsequent processing includes synthesizing the audio and video images and adjusting the audio based on an audio waveform adjustment algorithm.

6. A video image processing system acquired by multiple lenses, which is applicable to a video image processing method acquired by multiple lenses as claimed in any one of claims 1 to 5, characterized in that: Including image acquisition module, image adjustment module, audio acquisition module, audio adjustment module, video detection module: The image acquisition module is used to collect different video images acquired by multiple lenses, and according to the time when the multiple lenses acquired the images, the different video images acquired by the multiple lenses at the same time are uniformly processed, and the screen images are sorted; The image adjustment module is used to receive different video images collected by multiple lenses at the same time and sent by the image acquisition module, and to splice two adjacent pictures; The audio acquisition module is used to collect the audio of the picture collected when multiple lenses capture video images, and perform audio processing on the audio of the picture, wherein the audio processing includes audio denoising, audio keyword recognition and audio volume adjustment; The audio adjustment module is used to match the processed video image with the processed multiple pictures and sounds by using a data matching algorithm, and synthesize the pictures and audios for which matching results exist; for pictures and sounds for which matching results do not exist, perform audio adjustment on the sounds based on an audio waveform adjustment algorithm, and synthesize the adjusted audio with the video image; The video detection module detects the lens image and audio playback effect simultaneously based on the video detection algorithm.

Images