Gait recognition sound production method based on artificial intelligence

Abstract

The application discloses a gait recognition sound production method based on artificial intelligence and belongs to the technical field of film and television production.The method is implemented as follows: firstly, input image data is preprocessed, and the whole image data is divided into a plurality of shots; behavior recognition is performed on each shot to obtain a plurality of shots with gait behavior; high-precision skeleton recognition is performed on the shots to obtain high-precision posture skeleton information; then, each role gait time point or inference gait time point of each shot is obtained through gait recognition; and sound output and picture visualization are performed according to the recognized footstep time point information.The application accurately and rapidly detects role motion gait information through computer vision, outputs in an audio format or accesses an existing system in an application programming interface mode, assists intelligent production of movie sound, reduces the cost input of sound footstep production, and improves the accuracy of gait compliance.

Description

Technical Field

[0001] This invention relates to a method for producing film sound, and more particularly to a method for producing sound based on gait recognition using artificial intelligence, belonging to the field of film and television production technology. Background Technology

[0002] In film production, the realism of sound effects is a crucial part of the audience experience. Good sound effects enhance the immersive experience, allowing viewers to fully enjoy the film's content. Conversely, poor sound effects can break the immersion and disrupt the viewing experience. Gait sound simulation is a particularly important aspect of sound production. Gait sounds significantly impact the realism of character movements and the sense of spatial proximity.

[0003] In today's film production process, footsteps are typically created manually by professional sound engineers. They rely on experience to judge movement patterns on screen and manually mark each gait point. For gait points outside the frame, such as in medium shots, close-ups, and extreme close-ups, sound engineers need extensive experience to infer gait timing. Some gait points are even difficult to judge accurately manually, requiring repeated trials and adjustments to determine the correct timing. For shots with multiple characters or a large number of shots, the workload for sound engineers is immense. To achieve excellent sound quality, this step often consumes significant time and manpower, and is also subject to considerable possibility of experiential misjudgment. Summary of the Invention

[0004] To address the problems of high time and labor costs and high misjudgment rates in existing sound footstep production methods, the main objective of this invention is to propose an artificial intelligence-based gait recognition sound production method. This method uses computer vision to accurately and quickly detect character movement gait information, outputting it in audio format or integrating it into existing systems via an Application Programming Interface (API). This assists in the intelligent production of film sound, reduces the cost of producing sound footsteps, and improves the accuracy of gait matching.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] This invention discloses an artificial intelligence-based method for creating gait recognition sound. First, the input image data is preprocessed, dividing the entire image data into several shots. For each shot, behavior recognition is performed to obtain several shots with gait behavior. High-precision skeleton recognition is then performed on these shots to obtain high-precision posture skeleton information. Next, gait recognition is used to obtain the gait time points or inferred gait time points for each character in each shot. Based on the recognized footstep time point information, sound output and visual visualization are performed. This method aims to reduce the investment in creating sound footsteps and improve the accuracy of gait matching.

[0007] This invention discloses a method for generating gait recognition sound based on artificial intelligence, comprising the following steps:

[0008] Step 1: Based on the scene production requirements of the film, select a specified portion of the images as data input and perform preprocessing before gait recognition.

[0009] Based on the input video file, it is converted into a corresponding low-bitrate proxy file. Then, based on the similarity of shots, inter-frame similarity is measured using the Sum of Absolute Differences (SAD), comparing the absolute difference of pixels at the same spatial location before and after time, i.e., the Manhattan distance L between two frames.

[0010] L=∑(P1-P′1) where P1 and P′1 are the absolute values ​​of the pixels at the same spatial location before and after time.

[0011] If L exceeds the scene segmentation threshold T, it is considered as two shots with a clipping point; otherwise, it is determined as the same continuous shot.

[0012] Step 2: Based on the scene segmentation footage from Step 1, perform behavior recognition to obtain high-precision skeleton pose information.

[0013] To accelerate the efficiency of behavior recognition, frame extraction optimization is performed. The behavior sequence of each independent shot is filtered, and only shots with gait behavior are retained.

[0014] Step 3: Based on the footage with gait behavior selected in Step 2, perform high-precision skeleton pose recognition.

[0015] Different recognition models were used for different numbers of skeletal joints, and the recognition results were the position coordinates of each joint and its recognition confidence αc.

[0016] Step 4: Based on the high-precision skeleton posture information identified in Step 3, perform gait detection.

[0017] For each shot with gait motion, data preprocessing and filtering analysis are performed: shots with empty high-precision skeleton posture detection data identified in step three are skipped; otherwise, they are retained.

[0018] In shots with gait movement, gait analysis is performed on the movement data of each character: for shots where the height of the human body in the frame is less than the proportion threshold, it is determined that the physical distance between the character and the camera is too far, and gait sound simulation is not required in the production, so it is skipped; otherwise, it is retained.

[0019] Define the processing between In-Out (IO) time points, and do not perform gait judgment for non-IO ranges; exclude shots with a single shot time less than a certain number of frames, and the frame number threshold is Tf frames by default.

[0020] After selecting the footage, perform FFT transformation on each joint.

[0021] Part′ = FFT(Part)

[0022] Where FFT is the Fourier transform function, Part is the joint data of each part, and Part′ is the Fourier transform result of each joint.

[0023] Calculate the y-axis position (HZ) of each joint transformed data Part′, remove the maximum fundamental frequency, and then superimpose the average HZ values ​​of each part to obtain the overall attitude motion frequency (HZ′). If it falls within the set HZ range... min and HZ max In this context, data for a specific part is temporarily retained; if it falls outside the specified range, the shot is discarded. Here, n represents the number of parts.

[0024]

[0025] Loss HZ =|HZ′-HZ part |

[0026] Take the K largest dominant frequencies maxk from Part′ and filter them. The difference between the filtered data V′ after inverse Fourier transform (IFFT) and the original attitude detection data V is used as the numerical loss. V Calculate its mean V Mean .

[0027] V′=IFFT(maxk Part′)

[0028] Loss V =|VV′|

[0029]

[0030] Calculate the signal-to-noise ratio (SNR) V′ based on the inverse-transformed data V′. SNR The SNR is also calculated using the original attitude detection data to obtain V. SNR Calculate its loss. SNR .

[0031] V SNR =SNR(V)

[0032] V′ SNR =SNR(V′)

[0033] Loss SNR =|V′ SNR -V SNR |

[0034] Calculate the frequency loss F' of the original attitude detection data frequency F and V′. FFT ,

[0035] Loss FFT =|FF′|

[0036] and over a thousand main frequency distribution loss MFreq Fk is the frequency threshold:

[0037] Loss MFreq =arg(F>Fk)

[0038] Here, arg represents the subset consisting of all parts that meet the conditions.

[0039] The above indicators are integrated and statistically analyzed, including frequency loss (Hz), relative numerical loss, data confidence level (SNR) loss, frequency loss, and loss exceeding 1000MHz, with the loss calculated by summing these factors. Here, α, β, γ, δ, and ε represent the loss coefficients for each indicator.

[0040] Loss = α·Loss HZ +β·Loss V +γ·Loss SNR +δ·Loss FFT +ε·Loss MFreq

[0041] The criteria for determining if a step is disconnected are: Prioritize determining if the frequency loss (Hz) falls within the specified Hz range. Among them, priority is given to the joint data that match the overall data at a single HZ level, and the confidence level αc is required to be greater than the threshold αc. T The standard deviation s is greater than the threshold s T To ensure a certain range of motion, the selected part is Part select .

[0042] Part select =argmin(Loss HZ ,ac,s)

[0043] Here, argmin represents a subset of all locations where the loss is minimized while satisfying the conditions.

[0044] If no conditions are met For the joint data, an integral metric is used for selection. Under the condition of having a good number of n curves and a small number of large dominant frequencies k, the integral S is selected if it is greater than the integration threshold T. S Furthermore, the overall confidence level (ac) and standard deviation are both greater than T. ac and T std The curve Part select :

[0045] Part select =argmax S(Loss, n, k)

[0046] The integral S is calculated using the HZ integral S. HZ Numerical loss integral S V Signal-to-noise ratio loss integral S SNR Frequency loss integral S FFT , frequency loss integral S MFreq Accumulation. The T mentioned above. V T SNR T FFT T MFreq These are the numerical integration threshold, signal-to-noise ratio integration threshold, frequency integration threshold, and dominant frequency integration threshold, respectively. argmax represents a subset of all parts where the loss is minimized when all conditions are met.

[0047] S = S HZ +S V +S SNR +S FFT +S MFreq

[0048] S HZ =1 if HZ∈[HZ min HZ max ], else 0

[0049] S V =1if Loss V <T V else 0

[0050] S SNR =1if Loss SNR <T SNR else 0

[0051] S FFT =1if Loss FFT <T FFT else 0

[0052] S MFreq =1if Loss MFreq <T MFreq else 0

[0053] If the integral condition cannot select the information with steps, i.e., S < T S Then, through Loss MFreq Choose a value greater than the product threshold T. MFreq Furthermore, the overall confidence level (ac) and standard deviation are both greater than T. ac and T std The curve Part select :

[0054] Part select =argmaxS(Loss MFreq ,αc,s)

[0055] If the step-break condition is still not met If the data for that part or that lens is missing, then it is determined that the step is not present.

[0056] Part select =0.

[0057] Step 5: Based on the gait detection results, generate sound and create reference drawings;

[0058] After determining the gait of each character within each shot, the gradient dS of the selected inverse transform line S is calculated. Users can choose from multiple post-processing modes, including viewing only the deceleration phase (dS < 0) or only the landing phase (S < 0). Gait information is then placed individually on each track according to each character's movement, marked using chirps or a specified sound source. For shots lacking character information, or shots where no information is detected, gait frequency extension is performed for N shots forward or backward to fill in the gaps and generate corresponding gait information. Audio can be saved as multiple mono or a single multi-channel audio file, or accessed via a third-party API.

[0059] Beneficial effects

[0060] 1. The present invention discloses an artificial intelligence-based gait recognition sound production method, which adopts a combination of posture detection technology and gait detection analysis to achieve relatively accurate and high-speed detection of character movement gait information.

[0061] 2. The present invention discloses an artificial intelligence-based gait recognition sound production method, which adopts multiple gait judgment mechanisms to meet the possibility of adaptive application in various movie shots, and has greater versatility and generalization.

[0062] 3. The present invention discloses an artificial intelligence-based gait recognition sound production method, which uses an integral system for gait integration switching to meet the high accuracy requirements of gait judgment in movie shots.

[0063] 4. The present invention discloses an artificial intelligence-based gait recognition sound production method, which uses audio visualization to assist the recording studio in better checking and secondary creation based on it, saving a lot of time and manpower costs. Attached Figure Description

[0064] Figure 1 This is a flowchart of a gait recognition sound generation method based on artificial intelligence disclosed in this invention. Detailed Implementation

[0065] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The technical problems solved by the present invention and its beneficial effects are also described. It should be noted that the described embodiments are only intended to facilitate understanding of the present invention and do not constitute any limitation thereof.

[0066] This invention is an artificial intelligence-based method for producing gait recognition sound. Taking a movie volume as an example, the implementation of this invention is described in detail. According to the technical solution of this invention, gait data in the volume is automatically identified. The automatically identified data is compared with the actual gait data identified manually. Experimental data shows that this invention has a very high gait recognition rate and can achieve the goal of automatically identifying gait based on video information, saving manual labor costs and time costs.

[0067] The implementation of this invention in this example includes the following steps:

[0068] Step 1: Input Image Data and Preprocessing: Based on the scene production requirements of the film, select a specified portion of the images as data input and perform gait recognition preprocessing on them.

[0069] The specific process is as follows: Based on the input MOV video file or RAW clip file captured by the camera (the clip in this case is a 22-minute MOV file), it is converted into a corresponding low-bitrate proxy file, in this case, MP4. Then, based on the similarity of the shots, inter-frame similarity is measured by the sum of absolute differences (SAD). The difference in absolute pixel values ​​(P1 and P1') between frames at the same spatial location before and after time is compared, which is equivalent to calculating the Manhattan distance L between two frames. If the distance exceeds the scene segmentation threshold T, they are considered two shots S with a clipping point. If it does not exceed the threshold, they are determined to be the same continuous shot. Here, T is set to 0.2% by default based on most video test results. The segmented footage yields 529 shots.

[0070] L=∑(P1-P′1)

[0071] Step 2: Based on the scene segmentation footage from Step 1, perform behavior recognition to obtain high-precision skeleton pose information.

[0072] For shots with completed scene segmentation, behavior recognition is performed. To accelerate the efficiency of behavior recognition, frame extraction optimization is selected. Specifically, various optimization schemes, such as reducing the frame rate or bitrate, are used to optimize the material and processing. Here, we use 24 frames, 12 frames, or 6 frames per second for frame extraction optimization to accelerate the process. At the same time, the video bitrate is reduced to achieve rendering speeds above real-time; the default is 1500kbps, with a frame rate of 6 frames per second. After passing through the behavior recognition module, the behavior sequence of each independent shot is filtered, retaining only shots with gait behaviors such as walking and running for subsequent processing. The behavior filtering can be customized to select shots with only walking, only running, or other combined filtering conditions. Finally, a total of 258 shots with walking or running behaviors are obtained, resulting in 3961 frames with behavioral behavior after frame extraction.

[0073] Step 3: Based on the footage with gait behavior obtained in Step 2, perform high-precision skeleton pose recognition.

[0074] The appropriate CPU / GPU / XPU was selected for computation based on the actual image conditions and hardware specifications, and different recognition models were used for different numbers of skeleton joints. The recognition results are the coordinates (x, y) of each joint position and its recognition confidence αc.

[0075] Step 4: Based on the high-precision skeleton posture information obtained in Step 3, perform gait detection.

[0076] For each shot featuring gait movement, data analysis is performed. If the detection result data is empty, the process is skipped. Within the gait movement shot S, gait analysis is performed on the motion data C of each character. If the proportion of the human body's height within the frame is less than a certain threshold, the physical distance between the character and the camera is too far, and gait sound simulation is not required for production purposes; therefore, this step is skipped. In 2K images, the default threshold is T. P The default value for each pixel, such as the T value for a 2K image (1920x1080pix / 2048x1080pix). P The value is 150 pixels; the T value of a 4K image (3840x2160pix / 4096x2160pix). P The value is 300 pixels, meaning the character occupies approximately 14% of the frame. Processing is performed within a specified I / O timeframe; gait detection is not performed outside of I / O ranges. Shots shorter than a certain frame count are excluded due to their brevity; the frame count threshold defaults to T. F Frame. Here, T F The frame rate depends on the human body's movement, generally T F In a movie with a frame rate of 24, T F Shots with a frame rate of 40 frames or less are deemed not to meet the conditions for attitude prediction.

[0077] After filtering the footage, FFT transformations are performed on various joints, such as the shoulder, elbow, wrist, hip, knee, and ankle. In this case, the shoulder, stride, knee, and ankle are selected for recognition.

[0078] Part′=FFT(Part), part={Shoulder, Hip, Knee, Ankle}

[0079] Where FFT is the Fourier transform function, and Part is the joint data for each part.

[0080] Calculate its y-axis position HZ, remove the maximum fundamental frequency, and then superimpose the average HZ of each part to obtain the overall perceived motion HZ'. If it is within a reasonable HZ setting range, the data of that part is temporarily retained.

[0081]

[0082] Loss HZ =|HZ′-HZ part |

[0083] Take the K largest main frequencies (K=2 here), perform filtering, and then perform inverse transform on the filtered data. The difference between this filtered data and the original data V is used as the numerical loss. VCalculate their mean.

[0084] After inverse transformation, the SNR value is calculated, aiming to minimize the SNR loss, with a threshold of T. SNR The default value is 20. Calculate the frequency loss F. FFT and over a thousand main frequency distribution loss MFreq Here, Fk = 1000:

[0085] Loss MFreq =arg(F>Fk), Fk=1000

[0086] Here, arg represents the subset consisting of all parts that meet the conditions.

[0087] The above indicators are statistically integrated, and the following are statistically analyzed: HZ loss, relative numerical loss, data confidence SNR loss, frequency loss, and loss exceeding 1000 MHz. α, β, γ, δ, and ε are the loss coefficients for each indicator.

[0088] Loss = α·Loss HZ +β·Loss V +γ·Loss sNR +δ·Loss FFT +ε·Loss MFreq

[0089] The step-out condition prioritizes evaluating HZ (Hardship-Zone) matching data. Specifically, it prioritizes data from joints where a single HZ matches the overall data, while also requiring a high confidence level αc, greater than the threshold αc. T And the standard deviation s is high, exceeding the threshold s. T This is to ensure that there are certain fluctuations in its movements. Among them, αc T The default value is 0.5. T The default value is 10. This example uses the default value.

[0090] Part select =argmin(Loss HZ ,ac,s)

[0091] Here, argmin represents a subset of all locations where the loss is minimized while satisfying the conditions.

[0092] If no suitable joint data is available, an integral metric is used for selection, choosing the best n curves (n defaults to 4). Curves with a relatively low dominant frequency k (default k=2) and high overall confidence level (ac) and standard deviation are selected.

[0093] Part select =argmax S(Loss, n, k), n = 4, k = 2

[0094] The integral S is calculated using the HZ integral S. HZ Numerical loss integral S V Signal-to-noise ratio loss integral S SNR Frequency loss integral S FFT , frequency loss integral S MFreq Accumulation. T V T SNR T FFT T MFreq These are the numerical integration threshold, signal-to-noise ratio integration threshold, frequency integration threshold, and dominant frequency integration threshold, respectively. argmax represents a subset of all locations where the loss is minimized while satisfying the conditions. Here, T... V Take the mean (Loss) of the numerical losses for all parts. V ), T MFreq In this context, n(Freq) represents the total number of frequencies.

[0095] S = S HZ +S V +S SNR +S FFT +S MFreq ;

[0096] S HZ =1 if HZ∈[HZ min HZ max ], elSe 0;

[0097] S V =1if Loss V <T V else 0, T V =mean(Loss) V );

[0098] S SNR =1if Loss SNR <T SNR else 0, T SNR =20;

[0099] S FFT =1if Loss FFT <T FFT else 0, T FFT =2;

[0100] S MFreq =1if Loss MFreq <T MFreq else 0, T MFreq = 0.1*n(Freq);

[0101] If the integral condition cannot select the information with steps, i.e., S < T S (T s =4), then through Loss MFreq Choose a value greater than the product threshold T. MFreq Furthermore, the overall confidence level (ac) and standard deviation are both greater than T. ac and T std The curve Part select :

[0102] Part select =argmax(Loss MFreq , αc, s), ac=0.5, s=10

[0103] T MFreq = 0.1*n(Freq)

[0104] If the condition for termination is still not met, then it is determined that the data for that part or the camera lens does not exist.

[0105] After gait detection, a total of 184 characters had gait information.

[0106] Step 5: Based on the above steps, after gait detection and recording, generate sound and create reference drawings:

[0107] After determining the gait of each character within each shot, the gradient dS is calculated for the selected inverse transform line S. Here, dS can be amplified close to zero; the default is 0.01 amplified to 10, facilitating sound engineers' review and backtracking. Multiple post-processing modes are available, including those focusing only on deceleration phases (dS < 0) or landing phases (S < 0). Gait information is then placed individually on each track according to each character's movement, marked using kilohertz or a specified audio source. Furthermore, if some shots lack character information, the missing information can be filled by extending the gait frequency across N shots before and after, generating corresponding gait information. Audio can be saved as multiple mono or a single multi-channel audio file, or integrated via a third-party API. The maximum number of characters per shot is 18; depending on actual needs, the gait of at most the first five characters from each shot is output as WAV audio, with different characters marked at different kilohertz frequencies.

[0108] Comparison of case implementation results:

[0109] By statistically analyzing the actual gait data GT, the number of detected gait data DT, the gait error d, and the accuracy rate of each shot in some shots with gait requirements, as shown in Table 1, most shots can achieve a high accuracy rate. The experiments in this example demonstrate that the present invention can achieve the goal of automatically recognizing gait based on video information, saving manual labor costs and time costs.

[0110] Table 1: Comparison of some lens test results with actual values.

[0111] Lens ID Actual gait count GT Gait detection count DT Gait error number d Accuracy AC 1 6 7 1 83％ 2 5 6 1 80％ 3 3 3 0 100％ 4 4 4 0 100％ 5 10 11 1 90％ 6 7 8 1 86％ 7 4 4 0 100％ 8 4 4 0 100％ 9 2 2 0 100％ 10 11 12 1 91％ 11 3 3 0 100％ 12 4 4 0 100％ 13 2 2 0 100％ 14 5 8 3 40％ 15 9 9 0 100％ 16 10 12 2 80％ 17 7 7 0 100％ 18 12 11 1 92％ 19 6 6 0 100％ 20 4 3 1 75％

[0112] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating gait recognition sound based on artificial intelligence, characterized in that: Includes the following steps, Step 1: Based on the scene production requirements of the film, select a specified portion of the images as data input and perform preprocessing before gait recognition; Step 2: Based on the scene segmentation footage from Step 1, perform behavior recognition; Step 3: Based on the footage with gait behavior selected in Step 2, perform high-precision skeleton pose recognition to obtain high-precision skeleton pose information; Step 4: Based on the high-precision skeleton pose information identified in Step 3, perform gait detection; The implementation method for step four is as follows: For each shot with gait motion, perform data preprocessing and filtering analysis: skip shots where the high-precision skeleton posture detection data identified in step three is empty; otherwise, retain them. In shots with gait movement, gait analysis is performed on the movement data of each character: for shots where the height of the human body in the frame is less than the proportion threshold, it is determined that the physical distance between the character and the camera is too far, and gait sound simulation is not required in the production, so it is skipped; otherwise, it is retained. Define the processing between entry and exit time points, and do not perform gait judgment for non-IO ranges; exclude shots with a single shot time of less than a certain number of frames, with the frame number threshold defaulting to Tf frames; The selected footage is then subjected to FFT transformation on each joint. Where FFT is the Fourier transform function, Data for joints in various parts, The Fourier transform results for each joint; Calculate the data after each joint transformation Its y-axis position HZ, after removing the maximum fundamental frequency, the average HZ of each part is calculated to obtain the overall attitude motion frequency. If it is set within the HZ range , In this process, the data for a specific part is temporarily retained; if the part is not within the specified range, the shot is discarded; where n is the number of parts. ； ,Pick The K largest main frequencies The data is filtered; after filtering, an inverse Fourier transform is performed to obtain the reconstructed data. ,calculate The difference between the original attitude detection data V and the original attitude detection data V is used as the numerical loss. Then calculate its mean. ; ； ； ； Based on the data after inverse transformation Calculate the signal-to-noise ratio value The SNR was also calculated using the original attitude detection data, resulting in... ; calculate its loss ; ； ； ； Calculate the frequency of original attitude detection data and frequency The difference is used as the loss. , ; Calculate the main frequency distribution of over 1000 , : in, Here, arg represents the frequency threshold, and arg represents the subset consisting of all parts that meet the conditions. Integral statistics were performed on the above indicators, and the frequency loss (Hz), relative numerical loss, data confidence level (SNR) loss, frequency loss, and loss exceeding 1000 MHz were summed to obtain the final result. , ,in and These are the loss coefficients for each item; The criteria for determining if a step is disconnected are: first, whether the frequency loss in Hz falls within the specified range. Priority is given to joint data that match the overall data at a single Hz level, while also requiring a certain confidence level. Greater than the threshold Standard deviation Greater than the threshold To ensure that it has a certain range of motion, the selected parts are : Where argmin represents a subset of all locations where the loss is minimized while satisfying the conditions; If no conditions are met For the joint data, an integral metric is used for selection. The integral metric is chosen when there are a good number of n curves and a relatively small number of large dominant frequencies k. Greater than the integration threshold And overall confidence level and standard deviation are respectively greater than and part : The integral S is calculated using the HZ integral. Numerical loss integral Accumulation; ； ； ； ； ； ； in, These are the numerical integration threshold, signal-to-noise ratio integration threshold, frequency integration threshold, and main frequency integration threshold, respectively. This represents a subset of all locations where the maximum loss is achieved while satisfying all conditions. If the integral criterion cannot select information with steps, that is... At that time, through Select a value greater than the main frequency integral threshold. And overall confidence level and standard deviation are respectively greater than and part : ； If the step-break condition is still not met If the location data or camera data does not contain steps, then it is determined that there is no movement. ; Step 5: Based on the gait detection results, generate sound and create reference drawings.

2. The method for generating gait recognition sound based on artificial intelligence as described in claim 1, characterized in that: The implementation method for step one is as follows: The input video file is converted into a corresponding low-bitrate proxy file; then, based on the similarity of shots, inter-frame similarity is measured by summing the absolute differences, comparing the absolute difference of pixels at the same spatial location before and after time, i.e., the Manhattan distance L between two frames: in The absolute values ​​of pixels at the same spatial location before and after time; If L exceeds the scene segmentation threshold T, it is considered as two shots with a clipping point; otherwise, the shots are determined to be the same continuous shot.

3. The method for generating gait recognition sound based on artificial intelligence as described in claim 1, characterized in that: The implementation method for step two is as follows: To accelerate the execution efficiency of behavior recognition, frame extraction optimization is performed; behavior recognition is performed based on the scene segmentation in step one, and the behavior sequence of each independent shot is filtered to retain only the shots with gait behavior.

4. The method for generating gait recognition sound based on artificial intelligence as described in claim 1, characterized in that: The implementation method for step three is as follows: For different numbers of skeleton joints, different recognition models are used for high-precision skeleton pose recognition. The recognition results are the position coordinates of each joint and their recognition confidence scores. .

5. The method for generating gait recognition sound based on artificial intelligence as described in claim 1, characterized in that: The implementation method for step five is as follows: After determining the gait of each character in each shot, the selected inverse transform line is calculated. corresponding gradient Users can choose from a variety of post-processing modes, including viewing only the deceleration phase. Or just look at the landing stage In each track, gait information is placed separately according to the movement of each character, and gait is marked by a thousand cycles or a specified sound source; for some shots without character information, or shots without information after detection, gait frequency extension is performed to fill in the gaps by selecting N shots forward or backward to generate the corresponding gait information; Audio can be saved as multiple mono or a single multi-channel audio, or accessed via a third-party API.

Images