Lighting linkage method, system and device based on feature recognition

Abstract

This application discloses a lighting linkage method, system, and device based on feature recognition, relating to the technical field of machine learning. The method includes: acquiring a real-time audio stream of a target online concert and pre-extracted static features of the current song; extracting features from the real-time audio stream to obtain dynamic features of the song; generating global static atmosphere parameters based on the song's static features; generating partitioned dynamic atmosphere parameters for each logical device area based on the partition mapping rules of each logical device area and the song's dynamic features; and for each logical device area, fusing the partitioned dynamic atmosphere parameters of that logical device area with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures. This application enables differentiated and precise lighting control of various peripheral areas in online concerts, enhancing the audience's immersive experience.

Description

Technical Field

[0001] This application relates to the technical field of machine learning, and in particular to a method, system and device for lighting linkage based on feature recognition. Background Technology

[0002] With the development of the internet and streaming technology, online concerts have become an important way for the public to enjoy musical performances. To enhance the immersive experience for users watching online performances, technologies have emerged that use the lighting of computer peripherals to create atmosphere. These technologies typically involve controlling the color and brightness changes of devices such as monitor backlights, keyboards, mice, host computer backlight strips, and secondary screen ambient lighting to match the audio or video content being played.

[0003] Existing music and lighting synchronization methods can be broadly categorized into two types. One type is based on overall audio characteristics, using global features such as rhythm, volume, or total spectral energy of the audio signal to drive lighting changes. While this method can make the lights dynamically flash or change color in sync with the music, its drawback lies in its inability to recognize the song's segment structure, such as intros, verses, and choruses, and its difficulty in distinguishing between vocals, instruments, and ambient sounds. This results in a monotonous lighting pattern that fails to reflect the emotional progression and segment differences within the music. The other type uses pre-programmed lighting scripts, where lighting effects are pre-arranged for specific tracks and triggered only according to the timeline during playback. This method cannot dynamically respond to real-time audio content; the lights cannot adjust accordingly to singer improvisation, audience interaction, or sudden changes in atmosphere, resulting in a rigid and inflexible appearance.

[0004] Furthermore, in the actual viewing scenario of online concerts, the spatial positions and visual functions of different peripherals vary. For example, monitor backlights are mainly used to extend the atmosphere of the screen, while keyboards and mice are closer to the user's tactile perception area, and secondary screen ambient lighting can create ambient light. However, most existing methods use the same control logic for all devices without differentiated zone control, resulting in a lack of layering and coordination in the lighting of multiple device zones, making it difficult to create a three-dimensional, immersive light and shadow effect.

[0005] Since the audio content of online concerts includes not only the overall emotional evolution preset by the song segments, but also the technical details of the singer's live performance and the real-time reactions of the audience, the current lighting linkage technology cannot simultaneously integrate this information and generate coordinated and refined lighting instructions for different viewing peripheral areas. This has become a key issue restricting the improvement of the immersive experience of online concerts. Summary of the Invention

[0006] To achieve differentiated and precise lighting control for various peripheral areas in online concerts and enhance audience immersion, this application provides a lighting linkage method, system, and device based on feature recognition.

[0007] Firstly, this application provides a light-linking method based on feature recognition, employing the following technical solution: Lighting linkage methods based on feature recognition include: Acquire the real-time audio stream of the target online concert and the pre-extracted static features of the current song, wherein the static features of the song include the segment features corresponding to each segment of the current song. Feature extraction is performed on the real-time audio stream to obtain the song's dynamic features; Based on the static features of the song, global static atmosphere parameters are generated. The global static atmosphere parameters are used to drive each logical device area to execute a basic lighting state that matches the current song segment. The logical device area includes at least one of the following: main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area. Based on the partition mapping rules of each logical device area, and according to the dynamic characteristics of the song, partition dynamic atmosphere parameters of each logical device area are generated. For each of the aforementioned logical device areas, the partition dynamic atmosphere parameters of that logical device area are fused with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures.

[0008] By employing the above technical solution, the real-time audio stream of the target online concert and the pre-extracted static features of the current song are obtained. The static features of the song include the segment features corresponding to each segment of the current song. Then, feature extraction is performed on the real-time audio stream to obtain the dynamic features of the song. Based on the static features of the song, global static atmosphere parameters are generated. The global static atmosphere parameters are used to drive each logical device area to execute the basic lighting state matching the current song segment. The logical device area includes at least one of the main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area. Then, based on the partition mapping rules of each logical device area, partition dynamic atmosphere parameters of each logical device area are generated according to the dynamic features of the song. Finally, for each logical device area, the partition dynamic atmosphere parameters of the logical device area are fused with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lights. This invention overcomes the problems of existing technologies, such as single lighting modes and inability to respond to live dynamics, by integrating real-time audio streams with pre-extracted song segment features. Static atmosphere parameters drive smooth transitions between device zones with each segment, restoring emotional progression, while dynamic features capture improvisation and audience feedback in real time, achieving instant response. Through differentiated partitioning mapping rules, each peripheral area responds to beats, timbres, etc., according to its spatial function, constructing a layered three-dimensional light and shadow, integrating dynamic and static parameters, and superimposing brightness, color modulation, and special effects on a unified tone to achieve refined control that combines movement and stillness, thereby enhancing the immersive experience of online concerts.

[0009] Optionally, the step of extracting features from the real-time audio stream to obtain the song's dynamic features includes: The real-time audio stream is segmented and windowed to obtain multiple audio frames; The audio frames are subjected to sound source separation processing to obtain the singer's performance audio frames and the live atmosphere audio frames; Feature extraction is performed on the singer's performance audio frames to obtain performance dynamic features; and feature extraction is performed on the ambient sound audio frames to obtain ambient dynamic features. Align the singing dynamic features and the atmospheric dynamic features on the time axis and then stitch the features together to obtain the song dynamic features.

[0010] By adopting the above technical solution, in order to achieve feature extraction of real-time audio stream, the real-time audio stream is segmented and windowed to obtain multiple audio frames. Then, the audio frames are processed by sound source separation to obtain singer performance audio frames and ambient sound frames. Then, feature extraction is performed on the singer performance audio frames to obtain performance dynamic features; and feature extraction is performed on the ambient sound audio frames to obtain ambient dynamic features. Then, the performance dynamic features and ambient dynamic features are aligned on the time axis and spliced ​​together to obtain song dynamic features.

[0011] Optionally, the step of extracting features from the singer's audio frames to obtain dynamic features of the performance includes: A short-time Fourier transform is performed on the audio frames of the singer's performance to obtain the first spectrogram; Multidimensional feature extraction is performed on the first spectrogram to obtain the corresponding basic acoustic features, wherein the basic acoustic features include the spectral centroid, the fundamental frequency trajectory of the human voice, the harmonic energy ratio, the spectral envelope features, and the energy distribution features; The frequency range corresponding to the first spectrogram is divided into multiple sub-bands, and the energy envelope of each sub-band is extracted; and the starting point is detected for the energy envelope of each sub-band to obtain the starting point intensity of each sub-band; and the starting point intensity is weighted and fused based on the preset contribution weight of each sub-band to human ear rhythm perception to obtain the beat intensity sequence. Based on the spectral envelope characteristics and the energy distribution characteristics, the singer's audio frames are classified to obtain a singing technique classification identifier, wherein the singing technique classification identifier includes at least one of true voice, falsetto, mixed voice and breathy voice; The spectral centroid and the fundamental frequency trajectory of the human voice are normalized and weighted to obtain the timbre brightness characteristics; The dynamic characteristics of the performance are obtained by splicing together the beat intensity sequence, the timbre brightness and darkness characteristics, the harmonic energy ratio and the singing technique classification identifier.

[0012] By employing the above technical solution, in order to achieve feature extraction from the singer's audio frames, a short-time Fourier transform is performed on the singer's audio frames to obtain a first spectrogram. Then, multi-dimensional feature extraction is performed on the first spectrogram to obtain the corresponding basic acoustic features. These basic acoustic features include the spectral centroid, the fundamental frequency trajectory of the human voice, the harmonic energy ratio, the spectral envelope features, and the energy distribution features. Next, the frequency range corresponding to the first spectrogram is divided into multiple sub-bands, and the energy envelope of each sub-band is extracted. Furthermore, the starting point of the energy envelope of each sub-band is detected to obtain the starting point strength corresponding to each sub-band. The intensity of the starting point is weighted and fused based on the preset contribution weights of each sub-band to the human ear's rhythm perception to obtain a beat intensity sequence. Then, the singer's audio frames are classified according to the spectral envelope features and energy distribution features to obtain singing technique classification labels. The singing technique classification labels include at least one of true voice, falsetto, mixed voice, and breathy voice. Then, the spectral centroid and the fundamental frequency trajectory of the human voice are normalized and weighted to obtain timbre brightness features. Finally, the beat intensity sequence, timbre brightness features, harmonic energy ratio, and singing technique classification labels are spliced ​​together to obtain singing dynamic features.

[0013] Optionally, the step of extracting features from the ambient audio frame to obtain ambient dynamic features includes: A short-time Fourier transform is performed on the ambient audio frame to obtain a second spectrogram; Based on multiple preset frequency band ranges, the frequency band energy values ​​corresponding to each preset frequency band are extracted from the second spectrum map, and the total energy of the entire frequency band of the second spectrum map is calculated. Based on the frequency band energy value and the total energy of the entire frequency band, the energy proportion of each preset frequency band is calculated to obtain the on-site spectrum energy distribution vector; The total energy across the entire frequency band is smoothed by a sliding window to obtain a smooth energy curve, which is then used as the field energy sequence. Based on the smooth energy curve, the energy surge ratio of the smooth energy of the current audio frame relative to the average energy of the previous window is calculated. When the energy surge ratio exceeds a preset threshold, the current audio frame is marked as an atmosphere burst point, and the corresponding burst intensity level is determined according to the energy surge ratio. The atmospheric dynamic characteristics are obtained by splicing together the on-site spectral energy distribution vector, the on-site energy sequence, the atmosphere burst point, and the burst intensity level.

[0014] By adopting the above technical solution, in order to extract features from the ambient audio frame, a short-time Fourier transform is performed on the ambient audio frame to obtain a second spectrogram. Then, based on multiple preset frequency bands, the frequency band energy values ​​corresponding to each preset frequency band are extracted from the second spectrogram, and the total energy of the entire frequency band of the second spectrogram is calculated. Then, based on the frequency band energy values ​​and the total energy of the entire frequency band, the energy proportion of each preset frequency band is calculated to obtain the ambient spectrum energy distribution vector. Then, a sliding window smoothing filter is applied to the total energy of the entire frequency band to obtain a smooth energy curve, and the smooth energy curve is used as the ambient energy sequence. Then, based on the smooth energy curve, the energy surge ratio of the smooth energy of the current audio frame relative to the average energy of the previous window is calculated. When the energy surge ratio exceeds a preset threshold, the current audio frame is marked as an atmosphere burst point, and the corresponding burst intensity level is determined according to the energy surge ratio. Finally, the ambient spectrum energy distribution vector, the ambient energy sequence, the atmosphere burst point, and the burst intensity level are concatenated to obtain the ambient dynamic features.

[0015] Optionally, the step of generating global static atmosphere parameters based on the static features of the song includes: Obtain the current playback timestamp of the real-time audio stream; Based on the current playback timestamp, the start and end times of each song segment are found in the static features of the song to determine the target song segment currently in use. Extract the target segment features corresponding to the target song segment from the static features of the song, wherein the target segment features include the target segment type, the target emotion intensity level, and the target main color tone; Based on the target paragraph type, the target emotion intensity level, and the target main color tone, the corresponding basic brightness range, basic light change mode, and basic color temperature are queried from the preset lighting mode library; The basic brightness range, the basic lighting change mode, and the basic color temperature are combined and smoothly interpolated within the time neighborhood of song segment transitions to generate continuously changing global static atmosphere parameters, so as to control the basic lighting state of each logical device area to smoothly transition with the progression of song segments.

[0016] By adopting the above technical solution, in order to generate global static atmosphere parameters, the current playback timestamp of the real-time audio stream is obtained. Then, based on the current playback timestamp, the start and end times of each song segment are found in the song's static features to determine the current target song segment. Then, the target segment features corresponding to the target song segment are extracted from the song's static features. The target segment features include the target segment type, target emotional intensity level, and target dominant color. Then, based on the target segment type, target emotional intensity level, and target dominant color, the corresponding basic brightness range, basic lighting change mode, and basic color temperature are queried from the preset lighting mode library. Then, the basic brightness range, basic lighting change mode, and basic color temperature are combined and smoothly interpolated within the time neighborhood of the song segment transition to generate continuously changing global static atmosphere parameters, so as to control the basic lighting state of each logical device area to smoothly transition with the progress of the song segment.

[0017] Optionally, the step of generating partition dynamic atmosphere parameters for each logical device area based on the partition mapping rules of each logical device area and according to the song dynamic features includes: For each logical device area, obtain the target partition mapping table corresponding to that logical device area. The target partition mapping table includes feature parameter data and dynamic atmosphere parameter data. The target partition mapping table is used to characterize the mapping relationship between the feature parameter data and the dynamic atmosphere parameter data. Based on the target partition mapping table, corresponding target feature parameters are extracted from the song's dynamic features, and a mapping transformation is performed based on the target partition mapping table to generate partition dynamic atmosphere parameters for each logical device area. The partition dynamic atmosphere parameters include dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal.

[0018] By adopting the above technical solution, in order to generate the partition dynamic atmosphere parameters of each logical device area, for each logical device area, a target partition mapping table corresponding to that logical device area is obtained. The target partition mapping table includes feature parameter data and dynamic atmosphere parameter data. The target partition mapping table is used to represent the mapping relationship between feature parameter data and dynamic atmosphere parameter data. Then, based on the target partition mapping table, the corresponding target feature parameters are extracted from the song's dynamic features, and a mapping transformation is performed based on the target partition mapping table to generate the partition dynamic atmosphere parameters of each logical device area. The partition dynamic atmosphere parameters include dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal.

[0019] Optionally, the step of fusing the partition dynamic atmosphere parameters of each logical device area with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures includes: For each of the logical device areas, the basic brightness range, the basic lighting change mode, and the basic color temperature in the global static atmosphere parameters corresponding to that logical device area are obtained, and the current basic brightness is determined within the basic brightness range based on the current timestamp. The dynamic brightness modulation amount, the dynamic color offset amount, and the dynamic effect trigger signal of the logical device area in the dynamic atmosphere parameters of the partition are obtained; The current base brightness is superimposed with the dynamic brightness modulation amount to obtain the first target brightness; The base color temperature is shifted according to the dynamic color shift to obtain the first target color; Based on the basic lighting change pattern, the brightness and color of the first target are time-varyingly modulated to obtain the brightness and color of the second target. When an atmosphere burst point marker is present in the dynamic effect trigger signal, a momentary lighting effect is superimposed on the second target brightness and the second target color based on the burst intensity level corresponding to the atmosphere burst point marker, and a lighting control command is generated; otherwise, a lighting control command is generated based on the second target brightness and the second target color.

[0020] By adopting the above technical solution, in order to control physical lighting fixtures, for each logical device area, the basic brightness range, basic lighting change mode, and basic color temperature in the global static atmosphere parameters corresponding to that logical device area are obtained. The current basic brightness is determined within the basic brightness range based on the current timestamp. Then, the dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal in the partition dynamic atmosphere parameters of that logical device area are obtained. The current basic brightness and the dynamic brightness modulation amount are then superimposed to obtain the first target brightness. The basic color temperature is then offset according to the dynamic color offset amount to obtain the first target color. Based on the basic lighting change mode, the first target brightness and the first target color are time-varyingly modulated to obtain the second target brightness and the second target color. When there is an atmosphere burst point marker in the dynamic effect trigger signal, the instantaneous lighting effect is superimposed on the second target brightness and the second target color according to the burst intensity level corresponding to the atmosphere burst point marker to generate a lighting control command. Otherwise, a lighting control command is generated based on the second target brightness and the second target color.

[0021] Optionally, the step of generating a lighting control command by superimposing a momentary lighting effect on the second target brightness and the second target color based on the burst intensity level corresponding to the atmospheric burst point marker includes: Obtain the burst intensity level corresponding to the atmospheric burst point marker; Based on the burst intensity level, the corresponding instantaneous effect type and effect parameters are queried from the preset instantaneous effect library. The instantaneous effect type includes at least one of brightness pulse, strobe, and color flash, and the effect parameters include effect duration and effect intensity value. Based on the instantaneous effect type and the effect parameters, the brightness and color of the second target are modulated, and a lighting control command is generated according to the modulated brightness and color.

[0022] By adopting the above technical solution, in order to generate lighting control commands, the burst intensity level corresponding to the atmosphere burst point marker is obtained. Then, based on the burst intensity level, the corresponding instantaneous effect type and effect parameters are queried from the preset instantaneous effect library. The instantaneous effect type includes at least one of brightness pulse, strobe, and color flash. The effect parameters include effect duration and effect intensity value. Then, based on the instantaneous effect type and effect parameters, the brightness of the second target and the color of the second target are modulated, and the lighting control commands are generated based on the modulated brightness and color.

[0023] Secondly, this application also provides a feature-based lighting linkage system, which adopts the following technical solution: A feature-recognition-based lighting linkage system includes: The data acquisition module is used to acquire the real-time audio stream of the target online concert and the pre-extracted static features of the current song, wherein the static features of the song include the segment features corresponding to each segment of the current song. The dynamic feature extraction module is used to extract features from the real-time audio stream to obtain the dynamic features of the song; A global static atmosphere parameter generation module is used to generate global static atmosphere parameters based on the static features of a song. The global static atmosphere parameters are used to drive each logical device area to execute a basic lighting state that matches the current song segment. The logical device area includes at least one of the following: main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area. The partition dynamic atmosphere parameter generation module is used to generate partition dynamic atmosphere parameters for each of the logical device zones based on the partition mapping rules of each of the logical device zones and according to the dynamic characteristics of the song. The fusion control module is used to fuse the partition dynamic atmosphere parameters of each logical device area with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures.

[0024] Thirdly, this application also provides a computer device, which adopts the following technical solution: A computer device includes a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the computer program to implement the method described in the first aspect.

[0025] In summary, this application includes at least the following beneficial technical effects: acquiring the real-time audio stream of the target online concert and the pre-extracted static features of the current song, wherein the static features of the song include the segment features corresponding to each segment of the current song; then, feature extraction is performed on the real-time audio stream to obtain the dynamic features of the song; then, based on the static features of the song, global static atmosphere parameters are generated, wherein the global static atmosphere parameters are used to drive each logical device area to execute a basic lighting state matching the current song segment; the logical device area includes at least one of the main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area; then, based on the partition mapping rules of each logical device area, partition dynamic atmosphere parameters of each logical device area are generated according to the dynamic features of the song; then, for each logical device area, the partition dynamic atmosphere parameters of the logical device area are fused with the global static atmosphere parameters to generate corresponding real-time lighting control instructions to control the corresponding physical lighting fixtures. This invention overcomes the problems of existing technologies, such as single lighting modes and inability to respond to live dynamics, by integrating real-time audio streams with pre-extracted song segment features. Static atmosphere parameters drive smooth transitions between device zones with each segment, restoring emotional progression, while dynamic features capture improvisation and audience feedback in real time, achieving instant response. Through differentiated partitioning mapping rules, each peripheral area responds to beats, timbres, etc., according to its spatial function, constructing a layered three-dimensional light and shadow, integrating dynamic and static parameters, and superimposing brightness, color modulation, and special effects on a unified tone to achieve refined control that combines movement and stillness, thereby enhancing the immersive experience of online concerts. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall process of an embodiment of this application.

[0027] Figure 2 This is a schematic diagram of the system structure of this application.

[0028] Figure 3 This is a structural block diagram of the computer device described in this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] This application discloses a lighting linkage method based on feature recognition.

[0031] Reference Figure 1 The light linkage method based on feature recognition includes: Step S11: Obtain the real-time audio stream of the target online concert and the pre-extracted static features of the current song. The static features of the song include the segment features corresponding to each segment of the current song.

[0032] It's important to note that this step serves as the data foundation for the entire lighting coordination process, acquiring two key pieces of information simultaneously. The real-time audio stream is the direct driver of the dynamic lighting response, carrying the real-time acoustic changes in the singer's performance, instrumental playing, and the overall atmosphere. The pre-extracted static features of the song are obtained through offline analysis of the song's audio or sheet music. These features characterize the song's macro-structure, such as dividing a song into sections like intro, verse, chorus, interlude, and outro, and labeling each section with its corresponding mood, style, or color tendency. The combination of these two types of data allows the system to grasp both the overall emotional evolution of the song and capture subtle changes at every moment.

[0033] Step S12: Extract features from the real-time audio stream to obtain the dynamic features of the song.

[0034] It should be noted that this step involves in-depth processing of the real-time audio stream acquired in step S11. The aim is to extract multi-dimensional dynamic information from the continuous sound signal that can characterize the instantaneous changes in the music. Unlike the macroscopic attributes of the segments depicted by the pre-extracted static features of the song, this step focuses on capturing the vivid details in the real-time performance that cannot be pre-arranged. These dynamic features encompass a variety of acoustic attributes closely related to the perception of changes in lighting: they include the beat intensity reflecting the vocal power and rhythmic points, the timbre brightness reflecting the texture of the voice, and classification markers for identifying the singer's vocal techniques. More importantly, it can keenly capture the singer's personalized treatment of the original song during live improvisation, such as deliberate changes in vibrato, impromptu key deviations, on-the-spot rhythmic expansion or contraction, or subtle changes in timbre caused by emotional injection. At the same time, it also includes the immediate feedback from the live audience, such as cheers, spontaneous participation in singing along, and sudden changes in atmosphere and bursts of energy triggered by live interaction. Through this feature extraction process, the original real-time audio is transformed into structured dynamic feature parameters that faithfully reflect the current performance, providing a driving basis for the subsequent lighting generation module to be truly synchronized with "this moment".

[0035] Step S13: Based on the static features of the song, generate global static atmosphere parameters. The global static atmosphere parameters are used to drive each logical device area to execute the basic lighting state that matches the current song segment. The logical device area includes at least one of the main display backlight area, keyboard area, mouse area, host light area and secondary screen ambient light area.

[0036] It's important to note that this step involves setting a unified basic lighting tone for all logical device areas based on the song's segment characteristics. First, the system determines the current segment based on the playback progress. Then, it extracts features such as the segment's type, emotional intensity level, and dominant color tone. Next, it matches the corresponding basic brightness range, lighting variation mode, and basic color temperature from a pre-defined lighting mode library. This basic lighting state determines whether the current segment's lighting is soft and tranquil or vibrant and energetic. Furthermore, the system smoothly transitions these parameters during segment changes to avoid abrupt lighting jumps. It's worth noting that the logical device area here is an abstraction and partitioning of the lighting of multiple physical peripherals in the user's online concert environment. It divides these into different functional areas such as the main monitor backlight area, keyboard area, mouse area, host computer backlight area, and secondary screen ambient lighting area, aiming to achieve a spatialized and hierarchical lighting layout.

[0037] Step S14: Based on the partition mapping rules of each logical device area, generate partition dynamic atmosphere parameters for each logical device area according to the dynamic characteristics of the song.

[0038] It's important to note that this step introduces differentiated zoned dynamic modulation on top of the overall basic lighting atmosphere. Since different peripheral areas occupy different positions in the user's field of vision and perform different visual functions, their appropriate dynamic characteristics for response also differ. Zone mapping rules define this differentiation. For example, the keyboard and mouse areas might primarily respond to beat intensity and timbre brightness to enhance the rhythmic feel of the tactile area; while the secondary screen ambient lighting area might be more sensitive to the distribution of the ambient spectrum and atmospheric bursts to create changes in ambient light. Based on these rules, this step extracts corresponding target feature parameters from the song's dynamic characteristics and maps them into dynamic brightness modulation, dynamic color shift, and dynamic effect trigger signals specific to each logical device area.

[0039] Step S15: For each logical device area, the partition dynamic atmosphere parameters of the logical device area are fused with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures.

[0040] It should be noted that this step is the key to generating the final instruction in this method, realizing the fusion of zone dynamic atmosphere parameters and global static atmosphere parameters. For each logical device zone, the system first obtains the basic brightness and basic color of that zone determined by the current song segment, and then superimposes the brightness modulation and color offset from the zone dynamic atmosphere parameters generated in the previous step, causing the light to produce subtle or significant fluctuations in the segment's base color. Then, based on the basic lighting change pattern, these fused brightness and color are time-varyingly modulated to form a rhythmic breathing or flowing effect. Finally, when special dynamic trigger signals such as atmosphere bursts are detected, corresponding instantaneous lighting effects, such as brightness pulses or color flashes, are superimposed on the above effects. Through this layer-by-layer parameter fusion and modulation, real-time control instructions that can directly drive physical lighting fixtures are finally generated, making the lighting in each peripheral zone both coordinated and unified, yet possessing dynamic layers, thus presenting a highly synchronized and immersive light and shadow experience.

[0041] In the above implementation, the real-time audio stream of the target online concert and the pre-extracted static features of the current song are obtained. The static features of the song include the segment features corresponding to each segment of the current song. Then, the real-time audio stream is subjected to feature extraction to obtain the dynamic features of the song. Then, based on the static features of the song, global static atmosphere parameters are generated. The global static atmosphere parameters are used to drive each logical device area to execute the basic lighting state matching the current song segment. The logical device area includes at least one of the main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area. Then, based on the partition mapping rules of each logical device area, the partition dynamic atmosphere parameters of each logical device area are generated according to the dynamic features of the song. Then, for each logical device area, the partition dynamic atmosphere parameters of the logical device area are fused with the global static atmosphere parameters to generate corresponding real-time lighting control instructions to control the corresponding physical lights. This invention overcomes the problems of existing technologies, such as single lighting modes and inability to respond to live dynamics, by integrating real-time audio streams with pre-extracted song segment features. Static atmosphere parameters drive smooth transitions between device zones with each segment, restoring emotional progression, while dynamic features capture improvisation and audience feedback in real time, achieving instant response. Through differentiated partitioning mapping rules, each peripheral area responds to beats, timbres, etc., according to its spatial function, constructing a layered three-dimensional light and shadow, integrating dynamic and static parameters, and superimposing brightness, color modulation, and special effects on a unified tone to achieve refined control that combines movement and stillness, thereby enhancing the immersive experience of online concerts.

[0042] As a further implementation of the method, the step of extracting features from the real-time audio stream to obtain the dynamic features of the song includes: Step S21: Perform frame segmentation and windowing processing on the real-time audio stream to obtain multiple audio frames.

[0043] Step S22: Perform sound source separation processing on the audio frames to obtain the singer's performance audio frames and the ambient sound audio frames.

[0044] Step S23: Extract features from the singer's audio frames to obtain dynamic features of the performance; and extract features from the ambient audio frames to obtain dynamic features of the atmosphere.

[0045] It is important to note that performance dynamics and ambient dynamics differ fundamentally in their acoustic sources, variation patterns, and lighting-related mapping targets. Specifically, performance dynamics are extracted from the singer's audio frames, primarily depicting the nuanced attributes directly related to the musical performance, such as pitch fluctuations, dynamic variations, timbre details, and improvisational techniques. These changes are continuous and possess clear musical semantics. Ambient dynamics, on the other hand, are extracted from ambient audio frames, capturing energy surges, spectral distributions, and timing of bursts from environmental sounds like cheers, choruses, and applause. These sounds are characterized by their suddenness, collective nature, and non-stationarity. The reason for separating and extracting these two features is that performance details and ambient noise can easily mask each other in mixed audio. Directly extracting overall features makes it difficult to accurately distinguish between the singer's improvisation and the audience's immediate reactions, potentially leading to blurred or false lighting responses. By separating the sound sources, performance dynamics can be precisely mapped to equipment areas closely related to musical expression, such as the main monitor's backlight area and the keyboard area, to enhance the visual presentation of performance details. Ambient dynamics, meanwhile, independently drive ambient lighting, such as the secondary screen's ambient lighting area, allowing it to sensitively capture the instantaneous bursts of atmosphere. This separation and extraction method allows different logic device areas to have their own focus and work together, which not only highlights the artistic expression of music, but also amplifies the immersive feeling of live interaction.

[0046] Step S24: Align the singing dynamic features and the ambient dynamic features on the timeline and stitch the features together to obtain the song dynamic features.

[0047] In the above implementation, in order to achieve feature extraction of the real-time audio stream, the real-time audio stream is segmented and windowed to obtain multiple audio frames. Then, the audio frames are processed by sound source separation to obtain singer performance audio frames and ambient sound frames. Then, features are extracted from the singer performance audio frames to obtain performance dynamic features; and features are extracted from the ambient sound audio frames to obtain ambient dynamic features. Then, the performance dynamic features and ambient dynamic features are aligned on the time axis and spliced ​​together to obtain song dynamic features.

[0048] As a further implementation of the method, the step of extracting features from the singer's audio frames to obtain dynamic features of the performance includes: Step S31: Perform a short-time Fourier transform on the singer's audio frame to obtain the first spectrogram.

[0049] Step S32: Perform multi-dimensional feature extraction on the first spectrogram to obtain the corresponding basic acoustic features, including the spectral centroid, fundamental frequency trajectory of human voice, harmonic energy ratio, spectral envelope features, and energy distribution features.

[0050] Step S33: Divide the frequency range corresponding to the first spectrogram into multiple sub-bands and extract the energy envelope of each sub-band; and perform start point detection on the energy envelope of each sub-band to obtain the start point intensity of each sub-band; and perform weighted fusion of the start point intensity based on the preset contribution weight of each sub-band to human ear rhythm perception to obtain the beat intensity sequence.

[0051] Step S34: Classify the singer's audio frames according to the spectral envelope characteristics and energy distribution characteristics to obtain singing technique classification labels, wherein the singing technique classification labels include at least one of true voice, falsetto, mixed voice and breathy voice.

[0052] Step S35: Normalize and weight the spectral centroid and the fundamental frequency trajectory of the human voice to obtain the timbre brightness characteristics.

[0053] Step S36: The beat intensity sequence, timbre brightness and darkness characteristics, harmonic energy ratio and singing technique classification labels are spliced ​​together to obtain the singing dynamic characteristics.

[0054] In the above implementation, to extract features from the singer's audio frame, a short-time Fourier transform is performed on the singer's audio frame to obtain a first spectrogram. Then, multi-dimensional feature extraction is performed on the first spectrogram to obtain the corresponding basic acoustic features. These basic acoustic features include the spectral centroid, the fundamental frequency trajectory of the human voice, the harmonic energy ratio, the spectral envelope features, and the energy distribution features. The frequency range corresponding to the first spectrogram is then divided into multiple sub-bands, and the energy envelope of each sub-band is extracted. Furthermore, the starting point of the energy envelope of each sub-band is detected to obtain the starting point intensity corresponding to each sub-band. Furthermore, based on the preset contribution weights of each sub-band to human ear rhythm perception, the starting point intensity is weighted and fused to obtain a beat intensity sequence. Then, according to the spectral envelope characteristics and energy distribution characteristics, the singer's audio frames are classified to obtain singing technique classification labels. The singing technique classification labels include at least one of true voice, falsetto, mixed voice, and breathy voice. Then, the spectral centroid and the fundamental frequency trajectory of the human voice are normalized and weighted to obtain timbre brightness characteristics. Finally, the beat intensity sequence, timbre brightness characteristics, harmonic energy ratio, and singing technique classification labels are spliced ​​together to obtain singing dynamic characteristics.

[0055] As a further implementation of the method, the step of extracting features from the ambient audio frames to obtain ambient dynamic features includes: Step S41: Perform a short-time Fourier transform on the ambient audio frame to obtain the second spectrogram.

[0056] Step S42: Based on multiple preset frequency band ranges, extract the frequency band energy values ​​corresponding to each preset frequency band from the second spectrum diagram, and calculate the total energy of the entire frequency band of the second spectrum diagram.

[0057] Step S43: Calculate the energy percentage of each preset frequency band based on the frequency band energy value and the total energy of the entire frequency band, and obtain the on-site spectrum energy distribution vector.

[0058] Step S44: Perform a sliding window smoothing filter on the total energy of the entire frequency band to obtain a smooth energy curve, and use the smooth energy curve as the field energy sequence.

[0059] Step S45: Based on the smooth energy curve, calculate the energy surge ratio of the smooth energy of the current audio frame relative to the average energy of the previous window. When the energy surge ratio exceeds a preset threshold, mark the current audio frame as an atmosphere burst point and determine the corresponding burst intensity level according to the energy surge ratio.

[0060] Step S46: The on-site spectral energy distribution vector, on-site energy sequence, atmosphere burst point and burst intensity level are spliced ​​together to obtain the dynamic characteristics of the atmosphere.

[0061] In the above implementation, in order to extract features from the ambient audio frame, a short-time Fourier transform is performed on the ambient audio frame to obtain a second spectrogram. Then, based on multiple preset frequency bands, the frequency band energy values ​​corresponding to each preset frequency band are extracted from the second spectrogram, and the total energy of the entire frequency band of the second spectrogram is calculated. Then, based on the frequency band energy values ​​and the total energy of the entire frequency band, the energy proportion of each preset frequency band is calculated to obtain the ambient spectrum energy distribution vector. Then, a sliding window smoothing filter is applied to the total energy of the entire frequency band to obtain a smooth energy curve, and the smooth energy curve is used as the ambient energy sequence. Then, based on the smooth energy curve, the energy surge ratio of the smooth energy of the current audio frame relative to the average energy of the previous window is calculated. When the energy surge ratio exceeds a preset threshold, the current audio frame is marked as an atmosphere burst point, and the corresponding burst intensity level is determined according to the energy surge ratio. Then, the ambient spectrum energy distribution vector, the ambient energy sequence, the atmosphere burst point, and the burst intensity level are concatenated to obtain the ambient dynamic features.

[0062] As a further implementation of the method, the step of generating global static atmosphere parameters based on the static features of the song includes: Step S51: Obtain the current playback timestamp of the real-time audio stream.

[0063] Step S52: Based on the current playback timestamp, find the start and end times of each song segment in the song's static features to determine the current target song segment.

[0064] Step S53: Extract the target segment features corresponding to the target song segment from the static features of the song. The target segment features include the target segment type, the target emotion intensity level, and the target main color tone.

[0065] Step S54: Based on the target paragraph type, target emotional intensity level, and target main color tone, query the corresponding basic brightness range, basic lighting change mode, and basic color temperature from the preset lighting mode library.

[0066] Step S55: Combine the basic brightness range, basic lighting change mode, and basic color temperature, and perform smooth interpolation within the time neighborhood of song segment transitions to generate continuously changing global static atmosphere parameters, so as to control the basic lighting state of each logical device area to smoothly transition with the progression of song segments.

[0067] It's important to note that the core idea of ​​this implementation is to transform the song's prior paragraph structure information into a smoothly evolving global lighting tone, enabling the lighting atmosphere to perceive the music's macro-narrative and ensure coherence in paragraph transitions. Traditional methods rely solely on real-time audio features, failing to predict the upcoming emotional paragraphs, resulting in delayed lighting changes or disconnects from paragraph transitions. This solution, however, precisely locates the current target paragraph within pre-extracted static features of the song using the current playback timestamp, acquiring high-level semantics such as the paragraph's type, emotional intensity, and dominant color tone. These are then mapped to preset base brightness, change patterns, and color temperatures, achieving a predictive response to musical paragraph changes and a forward-looking shift in the atmosphere. Furthermore, smooth interpolation of lighting parameters within the temporal neighborhood of paragraph transitions avoids abrupt changes, allowing the basic lighting state to transition smoothly with the progression of song paragraphs, forming a coherent, natural, and narrative-rich global atmosphere evolution. This provides a stable visual foundation for subsequent dynamic details.

[0068] In the above implementation, in order to generate global static atmosphere parameters, the current playback timestamp of the real-time audio stream is obtained. Then, based on the current playback timestamp, the start and end times of each song segment are found in the song's static features to determine the current target song segment. Then, the target segment features corresponding to the target song segment are extracted from the song's static features. The target segment features include the target segment type, the target emotional intensity level, and the target primary color. Then, based on the target segment type, the target emotional intensity level, and the target primary color, the corresponding basic brightness range, basic lighting change mode, and basic color temperature are queried from a preset lighting mode library. Then, the basic brightness range, basic lighting change mode, and basic color temperature are combined and smoothly interpolated within the time neighborhood of the song segment switching to generate continuously changing global static atmosphere parameters, so as to control the basic lighting state of each logical device area to smoothly transition with the progression of the song segment.

[0069] As a further implementation of the method, the step of generating partition dynamic atmosphere parameters for each logical device area based on the partition mapping rules of each logical device area and according to the dynamic characteristics of the song includes: Step S61: For each logical device area, obtain the target partition mapping table corresponding to that logical device area. The target partition mapping table includes feature parameter data and dynamic atmosphere parameter data. The target partition mapping table is used to represent the mapping relationship between feature parameter data and dynamic atmosphere parameter data.

[0070] Step S62: Based on the target partition mapping table, extract the corresponding target feature parameters from the song's dynamic features, and perform mapping transformation based on the target partition mapping table to generate partition dynamic atmosphere parameters for each logical device area. The partition dynamic atmosphere parameters include dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal.

[0071] It should be noted that the core idea of ​​this implementation method is to establish a differentiated dynamic feature mapping mechanism, enabling peripheral areas with different spatial functions to extract the dynamic features of the same song as needed and independently convert them into dynamic lighting parameters adapted to the characteristics of that area. By configuring a dedicated target partition mapping table for each logical device area, the dynamic response methods of each area are completely decoupled, so that the keyboard area, secondary screen ambient light area, etc., no longer execute the same dynamic effects, but each has its own set of exclusive "feature-parameter" translation rules, fundamentally solving the problem of the lack of hierarchy in the lighting layout. On this basis, the dynamic features of the song are extracted and independently mapped and converted on demand according to the partition mapping rules. For example, the near-field tactile area can focus on responding to the beat and intensity and convert it into highly sensitive dynamic brightness modulation, while the ambient light area can prioritize capturing the ambient energy and spectrum distribution and convert it into a wide range of dynamic color shifts. Finally, a complete set of differentiated control parameters, including dynamic brightness modulation, dynamic color shift, and dynamic effect trigger signals, is independently generated for each area, providing support for subsequent fine integration with the global basic atmosphere.

[0072] In the above implementation, in order to generate the partition dynamic atmosphere parameters of each logical device area, a target partition mapping table corresponding to each logical device area is obtained. The target partition mapping table includes feature parameter data and dynamic atmosphere parameter data. The target partition mapping table is used to represent the mapping relationship between the feature parameter data and the dynamic atmosphere parameter data. Then, based on the target partition mapping table, the corresponding target feature parameters are extracted from the song's dynamic features, and a mapping transformation is performed based on the target partition mapping table to generate the partition dynamic atmosphere parameters of each logical device area. The partition dynamic atmosphere parameters include dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal.

[0073] As a further implementation of the method, for each logical device area, the step of fusing the partition dynamic atmosphere parameters of that logical device area with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures includes: Step S71: For each logical device area, obtain the basic brightness range, basic lighting change mode and basic color temperature from the global static atmosphere parameters corresponding to that logical device area, and determine the current basic brightness within the basic brightness range based on the current timestamp.

[0074] Step S72: Obtain the dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal of the logical device area in the partition dynamic atmosphere parameters.

[0075] Step S73: The current base brightness and the dynamic brightness modulation amount are superimposed to obtain the first target brightness.

[0076] Step S74: Shift the base color temperature according to the dynamic color shift amount to obtain the first target color.

[0077] Step S75: Based on the basic light change pattern, perform time-varying modulation on the brightness and color of the first target to obtain the brightness and color of the second target.

[0078] Step S76: When there is an atmosphere burst point marker in the dynamic effect trigger signal, instantaneous lighting effects are superimposed on the second target brightness and the second target color according to the burst intensity level corresponding to the atmosphere burst point marker, and a lighting control command is generated; otherwise, a lighting control command is generated according to the second target brightness and the second target color.

[0079] It should be noted that the core idea of ​​this implementation method is to adopt a fusion architecture of layered overlay and time-varying modulation, organically integrating the basic lighting tone determined by the global static atmosphere parameters with the real-time details carried by the zone dynamic atmosphere parameters. First, the current basic brightness, basic lighting change mode, and basic color temperature determined by the current song segment are acquired, as well as the dynamic brightness modulation amount, dynamic color shift amount, and dynamic effect trigger signal dynamically mapped from the real-time audio. Then, the first fusion is performed on the two dimensions of brightness and color, that is, the dynamic modulation amount and color shift are directly superimposed on the basic values ​​to form the initial first target brightness and first target color. Next, using the basic lighting change mode as the rule of time-varying rhythm, continuous dynamic modulation is applied to the first target brightness and color to generate a second target brightness and second target color with a breathing or flowing feel. Finally, when an atmosphere burst point marker is detected, instantaneous lighting effects such as brightness pulses, strobe, or color flashing are superimposed on the existing results according to its burst intensity level. If there is no burst point, the second target brightness and color are directly output as the final instruction. The entire process follows a progressive logic of "keynote overlay and dynamic offset → time-varying rhythm modulation → transient effect enhancement", which ensures that the final generated lighting control commands are both faithful to the macro atmosphere of the song segment and can sensitively respond to the real-time changes in the details of the performance and the atmosphere of the scene, achieving a deep integration and harmonious unity between static preparation and dynamic details.

[0080] In the above implementation, in order to control the physical lighting fixtures, for each logical device area, the basic brightness range, basic lighting change mode, and basic color temperature in the global static atmosphere parameters corresponding to that logical device area are obtained, and the current basic brightness is determined within the basic brightness range based on the current timestamp. Then, the dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal in the partition dynamic atmosphere parameters of that logical device area are obtained. Then, the current basic brightness and the dynamic brightness modulation amount are superimposed to obtain the first target brightness. Then, the basic color temperature is offset according to the dynamic color offset to obtain the first target color. Then, based on the basic lighting change mode, the first target brightness and the first target color are time-varyingly modulated to obtain the second target brightness and the second target color. Then, when there is an atmosphere burst point marker in the dynamic effect trigger signal, the instantaneous lighting effect is superimposed on the second target brightness and the second target color according to the burst intensity level corresponding to the atmosphere burst point marker to generate a lighting control command. Otherwise, a lighting control command is generated according to the second target brightness and the second target color.

[0081] As a further implementation of the method, the step of generating a lighting control command by superimposing a momentary lighting effect on the second target brightness and the second target color based on the burst intensity level corresponding to the atmosphere burst point marker includes: Step S81: Obtain the burst intensity level corresponding to the atmosphere burst point marker.

[0082] Step S82: Based on the burst intensity level, query the corresponding instantaneous effect type and effect parameters from the preset instantaneous effect library. The instantaneous effect type includes at least one of brightness pulse, strobe, and color flash, and the effect parameters include effect duration and effect intensity value.

[0083] Step S83: Based on the instantaneous effect type and effect parameters, modulate the brightness and color of the second target, and generate a light control command based on the modulated brightness and color.

[0084] In the above implementation, in order to generate a lighting control command, the burst intensity level corresponding to the atmosphere burst point marker is obtained. Then, based on the burst intensity level, the corresponding instantaneous effect type and effect parameters are queried from a preset instantaneous effect library. The instantaneous effect type includes at least one of brightness pulse, strobe, and color flash. The effect parameters include effect duration and effect intensity value. Then, based on the instantaneous effect type and effect parameters, the brightness of the second target and the color of the second target are modulated, and the lighting control command is generated based on the modulated brightness and color.

[0085] This application also discloses a lighting linkage system based on feature recognition.

[0086] refer to Figure 2 A feature-based lighting linkage system includes: The data acquisition module is used to acquire the real-time audio stream of the target online concert and the pre-extracted static features of the current song. The static features of the song include the segment features corresponding to each segment of the current song. The dynamic feature extraction module is used to extract features from the real-time audio stream to obtain the dynamic features of the song; The global static atmosphere parameter generation module is used to generate global static atmosphere parameters based on the static features of the song. The global static atmosphere parameters are used to drive each logical device area to execute the basic lighting state that matches the current song segment. The logical device area includes at least one of the following: main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area. The partition dynamic atmosphere parameter generation module is used to generate partition dynamic atmosphere parameters for each logical device area based on the partition mapping rules of each logical device area and the dynamic characteristics of the song. The fusion control module is used to fuse the dynamic atmosphere parameters of each logical device area with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures.

[0087] The feature recognition-based lighting linkage system of the present invention can implement any of the feature recognition-based lighting linkage methods, and the specific working process of the feature recognition-based lighting linkage system of the present invention can refer to the corresponding process in the above-mentioned feature recognition-based lighting linkage method.

[0088] This application also discloses a computer device.

[0089] refer to Figure 3 A computer device includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement any of the above-described methods of light linkage based on feature recognition.

[0090] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A lighting linkage method based on feature recognition, characterized in that, include: Acquire the real-time audio stream of the target online concert and the pre-extracted static features of the current song, wherein the static features of the song include the segment features corresponding to each segment of the current song. Feature extraction is performed on the real-time audio stream to obtain the song's dynamic features; Based on the static features of the song, global static atmosphere parameters are generated. The global static atmosphere parameters are used to drive each logical device area to execute a basic lighting state that matches the current song segment. The logical device area includes at least one of the following: main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area. Based on the partition mapping rules of each logical device area, and according to the dynamic characteristics of the song, partition dynamic atmosphere parameters of each logical device area are generated. For each of the aforementioned logical device areas, the partition dynamic atmosphere parameters of that logical device area are fused with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures.

2. The light linkage method based on feature recognition according to claim 1, characterized in that, The step of extracting features from the real-time audio stream to obtain the dynamic features of the song includes: The real-time audio stream is segmented and windowed to obtain multiple audio frames; The audio frames are subjected to sound source separation processing to obtain the singer's performance audio frames and the live atmosphere audio frames; Feature extraction is performed on the singer's performance audio frames to obtain performance dynamic features; and feature extraction is performed on the ambient sound audio frames to obtain ambient dynamic features. Align the singing dynamic features and the atmospheric dynamic features on the time axis and then stitch the features together to obtain the song dynamic features.

3. The light linkage method based on feature recognition according to claim 2, characterized in that, The step of extracting features from the singer's audio frames to obtain dynamic features of the performance includes: A short-time Fourier transform is performed on the audio frames of the singer's performance to obtain the first spectrogram; Multidimensional feature extraction is performed on the first spectrogram to obtain the corresponding basic acoustic features, wherein the basic acoustic features include the spectral centroid, the fundamental frequency trajectory of the human voice, the harmonic energy ratio, the spectral envelope features, and the energy distribution features; The frequency range corresponding to the first spectrogram is divided into multiple sub-bands, and the energy envelope of each sub-band is extracted; and the starting point is detected for the energy envelope of each sub-band to obtain the starting point intensity of each sub-band; and the starting point intensity is weighted and fused based on the preset contribution weight of each sub-band to human ear rhythm perception to obtain the beat intensity sequence. Based on the spectral envelope characteristics and the energy distribution characteristics, the singer's audio frames are classified to obtain a singing technique classification identifier, wherein the singing technique classification identifier includes at least one of true voice, falsetto, mixed voice and breathy voice; The spectral centroid and the fundamental frequency trajectory of the human voice are normalized and weighted to obtain the timbre brightness characteristics; The dynamic characteristics of the performance are obtained by splicing together the beat intensity sequence, the timbre brightness and darkness characteristics, the harmonic energy ratio and the singing technique classification identifier.

4. The light linkage method based on feature recognition according to claim 2, characterized in that, The step of extracting features from the ambient audio frames to obtain dynamic ambient features includes: A short-time Fourier transform is performed on the ambient audio frame to obtain a second spectrogram; Based on multiple preset frequency band ranges, the frequency band energy values ​​corresponding to each preset frequency band are extracted from the second spectrum map, and the total energy of the entire frequency band of the second spectrum map is calculated. Based on the frequency band energy value and the total energy of the entire frequency band, the energy proportion of each preset frequency band is calculated to obtain the on-site spectrum energy distribution vector; The total energy across the entire frequency band is smoothed by a sliding window to obtain a smooth energy curve, which is then used as the field energy sequence. Based on the smooth energy curve, the energy surge ratio of the smooth energy of the current audio frame relative to the average energy of the previous window is calculated. When the energy surge ratio exceeds a preset threshold, the current audio frame is marked as an atmosphere burst point, and the corresponding burst intensity level is determined according to the energy surge ratio. The atmospheric dynamic characteristics are obtained by splicing together the on-site spectral energy distribution vector, the on-site energy sequence, the atmosphere burst point, and the burst intensity level.

5. The light linkage method based on feature recognition according to claim 1, characterized in that, The above is based on the static features of the song. The steps for generating global static atmosphere parameters include: Obtain the current playback timestamp of the real-time audio stream; Based on the current playback timestamp, the start and end times of each song segment are found in the static features of the song to determine the target song segment currently in use. Extract the target segment features corresponding to the target song segment from the static features of the song, wherein the target segment features include the target segment type, the target emotion intensity level, and the target main color tone; Based on the target paragraph type, the target emotion intensity level, and the target main color tone, the corresponding basic brightness range, basic light change mode, and basic color temperature are queried from the preset lighting mode library; The basic brightness range, the basic lighting change mode, and the basic color temperature are combined and smoothly interpolated within the time neighborhood of song segment transitions to generate continuously changing global static atmosphere parameters, so as to control the basic lighting state of each logical device area to smoothly transition with the progression of song segments.

6. The light linkage method based on feature recognition according to claim 5, characterized in that, The step of generating partition dynamic atmosphere parameters for each logical device area based on the partition mapping rules of each logical device area and according to the dynamic characteristics of the song includes: For each logical device area, obtain the target partition mapping table corresponding to that logical device area. The target partition mapping table includes feature parameter data and dynamic atmosphere parameter data. The target partition mapping table is used to characterize the mapping relationship between the feature parameter data and the dynamic atmosphere parameter data. Based on the target partition mapping table, corresponding target feature parameters are extracted from the song's dynamic features, and a mapping transformation is performed based on the target partition mapping table to generate partition dynamic atmosphere parameters for each logical device area. The partition dynamic atmosphere parameters include dynamic brightness modulation amount, dynamic color offset amount, and dynamic effect trigger signal.

7. The light linkage method based on feature recognition according to claim 6, characterized in that, The step of fusing the partition dynamic atmosphere parameters of each logical device area with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures includes: For each of the logical device areas, the basic brightness range, the basic lighting change mode, and the basic color temperature in the global static atmosphere parameters corresponding to that logical device area are obtained, and the current basic brightness is determined within the basic brightness range based on the current timestamp. The dynamic brightness modulation amount, the dynamic color offset amount, and the dynamic effect trigger signal of the logical device area in the dynamic atmosphere parameters of the partition are obtained; The current base brightness is superimposed with the dynamic brightness modulation amount to obtain the first target brightness; The base color temperature is shifted according to the dynamic color shift to obtain the first target color; Based on the basic lighting change pattern, the brightness and color of the first target are time-varyingly modulated to obtain the brightness and color of the second target. When an atmosphere burst point marker is present in the dynamic effect trigger signal, a momentary lighting effect is superimposed on the second target brightness and the second target color based on the burst intensity level corresponding to the atmosphere burst point marker, and a lighting control command is generated; otherwise, a lighting control command is generated based on the second target brightness and the second target color.

8. The light linkage method based on feature recognition according to claim 7, characterized in that, The step of generating a lighting control command by superimposing a momentary lighting effect on the second target brightness and the second target color based on the burst intensity level corresponding to the atmospheric burst point marker includes: Obtain the burst intensity level corresponding to the atmospheric burst point marker; Based on the burst intensity level, the corresponding instantaneous effect type and effect parameters are queried from the preset instantaneous effect library. The instantaneous effect type includes at least one of brightness pulse, strobe, and color flash, and the effect parameters include effect duration and effect intensity value. Based on the instantaneous effect type and the effect parameters, the brightness and color of the second target are modulated, and a lighting control command is generated according to the modulated brightness and color.

9. A lighting linkage system based on feature recognition, characterized in that, include: The data acquisition module is used to acquire the real-time audio stream of the target online concert and the pre-extracted static features of the current song, wherein the static features of the song include the segment features corresponding to each segment of the current song. The dynamic feature extraction module is used to extract features from the real-time audio stream to obtain the dynamic features of the song; A global static atmosphere parameter generation module is used to generate global static atmosphere parameters based on the static features of a song. The global static atmosphere parameters are used to drive each logical device area to execute a basic lighting state that matches the current song segment. The logical device area includes at least one of the following: main display backlight area, keyboard area, mouse area, host light area, and secondary screen ambient light area. The partition dynamic atmosphere parameter generation module is used to generate partition dynamic atmosphere parameters for each of the logical device zones based on the partition mapping rules of each of the logical device zones and according to the dynamic characteristics of the song. The fusion control module is used to fuse the partition dynamic atmosphere parameters of each logical device area with the global static atmosphere parameters to generate corresponding real-time lighting control commands to control the corresponding physical lighting fixtures.

10. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method of any one of claims 1 to 8.

Images