Display screen connection method and display screen connection system

Abstract

The invention is suitable for the technical field of display equipment, and provides a display screen connection method and a display screen connection system, and the method comprises the steps that a multi-modal recognition module receives multi-modal data, sent by user terminal equipment, of a display screen; the multi-modal data comprises position information of each box body of the display screen; inputting the multi-modal data into a computer vision model for multi-modal identification to obtain feature data; determining target mapping information based on the feature data; the target mapping information comprises a port number, an address serial number, a position coordinate, a row and column number, a direction and a connection relation of each box body of the display screen; the target mapping information is sent to the user terminal, after the user terminal detects confirmation information that a user confirms the target mapping information, the target mapping information is sent to the display screen system, and the display screen system executes screen connection operation on a display screen based on the target mapping information. Through the multi-modal data, the limitation that only picture recognition is supported, the application range is small and the function is single in the prior art is solved.

Description

Technical Field

[0001] This application belongs to the field of display device technology, and in particular relates to a display screen connection method and display screen connection system. Background Technology

[0002] In practical applications, light-emitting diodes (LEDs) and LED displays are widely used in scenarios such as conference all-in-one machines, advertising screens, and command and dispatch screens. Since LED screens are usually composed of multiple cabinet modules, after initial installation or replacement, a screen connection operation must be performed to determine the correct position and row and column order of each cabinet in the entire screen so that the control system can correctly map and display the content.

[0003] Current technology typically involves using a user terminal to photograph the number displayed on an LED screen, then performing image recognition on the photographed image to determine the location of the cabinet, thereby completing the automatic screen connection. It is evident that existing technology suffers from limitations due to its limited support for image recognition, narrow applicability, and single-function capability. Summary of the Invention

[0004] This application provides a display screen connection method and display screen connection system. Existing technologies only support image recognition, which has the problems of limited applicability and single function.

[0005] In a first aspect, embodiments of this application provide a method for connecting multiple displays, applied to a multimodal recognition module, the method comprising: Receive multimodal data of the display screen sent by the user terminal device; the multimodal data includes the position information of each cabinet of the display screen; The multimodal data is input into a computer vision model for multimodal recognition to obtain feature data; Target mapping information is determined based on the feature data; the target mapping information includes the port number, address number, position coordinates, row and column number, direction, and connection relationship of each cabinet of the display screen; The target mapping information is sent to the user terminal so that the user of the user terminal can confirm the target mapping information. After the user terminal detects the confirmation information, it sends the target mapping information to the display screen system so that the display screen system can perform a screen connection operation on the display screen based on the target mapping information.

[0006] The beneficial effects of the embodiments in this application compared with the prior art are: This application provides a method for connecting multiple displays. A multimodal recognition module receives multimodal data from a user terminal device, including the position information of each cabinet of the display. The multimodal data is input into a computer vision model for multimodal recognition to obtain feature data. Target mapping information is determined based on the feature data. The target mapping information includes the port number, address number, position coordinates, row and column number, direction, and connection relationship of each cabinet of the display. The target mapping information is sent to the user terminal for confirmation. After detecting the confirmation, the user terminal sends the target mapping information to the display system, enabling the display system to perform a screen connection operation based on the target mapping information. This application overcomes the limitations of existing technologies, which only support image recognition, resulting in limited applicability and single functionality. Furthermore, multimodal data can support irregular / rotated / mixed-layout scenarios (such as vertical screen matrices and curved splicing). Subsequently, multimodal data is processed using a computer vision model, which can automatically adapt to the orientation and positional offset of the enclosure, i.e., automatically identify the enclosure ports, coordinates, number of rows and columns, orientation, and connection relationships, avoiding manual input errors and improving processing efficiency and mapping accuracy. Next, this application generates standardized target mapping information based on feature data, reducing issues such as screen tearing, signal interruption, and synchronization delay caused by inconsistent configuration parameters, ensuring stability during long-term continuous operation. Finally, a user confirmation step is introduced, forming a closed loop from model recognition to manual verification and then to system execution, which can proactively avoid mapping errors and reduce on-site rework rates.

[0007] In one embodiment of the first aspect, determining the target mapping information based on the feature data includes: If there is historical data in the database that has a similarity greater than a set similarity with the feature data, then the historical mapping information corresponding to the historical data is obtained, and the historical mapping information is determined as the target mapping information; If the historical data is not found in the database, the feature data is input into the visual language big data model for processing to obtain the target mapping information.

[0008] In the above implementation, historical data with similarity exceeding a threshold in the database is prioritized for matching, directly reusing mature historical mapping information without requiring new calculations from the visual language large model. This significantly improves mapping efficiency for recurring deployment scenarios (such as regular equipment maintenance in the same showroom or batch splicing of the same model of cabinets). When no matching historical data is available, new feature data can be specifically processed through the visual language large model. Utilizing the model's multimodal understanding capabilities, parameters for special scenarios such as new cabinets and irregular splicing are accurately analyzed, ensuring full coverage and adaptability of the solution for unknown scenarios. Therefore, this implementation achieves a balance between efficiency and adaptability: for conventional scenarios, the historical matching path is used, balancing speed and stability; for innovative scenarios, the large model calculation path is used, ensuring the compatibility and scalability of the solution, adapting to the mixed splicing needs of display cabinets of different brands and specifications.

[0009] In one embodiment of the first aspect, the step of inputting the feature data into a large visual language model for processing to obtain the target mapping information includes: The visual language big model performs correction processing on the feature data to obtain the data to be processed. During the process of the visual language big model recognizing the data to be processed, if the position information of the first box in the data to be processed cannot be recognized, the predicted position information of the first box is generated based on the position information of the other boxes. The visual language big model outputs the target mapping information based on the position information of the other boxes and the predicted position information of the first box.

[0010] In the above implementation, the feature data is first corrected, which effectively filters noise caused by equipment errors and environmental interference during multimodal data acquisition, avoids mapping errors caused by defects in the original data, and improves the recognition accuracy of core parameters such as box position, number of rows and columns, and connection relationship. For extreme scenarios where the position information of the first box in the data to be processed cannot be identified, prediction and completion are performed based on the position information of the remaining boxes, overcoming the limitation of relying on complete data and enabling the complete generation of mapping information even with missing local data. Therefore, this implementation can avoid process interruptions caused by missing local data, eliminate the need for repeated data re-acquisition, shorten the mapping information generation cycle, and improve the overall smoothness of the multi-screen operation.

[0011] In one embodiment of the first aspect, after the visual language large model corrects the feature data to obtain the data to be processed, the method further includes: The visual language big model identifies the data to be processed and obtains the initial mapping information corresponding to the data to be processed. When the visual language model detects that the data to be processed is incomplete, it generates predicted position information of the third box based on the position information of the second box present in the data to be processed; the incomplete data is used to describe that the data to be processed is not a complete picture of the display screen; the third box refers to the box that does not exist in the data to be processed. The visual language big model performs a completion operation on the initial mapping information based on the predicted position information of the third box, so as to output the target mapping information.

[0012] In the above implementation, initial mapping information is first generated based on existing data, and then missing cabinets are filled in. This ensures the accuracy of existing cabinet mapping parameters and achieves full coverage of overall mapping information, avoiding local mapping problems caused by missing data for some cabinets. It also ensures that there are no blind spots in the splicing of the screen and no signal interruptions after screen networking. As can be seen, this implementation is suitable for scenarios where the data to be processed is an incomplete image (such as when the front-end device has a limited field of view during acquisition or when temporary screen networking lacks some cabinet data). The model can automatically detect data integrity and trigger the completion process, eliminating the need for manual data re-acquisition and solving the pain point of process interruption due to incomplete data in traditional solutions.

[0013] In one embodiment of the first aspect, after the visual language large model corrects the feature data to obtain the data to be processed, the method further includes: When the visual language big model detects that the multi-model data is a video type, it sequentially identifies each frame of video data in the data to be processed to obtain the position information corresponding to each frame of video data. The visual language big model combines and processes the various location information to output the target mapping information.

[0014] In the above embodiments, frame-by-frame recognition logic is designed for multimodal video data, breaking through the limitations of acquiring single image data. It supports the acquisition of the display cabinet's position information through dynamic video, eliminating the need to convert the video into a static image and reducing the complexity and computational consumption of data preprocessing. Furthermore, each frame of the video data is recognized separately to obtain multiple sets of independent cabinet position information, which are then combined to generate the final mapping parameters, effectively filtering out recognition errors caused by sudden changes in light, lens shake, or partial occlusion in a single frame image.

[0015] In one embodiment of the first aspect, the process of combining the location information includes: The location information is concatenated to obtain a two-dimensional matrix structure. If different location information corresponding to different target video data is detected to be the same location information of the same box, then the blurring degree of the target video data in each frame is detected. The location information corresponding to the target video data with the lowest degree of image blur is determined as the target location information of the same box. The target location information is used to update the two-dimensional matrix structure information to obtain the target mapping information.

[0016] In the above implementation, the positional information from multiple frames is first stitched together into a two-dimensional matrix structure to construct a global topological framework for the display cabinet positions. This ensures that the positions, rows, columns, and connections of all cabinets form a complete logical association, avoiding isolated errors from scattered data. Then, for different positional information of the same cabinet appearing in multiple video frames, the optimal value is determined through image blur filtering. The positional data corresponding to the frame with the highest clarity is prioritized, effectively eliminating recognition deviations caused by lens shake, light occlusion, and motion blur, thereby reducing errors in the positional coordinates of individual cabinets. Finally, the two-dimensional matrix structure information updated based on the optimal positional information can highly match the actual topology of the physical splicing of the display screens, providing stable and reliable parameter support for subsequent screen-to-screen operations and ensuring the consistency and synchronization of the image during long-term operation.

[0017] Secondly, embodiments of this application provide a method for connecting multiple displays, applied to a user terminal, the method comprising: When a user's data collection command is detected, multimodal data of the display screen is acquired; the multimodal data includes the position information of each cabinet of the display screen; The multimodal data is sent to the multimodal recognition module; The system receives target mapping information sent by the multimodal recognition module and outputs the target mapping information; the target mapping information is obtained by the multimodal recognition module processing the multimodal data. After receiving the user's confirmation, the target mapping information is sent to the display screen system so that the display screen system can perform a screen connection operation on the display screen based on the target mapping information.

[0018] The beneficial effects of the embodiments in this application compared with the prior art are: This application provides a display screen connection method. When a user terminal detects a user's data collection command, it acquires multimodal data of the display screen. The multimodal data includes the position information of each cabinet of the display screen. The multimodal data is sent to a multimodal recognition module. The user terminal receives target mapping information sent by the multimodal recognition module and outputs the target mapping information. The target mapping information is obtained by the multimodal recognition module based on the multimodal data. After receiving confirmation information from the user, the target mapping information is sent to the display screen system so that the display screen system can perform a screen connection operation based on the target mapping information. This application's terminal directly responds to the user's collection command to acquire multimodal data, eliminating the need for additional professional collection equipment. Ordinary smartphones, tablets, or computer cameras can complete the cabinet position information collection, significantly reducing hardware investment costs. Simultaneously, the terminal is only responsible for data collection and command forwarding, while the core multimodal recognition and mapping information generation, computationally intensive operations, are handled by the multimodal recognition module. This avoids consuming local terminal resources, ensuring smooth operation of the terminal during the screen connection process and not affecting the use of other functions. The target mapping information is then uniformly generated and distributed by the multimodal recognition module, ensuring the standardization and accuracy of mapping parameters. This avoids parameter deviations caused by insufficient computing power or algorithm differences on the terminal side, improving the consistency of the three-level collaboration between the terminal, cloud, and screen. Afterward, the terminal outputs the target mapping information for user confirmation. Users can intuitively verify whether core parameters such as cabinet ports, location coordinates, and connection relationships match actual needs, proactively avoiding screen-to-screen failures caused by multimodal recognition module errors or data transmission mistakes, reducing on-site rework rates. Finally, only after user confirmation does the terminal distribute the mapping information to the display screen system, forming a user authorization-command execution control mechanism. This prevents illegal commands or misoperations from directly controlling the display screen, avoiding risks such as image splicing errors and signal interruptions, and improving operational security.

[0019] In one embodiment of the second aspect, acquiring the multimodal data of the display screen includes: The data acquisition method is determined based on the data acquisition command; the data acquisition method includes photo capture and video capture. Based on the data acquisition method, a data acquisition operation is performed on the display screen to obtain initial data; The initial data is augmented to obtain the multimodal data.

[0020] The above implementation supports both photo and video capture methods, allowing users to flexibly choose according to the actual scenario: static splicing screens can use photo capture to quickly acquire single-frame high-definition data; splicing screens in dynamic environments can use video capture to acquire multiple frames of data, avoiding the accidental interference of single-frame capture. Subsequently, data augmentation processing is performed on the initial captured data, effectively correcting initial data defects caused by insufficient light, lens shake, or obstruction, and improving the clarity and recognizability of the cabinet position information. Simultaneously, data augmentation processing gives the captured data stronger anti-interference capabilities, preserving core feature information such as cabinet position and ports even in complex environments, preventing interruptions in the splicing process due to data quality issues.

[0021] Thirdly, embodiments of this application provide a method for connecting multiple displays, applied to a display system, the method comprising: After the display screen is powered on, control the display screen to output the position information of each cabinet; The system receives target mapping information sent by a user terminal. The target mapping information is obtained by the user terminal sending multimodal data containing the position information of each box to a multimodal recognition module, which is obtained by the user terminal through data acquisition of the display screen. The multimodal recognition module processes the multimodal data. Control parameters are generated based on the target mapping information; The display screen is controlled based on the control parameters to achieve screen-to-screen operation.

[0022] The beneficial effects of the embodiments in this application compared with the prior art are: This application provides a display screen linkage method. After the display screen is powered on, the display screen system controls the display screen to output the position information of each cabinet; receives target mapping information sent by the user terminal; the target mapping information is obtained by the user terminal sending multimodal data containing the position information of each cabinet, obtained by data acquisition of the display screen, to a multimodal recognition module, which processes the multimodal data; generates control parameters based on the target mapping information; and controls the display screen to display based on the control parameters to realize the screen linkage operation. The display screen system of this application automatically outputs the position information of each cabinet after the display screen is powered on, eliminating the need for manual annotation or configuration of positioning parameters. This provides a standardized and high-precision raw data source for the user terminal's multimodal data acquisition, avoiding mapping deviations caused by manual annotation errors. After receiving the target mapping information, the display screen system can automatically complete the generation of control parameters and the execution of the screen linkage operation, eliminating the need for technicians to debug drivers and calibrate screen parameters on-site, achieving automated screen linkage that is executed immediately upon receipt. Specifically, the display screen system executes the screen linkage operation based on standardized control parameters, avoiding the risks of hardware overload and signal conflicts that may be caused by manual debugging, ensuring the long-term stable operation of the display screen system and reducing the hardware failure rate. Furthermore, the system only executes the screen-to-screen operation after receiving target mapping information from the user terminal, forming a four-level control closed loop of multimodal recognition module generation, user confirmation, terminal distribution, and system execution. This prevents unauthorized commands from tampering with display parameters and improves operational security. In addition, the control parameters are generated entirely based on the standardized target mapping information (including core parameters such as cabinet ports, coordinates, number of rows and columns, and connection relationships) generated by the multimodal recognition module. This avoids the random errors of manually adjusting parameters box by box in traditional solutions, improving the splicing alignment accuracy between cabinets to the millimeter level and completely solving problems such as screen tearing, edge misalignment, and signal asynchrony.

[0023] Fourthly, embodiments of this application provide a display screen connection device for use in a multimodal recognition module, the device comprising: The receiving unit is used to receive multimodal data of the display screen sent by the user terminal device; the multimodal data includes the position information of each cabinet of the display screen; The first input unit is used to input the multimodal data into a computer vision model for multimodal recognition to obtain feature data; The first information determination unit is used to determine target mapping information based on the feature data; the target mapping information includes the port number, address number, position coordinates, row and column number, direction, and connection relationship of each cabinet of the display screen; A sending unit is configured to send the target mapping information to the user terminal so that the user of the user terminal can confirm the target mapping information. After the user terminal detects the confirmation information, it sends the target mapping information to the display screen system so that the display screen system can perform a screen connection operation on the display screen based on the target mapping information.

[0024] Fifthly, embodiments of this application provide a display screen connection device applied to a user terminal, the device comprising: The acquisition unit is used to acquire multimodal data of the display screen when a user's data acquisition command is detected; the multimodal data includes the position information of each cabinet of the display screen; The first sending unit is used to send the multimodal data to the multimodal recognition module; The receiving unit is configured to receive target mapping information sent by the multimodal recognition module and output the target mapping information; the target mapping information is obtained by the multimodal recognition module through processing based on the multimodal data. The second sending unit is used to send the target mapping information to the display screen system after receiving the user's confirmation information, so that the display screen system can perform a screen connection operation on the display screen based on the target mapping information.

[0025] Sixthly, embodiments of this application provide a display screen interconnection device, applied to a display screen system, the device comprising: The output unit is used to control the display screen to output the position information of each cabinet after the display screen is powered on; A receiving unit is used to receive target mapping information sent by a user terminal; the target mapping information is obtained by the user terminal sending multimodal data containing the position information of each box obtained by data acquisition of the display screen to a multimodal recognition module, and the multimodal recognition module processes the multimodal data. The generation unit is used to generate control parameters based on the target mapping information; The display unit is used to control the display screen to display based on the control parameters, so as to realize the screen connection operation.

[0026] In a seventh aspect, embodiments of this application provide a multimodal recognition module, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the display screen linkage method as described in any one of the first aspects above.

[0027] Eighthly, embodiments of this application provide a user terminal, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the display screen linkage method as described in any one of the second aspects above.

[0028] Ninthly, embodiments of this application provide a display screen system, including: a display screen, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the display screen linkage method as described in any of the third aspects above.

[0029] In a tenth aspect, embodiments of this application provide a display screen interconnection system, including a multimodal recognition module, a user terminal, and a display screen system; The multimodal recognition module is connected to the user terminal and is used to execute the display screen connection method as described in any of the first aspects; The user terminal is connected to the display screen system and is used to execute the display screen connection method as described in any of the second aspects. The display system is used to perform the display screen interconnection method as described in the third aspect.

[0030] Eleventhly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor of the seventh aspect, implements the display screen linkage method as described in any one of the first aspects; when executed by a processor of the eighth aspect, implements the display screen linkage method as described in any one of the second aspects; and when executed by a processor of the ninth aspect, implements the display screen linkage method as described in any one of the third aspects.

[0031] In a twelfth aspect, embodiments of this application provide a computer program product that, when running on a multimodal recognition module, enables the multimodal recognition module to execute the display screen linking method described in any one of the first aspects; when running on a user terminal, enables the user terminal to execute the display screen linking method described in any one of the second aspects; and when running on a display system, enables the display system to execute the display screen linking method described in any one of the third aspects. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of the structure of a display screen interconnection system provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the implementation of a screen-to-screen method according to an embodiment of this application; Figure 3 This is a schematic diagram of multimodal data provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the specific implementation of step S106 in a display screen interconnection method provided in an embodiment of this application; Figure 5 This is a flowchart illustrating the specific implementation of step S202 in a display screen interconnection method provided in an embodiment of this application; Figure 6 This is a flowchart illustrating the implementation of a display screen interconnection method provided in another embodiment of this application; Figure 7 This is a flowchart illustrating the overall implementation of a display screen interconnection method provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a multimodal recognition module provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a user terminal provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a display screen system provided in an embodiment of this application. Detailed Implementation

[0034] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0035] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0036] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0037] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0038] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0039] In practical applications, light-emitting diodes (LEDs) and LED displays are widely used in scenarios such as conference all-in-one machines, advertising screens, and command and dispatch screens. Since LED screens are usually composed of multiple cabinet modules, after initial installation or replacement, a screen connection operation must be performed to determine the correct position and row and column order of each cabinet in the entire screen so that the control system can correctly map and display the content.

[0040] Current technology typically involves using a user terminal to photograph the number displayed on an LED screen, then performing image recognition on the photographed image to determine the location of the cabinet, thereby completing the automatic screen connection. It is evident that existing technology suffers from limitations due to its limited support for image recognition, narrow applicability, and single-function capability.

[0041] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of a display screen interconnection system provided in one embodiment of this application. Figure 1 As shown, the display screen interconnection system 1 includes: a display screen system 10, a user terminal 20, and a multimodal recognition module 30. The user terminal 20 is communicatively connected to both the display screen system 10 and the multimodal recognition module 30.

[0042] It should be noted that the above communication connection method can be a wireless communication connection.

[0043] In this embodiment, the display screen system 10 may include a display screen control system and a display screen. The display screen control system may include a sending card, a receiving card, and other hardware devices for processing and transmitting display content.

[0044] User terminal 20 includes, but is not limited to, devices such as smartphones, tablets, laptops, desktop computers, and computers.

[0045] The multimodal recognition module 30 may include a local recognition module and a cloud-based recognition module. The local recognition module may be equipped with a computer vision model or a computer vision library, while the cloud-based recognition module may be equipped with a large visual language model.

[0046] Among them, computer vision models can include optical character recognition (OCR) models.

[0047] Computer vision libraries can include OpenCV.

[0048] The visual language big model can be a multimodal big model.

[0049] Among them, the multimodal large model is a deep learning model obtained by training the model on multiple types of data such as text, images, videos, and audio.

[0050] Please see Figure 2 , Figure 2 This is a flowchart illustrating the implementation of a display screen interconnection method provided in an embodiment of this application.

[0051] It should be noted that the display screen in this application embodiment can be an LED display screen, and the display screen is composed of multiple cabinets spliced ​​together.

[0052] like Figure 2 As shown, a display screen connection method provided in one embodiment of this application may include steps S101 to S1013, which are described in detail below: In S101, after detecting that the display screen is powered on, the display screen system controls the display screen to output the position information of each cabinet.

[0053] In practical applications, after the initial installation or replacement of the display screen, in order to ensure that the display screen can correctly map the displayed content, it is necessary to perform a screen connection operation to determine the correct position and row and column order of each cabinet in the entire display screen.

[0054] Therefore, in this embodiment, after detecting the first power-on following initial installation or replacement of the display screen, the display screen system can control the display screen to output the position information of each cabinet. This position information includes, but is not limited to, the port number and address sequence number of the cabinet.

[0055] In practical applications, the port number is a unique identifier for the physical interface on the enclosure, used to identify the signal input / output channel and define the physical connection between the enclosure and the controller / other enclosures.

[0056] The address number is the logical identifier of the cabinet in the video wall, uniquely mapping the physical location (row and column coordinates) of the cabinet, and is used for system addressing and screen partitioning control.

[0057] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram of multimodal data provided in an embodiment of this application. For example... Figure 3 As shown, L represents the display screen, P represents the port number of its corresponding cabinet, S represents the address number of its corresponding cabinet, and the small square X containing each P and S is the sub-screen of its corresponding cabinet used for splicing the display screen.

[0058] In S102, when the user terminal detects the user's data acquisition command, it acquires multimodal data from the display screen.

[0059] In this embodiment of the application, after the user sees the position information of each box displayed on the above-mentioned display screen on their own screen, the user can send a data collection command to their own user terminal.

[0060] It should be noted that when a user terminal detects that its own camera application has been clicked, it means that the above-mentioned data collection command has been detected.

[0061] It should be noted that multimodal data is used to describe the data format and content presented on the display screen.

[0062] In this embodiment, when the user terminal detects a data collection command from the user, it can use its own camera to capture / record the display screen to obtain multimodal data of the display screen. The multimodal data includes the position information of each cabinet of the display screen.

[0063] In some possible embodiments, when the display screen is irregularly shaped or extra-large, the user can also hand-draw the positional information of each cabinet on the display screen to obtain a hand-drawn sketch. Then, the user can send a data acquisition command to the user terminal to capture the hand-drawn sketch, thereby obtaining the image corresponding to the sketch, i.e., obtaining the multimodal data of the display screen.

[0064] In one embodiment of this application, the user terminal obtains multimodal data according to the following steps, detailed below: The data acquisition method is determined based on the data acquisition command; the data acquisition methods include photo capture and video capture. Data acquisition is performed on the display screen based on the data acquisition method to obtain initial data; Data augmentation is performed on the initial data to obtain multimodal data.

[0065] It should be noted that the data acquisition command carries the data acquisition method. This includes both photo capture and video capture.

[0066] In this embodiment, after the user terminal detects the data acquisition instruction, it can extract the data acquisition method from the data acquisition instruction and perform corresponding data acquisition operations on the display screen based on the data acquisition method to obtain initial data.

[0067] Specifically, when the user terminal detects that the data collection method is photo capture, the user terminal can use the camera to perform a photo capture operation on the display screen to obtain the corresponding image on the display screen.

[0068] It should be noted that there can be one image or multiple images.

[0069] In some possible embodiments, when the display screen is irregularly shaped or extra-large, the camera of the user terminal may not be able to capture the complete picture of the display screen in a single image due to limitations in the hardware parameters of the user terminal's camera. Therefore, in this embodiment, the user can capture the complete picture of the display screen by taking multiple pictures of the display screen from multiple angles using the user terminal's camera.

[0070] When the user terminal detects that the data acquisition method is video, it can record the screen using a camera to obtain the corresponding video.

[0071] In this embodiment, after obtaining the initial data, the user terminal can perform data augmentation processing on the initial data to obtain multimodal data.

[0072] In practical applications, data augmentation methods include, but are not limited to: geometric transformations (such as rotation, flipping, scaling, cropping, and translation, which change the geometric properties of an image while preserving its semantics), color transformations (such as adjusting brightness, contrast, hue, and saturation to simulate different lighting conditions), noise and blurring (such as adding Gaussian noise or applying motion blur to enhance the model's robustness to changes in image quality), and advanced augmentation methods (such as Mixup, CutMix, and style transfer, which improve model performance by mixing or transforming image features).

[0073] In some possible embodiments, when there are multiple images, the user terminal can stitch the multiple images together to obtain a stitched image that includes the complete picture of the display screen, and identify the stitched image as multimodal data.

[0074] The method provided in this embodiment supports both photo and video acquisition methods, allowing users to flexibly choose according to the actual scenario: for static video walls, the photo method can be used to quickly acquire single-frame high-definition data; for video walls in dynamic environments, the video method can be used to acquire multiple frames of data, avoiding the accidental interference of single-frame acquisition. Subsequently, data augmentation processing is performed on the initially acquired data, effectively correcting initial data defects caused by insufficient light, lens shake, or obstruction, and improving the clarity and recognizability of the cabinet position information. Simultaneously, the data augmentation processing gives the acquired data stronger anti-interference capabilities, preserving core feature information such as cabinet position and ports even in complex environments, preventing interruptions in the screen connection process due to data quality issues.

[0075] In S103, the user terminal sends multimodal data to the multimodal recognition module.

[0076] In S104, the multimodal recognition module receives multimodal data from the display screen sent by the user terminal.

[0077] In S105, the multimodal recognition module inputs multimodal data into the computer vision model for multimodal recognition to obtain feature data.

[0078] In this embodiment, after receiving multimodal data from the display screen sent by the user terminal, the multimodal recognition module can input the multimodal data into a local computer vision model for multimodal recognition to obtain feature data. The feature data includes the features of the image / video / hand-drawn sketch, as well as the positional information of each box.

[0079] In S106, the multimodal recognition module determines the target mapping information based on feature data.

[0080] In this embodiment of the application, after obtaining the above-mentioned feature data, the multimodal recognition module can determine the target mapping information based on the feature data.

[0081] The target mapping information may include the port number, address number, position coordinates, number of rows and columns, direction, and connection relationship of each cabinet of the display screen.

[0082] In practical applications, position coordinates are used to describe the physical spatial coordinates of the display cabinet within the overall layout of the splicing screen, and are the core parameter for the geometric positioning of the cabinet. They can be represented using a two-dimensional rectangular coordinate system or a relative coordinate system. For example, with the top left corner of the splicing screen as the origin (0, 0), the position coordinates of a certain cabinet can be represented as (X=500mm, Y=300mm), or a row-column relative coordinate system can be used, such as (row 2, column 3).

[0083] The number of rows and columns includes two parameters: the number of rows and columns of a single cabinet (e.g., 1920×1080 pixels) and the total number of cabinet rows and columns in the entire splicing screen (e.g., 3 rows × 4 columns, a total of 12 cabinets).

[0084] The orientation is used to describe the physical installation orientation of the display cabinet, that is, the display angle of the sub-screen, to adapt to irregular splicing scenarios, such as horizontal (0°), vertical (90° / 270°), 180° rotation and inversion, etc.

[0085] The connection relationship is used to describe the physical signal links and logical networking relationships between multiple display cabinets. It is a comprehensive association of parameters such as port number and address sequence number.

[0086] The expression of continuous relationships can be: including port cascading relationships (such as the OUT port of enclosure A connecting to the IN port of enclosure B) and logical topology relationships (such as the enclosure with address number 001 being the first level, cascading the enclosures with control addresses 002-004).

[0087] In this embodiment, the multimodal recognition module can compare the obtained feature data with each historical data in the database to detect whether there is any historical data in the database with a similarity greater than a set similarity, i.e., similar data, and perform actions such as... based on the comparison results. Figure 4 The steps S201 or S202 are shown. The similarity setting can be determined according to actual needs and is not limited here.

[0088] In one embodiment of this application, the multimodal recognition module can specifically be implemented through, as follows: Figure 4 The steps shown are for determining the target mapping information, detailed below: In S201, if there is historical data in the database with a similarity greater than a set similarity to the feature data, then the historical mapping information corresponding to the historical data is obtained, and the historical mapping information is determined as the target mapping information.

[0089] In this embodiment, when the multimodal recognition module detects historical data in the database that has a similarity greater than a set similarity with the feature data, it indicates that there is historical data in the database that is similar to the feature data, i.e., there is similar data. Therefore, the multimodal recognition module can directly obtain the historical mapping information corresponding to the historical data and determine the historical mapping information as the target mapping information.

[0090] In S202, if historical data is not available in the database, the feature data is input into the visual language big model for processing to obtain target mapping information.

[0091] In this embodiment, when the multimodal recognition module detects that there is no historical data in the database that has a similarity greater than a set similarity with the feature data, it means that there is no historical data similar to the feature data in the database, that is, there is no similar data. Therefore, the multimodal recognition module can input the feature data into the visual language big model for processing to obtain target mapping information.

[0092] Combining steps S201-S202, the method provided in this embodiment prioritizes matching historical data in the database with similarity exceeding a threshold, directly reusing mature historical mapping information without needing to call a large visual language model for entirely new calculations. For repetitive deployment scenarios (such as regular equipment maintenance in the same showroom or batch splicing of the same model of cabinets), it significantly improves mapping efficiency. When there is no matching historical data, the large visual language model can be used to specifically process new feature data, utilizing the model's multimodal understanding capabilities to accurately analyze parameters for special scenarios such as new cabinets and irregular splicing, ensuring full coverage and adaptation of the solution to unknown scenarios. Therefore, this implementation method achieves a balance between efficiency and adaptability: for example, in conventional scenarios, it follows the historical matching path, balancing speed and stability; in innovative scenarios, it follows the large model calculation path, ensuring the compatibility and scalability of the solution, and adapting to the mixed splicing needs of display cabinets of different brands and specifications.

[0093] In one embodiment of this application, the large visual language model in the multimodal recognition module can be specifically implemented as follows: Figure 5 The steps shown implement step S202, which are detailed below: In S301, the visual language big model performs correction processing on the feature data to obtain the data to be processed.

[0094] In this embodiment, since each frame of video data in an image or video may have problems such as distortion, tilt, and perspective, the visual language big model can first perform correction processing on the feature data to obtain standard data to be processed.

[0095] It should be noted that the specific implementation process of the correction process can be found in existing image correction processes, and will not be elaborated here.

[0096] Then, the visual language big model can recognize and process the data to be processed to obtain target mapping information and output the target mapping information.

[0097] In some possible embodiments, the target mapping information output by the large visual language model is in the following format: { "rows":<number of rows>, "cols": <column number>, "matrix": [ [ {"P":x, "S":y}, {"P":x, "S":y}, ... ], [ {"P":x, "S":y}, {"P":x, "S":y}, ... ], ... ], "confidence_matrix": [ [0.98, 0.95, ...], ... ], "missing_filled": true / false, Comments: "Explain which parts are inferred completion and which are derived from multi-frame fusion." } In one embodiment of this application, after executing step S301, the visual language big model in the multimodal recognition module can specifically execute steps S301 to S303 to obtain target mapping information.

[0098] In another embodiment of this application, after performing step S301, the visual language large model can also be processed through methods such as... Figure 6 The steps S401~S402 shown below obtain the target mapping information, which is described in detail below: In S401, when the visual language big model detects that the multi-model data is of video type, it sequentially identifies each frame of video data in the data to be processed to obtain the position information corresponding to each frame of video data.

[0099] In S402, the visual language big model combines and processes the location information to output target mapping information.

[0100] In this embodiment, when the visual language big model detects that the multi-model data is of video type, the visual language big model can sequentially identify each frame of video data in the data to be processed to obtain the location information corresponding to each frame of video data.

[0101] Subsequently, the visual language big model can combine and process the positional information corresponding to each frame of video data to obtain the final target mapping information and output the target mapping information.

[0102] In some possible embodiments, the above combined processing method can be a splicing method.

[0103] Combining steps S401-S402, the method provided in this embodiment designs frame-by-frame recognition logic for multimodal video data, breaking through the limitations of acquiring single image data. It supports acquiring the position information of the display cabinet through dynamic video, eliminating the need to convert the video into a static image and reducing the complexity and computational consumption of data preprocessing. Furthermore, by recognizing each frame of the video data separately to obtain multiple sets of independent cabinet position information, and then combining these to generate the final mapping parameters, it effectively filters out recognition errors caused by sudden changes in light, lens shake, or partial occlusion in a single frame image.

[0104] In one embodiment of this application, the visual language large model in the multimodal recognition module specifically combines and processes the location information according to the following steps, detailed below: By concatenating the information from each location, a two-dimensional matrix structure is obtained; If different location information corresponding to different target video data is detected to be the same box location information, then the blur level of each frame of target video data is detected. The location information corresponding to the target video data with the lowest degree of image blur is determined as the target location information of the same box. The target mapping information is obtained by updating the two-dimensional matrix structure information based on the target location information.

[0105] In this embodiment, the visual language big model can stitch together the positional information corresponding to each frame of video data to obtain the initial two-dimensional matrix structure information.

[0106] Then, the visual language big model can detect whether each box has multiple different positional information.

[0107] In this embodiment, the visual language big data model detects that a certain box has multiple different location information. That is, the different location information corresponding to multiple different target video data are all the location information of the box. Therefore, in order to improve the recognition accuracy of the box, the visual language big data model can detect the blur degree of each frame of target video data.

[0108] Then, the visual language big model can determine the location information corresponding to the target video data with the lowest degree of image blur as the target location information of the same box, and update the above two-dimensional matrix structure information based on the target location information to obtain the final target mapping information.

[0109] The method provided in this embodiment first stitches together multi-frame position information into a two-dimensional matrix structure to construct a global topological framework for the display cabinet positions. This ensures that the positions, rows, columns, and connections of all cabinets form a complete logical association, avoiding isolated errors from scattered data. Then, for different position information of the same cabinet appearing in multiple video frames, the optimal value is determined through image blur filtering. The position data corresponding to the frame with the highest clarity is prioritized, effectively eliminating recognition deviations caused by lens shake, light obstruction, and motion blur, thereby reducing errors in the position coordinates of individual cabinets. Finally, the two-dimensional matrix structure information updated based on the optimal position information can highly match the actual topology of the physical splicing of the display screens, providing stable and reliable parameter support for subsequent screen-to-screen operations and ensuring image consistency and synchronization during long-term operation.

[0110] In another embodiment of this application, after performing step S301, the visual language large model can also obtain target mapping information according to the following steps, detailed below: The visual language big model identifies the data to be processed and obtains the initial mapping information corresponding to the data to be processed. When the visual language big data model detects that the data to be processed is incomplete, it generates the predicted position information of the third box based on the position information of the second box that exists in the data to be processed. Incomplete data is used to describe that the data to be processed is not a complete picture of the display screen. The third box refers to the box that does not exist in the data to be processed. The visual language big model performs a completion operation on the initial mapping information based on the predicted position information of the third box to output the target mapping information.

[0111] In this embodiment, the visual language big model can identify the data to be processed and obtain the initial mapping information corresponding to the data to be processed.

[0112] Meanwhile, the visual language big data model can detect in real time whether the data to be processed is incomplete. Incomplete data is used to describe data that is not a complete image of the display screen.

[0113] In this embodiment, when the visual language big data model detects that the data to be processed is incomplete, it indicates that the data does not contain the position information of all the boxes displayed on the screen. Therefore, the visual language big data model can generate the predicted position information of the third box based on the position information of the second box present in the data to be processed and the increasing pattern of the port number / address number. Here, the third box refers to the box that does not exist in the data to be processed.

[0114] Then, the visual language big model can perform a completion operation on the initial mapping information based on the predicted position information of the third box determined above, so as to obtain the final target mapping information and output the target mapping information.

[0115] The method provided in this embodiment first generates initial mapping information based on existing data, and then completes the mapping for missing cabinets. This ensures the accuracy of existing cabinet mapping parameters and achieves full coverage of overall mapping information, avoiding local mapping problems caused by missing data for some cabinets. It also ensures that there are no blind spots in the splicing of the screen and no signal interruptions after screen networking. As can be seen, this implementation method is suitable for scenarios where the data to be processed is an incomplete image (such as when the front-end device has a limited field of view during acquisition or when temporary screen networking lacks some cabinet data). The model can automatically detect data integrity and trigger the completion process, eliminating the need for manual data re-acquisition and solving the pain point of process interruption due to incomplete data in traditional solutions.

[0116] In S302, during the process of visual language large model recognizing the data to be processed, if the position information of the first box in the data to be processed cannot be recognized, the predicted position information of the first box is generated based on the position information of the other boxes.

[0117] In S303, the visual language big model outputs target mapping information based on the position information of the other boxes and the predicted position information of the first box.

[0118] In this embodiment, when the visual language big data model detects that the position information of the first box in the data cannot be identified during the recognition process, it indicates that some boxes in the data are blurred or occluded, resulting in unclear position information. Therefore, the visual language big data model can generate the predicted position information of the first box based on the changing patterns of the position information of the remaining boxes. Specifically, the remaining boxes refer to boxes other than the first box.

[0119] In this embodiment, the visual language big model can stitch together the predicted position information of the first box and the position information of the other boxes to obtain the final target mapping information and output the target mapping information.

[0120] Combining steps S301-S303, the method provided in this embodiment first performs correction processing on the feature data, which can effectively filter noise caused by equipment errors and environmental interference during multimodal data acquisition, avoid mapping errors caused by defects in the original data, and improve the recognition accuracy of core parameters such as box position, number of rows and columns, and connection relationship. For extreme scenarios where the position information of the first box in the data to be processed cannot be identified, prediction and completion are performed based on the position information of the remaining boxes, overcoming the limitation of data integrity dependence and enabling complete generation of mapping information even with missing local data. Therefore, this implementation method can avoid process interruptions caused by missing local data, eliminate the need for repeated data re-acquisition, shorten the mapping information generation cycle, and improve the overall smoothness of the connected screen operation.

[0121] In S107, the multimodal recognition module sends the target mapping information to the user terminal.

[0122] In S108, the user terminal receives target mapping information sent by the multimodal recognition module.

[0123] In S109, the user terminal outputs the target mapping information.

[0124] In this embodiment of the application, after obtaining the target mapping information, the multimodal recognition module can send the target mapping information to the user terminal.

[0125] After receiving the target mapping information sent by the multimodal recognition module, the user terminal can output the target mapping information, such as displaying it on its own display screen for the user to view.

[0126] After confirming that the target mapping information is correct, the user can send a confirmation message to the user terminal.

[0127] In S1010, after receiving the user's confirmation information, the user terminal sends the target mapping information to the display screen system.

[0128] In S1011, the display system receives target mapping information sent by the user terminal.

[0129] In S1012, the display system generates control parameters based on the target mapping information.

[0130] In this embodiment, after receiving the above confirmation information, the user terminal can send the target mapping information to the display screen system.

[0131] After receiving the target mapping information, the display screen system can generate a row and column matrix based on the target mapping information to establish a one-to-one correspondence between the position coordinates, address number, and port number of the cabinet.

[0132] For example, a row and column matrix can be as follows: { "rows": 5, "cols": 5, "matrix": [ [{"P":5,"S":1},{"P":1,"S":1},{"P":2,"S":1},{"P":4,"S":1}, {"P":3,"S":1} ], [{"P":5,"S":2},{"P":1,"S":2},{"P":2,"S":2},{"P":4,"S":2}, {"P":3,"S":2} ], [{"P":5,"S":3},{"P":1,"S":3},{"P":2,"S":3},{"P":4,"S":3}, {"P":3,"S":3} ], [{"P":5,"S":4},{"P":1,"S":4},{"P":2,"S":4},{"P":4,"S":4}, {"P":3,"S":4} ], [{"P":5,"S":5},{"P":1,"S":5},{"P":2,"S":5}, {"P":4,"S":5}, {"P":3,"S":5} ] ] }

[0133] Then, the display system can generate control parameters for each cabinet of the display based on this row and column matrix, i.e., the screen configuration information.

[0134] Specifically, the sending card in the display system can generate image segmentation parameters (to extract sub-images according to the display area of ​​each cabinet), signal distribution parameters (binding logical addresses and port numbers, setting frame synchronization delay) and driving parameters (grayscale level, scanning frequency, and brightness compensation, etc.) based on the above-mentioned row and column matrix, thereby obtaining the above-mentioned control parameters.

[0135] It should be noted that the logical address allocation rule can be either column-first or row-first, assigning a unique logical ID to each enclosure (e.g., P01-03-02: port 01, column 3, row 2).

[0136] In S1013, the display system controls the display screen to display based on control parameters in order to realize the screen connection operation.

[0137] In this embodiment, the sending card of the display system can send the data corresponding to the sub-image and the above-mentioned control parameters to the receiving card of the display screen in logical address order.

[0138] Afterwards, the receiving card can write the driving parameters into the driving register, output them to the cabinet driver board through the interface, and send test images to control the display screen to display, thereby realizing the screen connection operation.

[0139] As can be seen from the above, the display screen connection method provided in this application embodiment allows the user terminal to support both photo and video acquisition methods. Data enhancement preprocessing repairs initial data defects caused by lighting, occlusion, and equipment errors, providing high-recognition, low-noise multimodal data for the multimodal recognition module. Simultaneously, after the display system powers on, it actively outputs the position information of each cabinet, forming a standardized raw data source, avoiding manual annotation errors from the source and ensuring the accuracy of subsequent processing. The multimodal recognition module employs a two-layer logic of prioritizing historical data matching and using a large visual language model as a fallback to generate mapping information. In mature scenarios, validated historical mapping parameters are directly reused to avoid model calculation biases. For new scenarios or data gaps, the large visual language model uses a progressive processing of data correction, position prediction, and mapping completion. Even in extreme scenarios such as partial data loss or incomplete image acquisition, it can predict and complete position information based on the physical arrangement of the cabinets, with the position coordinate error of a single cabinet controlled within millimeters. For video-based multimodal data, the multimodal recognition module employs frame-by-frame recognition and ambiguity filtering to extract the positional information of the highest-resolution frame and construct a two-dimensional matrix topology. This further improves the accuracy of mapping parameters, ensuring that core parameters such as cabinet ports, row and column numbers, and connection relationships are highly compatible with the physical splicing scenario. Finally, based on the standardized target mapping information generated by the multimodal recognition module, the display system automatically generates control parameters deeply adapted to the hardware architecture. This avoids random errors from manual cabinet-by-cabin debugging and completely solves problems such as screen tearing, edge misalignment, and signal asynchrony in traditional solutions. The consistency of the screen after connection and the stability of long-term operation are significantly improved. Furthermore, this application establishes a four-level control mechanism: multimodal recognition module generation, user terminal confirmation, terminal distribution, and display system execution. The user terminal outputs target mapping information for manual verification, proactively avoiding recognition deviations or data transmission errors by the multimodal recognition module. Only after user confirmation is the mapping information distributed to the display system, effectively preventing display anomalies caused by illegal commands or misoperations, and ensuring the security and controllability of the connected screen operation.

[0140] Please see Figure 7 , Figure 7 This is a flowchart illustrating the overall implementation of a display screen interconnection method provided in an embodiment of this application. Figure 7 As shown, taking a display screen system including an LED control system and a display screen as an example, after power-on, the display screen system can control the LED display screen cabinet array (i.e., each cabinet) to display a unique number / QR code (such as...). Figure 2(See step S101 shown). After the user sees the above information displayed on the screen, they can obtain initial data through their user terminal's multimodal acquisition module via photography / video / hand-drawn sketches. The user terminal can then use a preprocessing module to perform image enhancement / stitching on this initial data to obtain multimodal data (such as...). Figure 2 The specific implementation process of step S102 shown is as follows: the multimodal data is sent to the multimodal recognition module through the recognition request interface. After obtaining the multimodal data, the multimodal recognition module can perform number / QR code recognition (e.g., using its local OCR recognition submodule) Figure 2 Step S105 is shown.

[0141] In some possible embodiments, the multimodal recognition module can quickly identify similar data through a database to rapidly determine familiar screens, and use a multiplexing module mechanism for familiarity mapping matching to quickly load historical configurations (such as...). Figure 4 Step S201 (as shown) determines the similarity mapping information (i.e., historical mapping information). Afterwards, the multimodal recognition module sends this similarity mapping information to the user terminal. The user terminal outputs this similarity mapping information for the user to trigger and confirm reuse, and to detect similar screen results. After confirmation by the user, the user terminal sends the similarity mapping information to the display system. The display system can configure screen connection based on this similarity mapping information, i.e., generate screen connection configuration parameters (i.e., control parameters). The receiving card in the display system can receive these screen connection configuration parameters and control the display driver module to display. Based on this, the display can display the splicing result, and the display system can detect whether the display screen's image display is normal to determine if the screen connection is successful.

[0142] It should be noted that "recognition screen" is used to describe that the screen has appeared before, and "recognition mapping matching" is used to describe matching the feature data with each historical mapping information stored in the database to obtain recognition mapping information.

[0143] In other possible embodiments, when historical data is not available in the database, the multimodal recognition module can input feature data into the VLM multimodal model in the cloud for processing, thereby understanding the image / video / sketch and obtaining target mapping information, i.e., generating the LED box number, row and column coordinates, connection direction, and port identification (e.g., ...). Figure 4 As shown in step S202, Figure 5 Steps S301 to S303 as shown, and as follows Figure 6(See steps S401-S402). Afterwards, the multimodal recognition module can send the two-dimensional mapping matrix, i.e., the target mapping information, to the user terminal through the result output interface. The user terminal can confirm / edit the target mapping information through user interaction and can view the screen-to-screen preview. If there are problems with the screen-to-screen preview, error messages and retrying can be displayed. After user confirmation, the user terminal can send the target mapping information to the display screen system. The display screen system can configure the screen-to-screen display based on this target mapping information, i.e., generate screen-to-screen configuration parameters (i.e., control parameters). The receiving card in the display screen control system can receive these screen-to-screen configuration parameters and control the display driver module to display the image. Based on this, the display screen can display the splicing result. The display screen system can detect whether the screen display is normal to determine if the screen-to-screen display is successfully connected.

[0144] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0145] Figure 8 This is a schematic diagram of the structure of a multimodal recognition module provided in one embodiment of this application. Figure 8 As shown, the multimodal recognition module 5 of this embodiment includes: at least one processor 50 ( Figure 8 Only one is shown in the diagram), memory 51, and computer program 52 stored in memory 51 and executable on at least one processor 50, wherein the processor 50 executes computer program 52 to implement the steps in any of the above-described embodiments of the screen connection method.

[0146] The multimodal recognition module 5 may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 8 This is merely an example of the multimodal recognition module 5 and does not constitute a limitation on the multimodal recognition module 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0147] The processor 50 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0148] In some embodiments, the memory 51 may be an internal storage unit of the multimodal recognition module 5, such as the RAM of the multimodal recognition module 5. In other embodiments, the memory 51 may be an external storage device of the multimodal recognition module 5, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the multimodal recognition module 5. Furthermore, the memory 51 may include both internal storage units and external storage devices of the multimodal recognition module 5. The memory 51 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0149] Figure 9 This is a schematic diagram of the structure of a user terminal provided in an embodiment of this application. Figure 9 As shown, the user terminal 6 in this embodiment includes: at least one processor 60 ( Figure 9 (Only one is shown in the diagram), memory 61, and computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 executes the computer program 62 to implement the steps in any of the above-described embodiments of the display screen interconnection method.

[0150] The user terminal 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 9 This is merely an example of user terminal 6 and does not constitute a limitation on user terminal 6. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0151] The processor 60 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0152] In some embodiments, the memory 61 may be an internal storage unit of the user terminal 6, such as the RAM of the user terminal 6. In other embodiments, the memory 61 may be an external storage device of the user terminal 6, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the user terminal 6. Furthermore, the memory 61 may include both internal storage units and external storage devices of the user terminal 6. The memory 61 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0153] Figure 10 This is a schematic diagram of the structure of a display screen system provided in one embodiment of this application. Figure 10 As shown, the display system 7 of this embodiment includes: at least one processor 70 ( Figure 10 (Only one is shown in the diagram), memory 71, and computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above-described embodiments of the screen connection method.

[0154] The display system 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 10 This is merely an example of the display system 7 and does not constitute a limitation on the display system 7. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0155] The processor 70 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0156] In some embodiments, the memory 71 may be an internal storage unit of the display system 7, such as the RAM of the display system 7. In other embodiments, the memory 71 may be an external storage device of the display system 7, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the display system 7. Furthermore, the memory 71 may include both internal storage units and external storage devices of the display system 7. The memory 71 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0157] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0158] This application provides a computer program product that, when run on a multimodal recognition module, enables the multimodal recognition module to implement the steps described in the above-described method embodiments; when run on a user terminal, enables the user terminal to implement the steps described in the above-described method embodiments; and when run on a display screen system, enables the display screen system to implement the steps described in the above-described method embodiments.

[0159] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a multimodal recognition module / user terminal / display system, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0160] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0161] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for connecting multiple displays, characterized in that, The method, applied to a multimodal recognition module, includes: Receive multimodal data of the display screen sent by the user terminal device; the multimodal data includes the position information of each cabinet of the display screen; The multimodal data is input into a computer vision model for multimodal recognition to obtain feature data; Target mapping information is determined based on the feature data; the target mapping information includes the port number, address number, position coordinates, row and column number, direction, and connection relationship of each cabinet of the display screen; The target mapping information is sent to the user terminal so that the user of the user terminal can confirm the target mapping information. After the user terminal detects the confirmation information, it sends the target mapping information to the display screen system so that the display screen system can perform a screen connection operation on the display screen based on the target mapping information.

2. The display screen interconnection method as described in claim 1, characterized in that, The step of determining the target mapping information based on the feature data includes: If there is historical data in the database that has a similarity greater than a set similarity with the feature data, then the historical mapping information corresponding to the historical data is obtained, and the historical mapping information is determined as the target mapping information; If the historical data is not found in the database, the feature data is input into the visual language big data model for processing to obtain the target mapping information.

3. The display screen interconnection method as described in claim 2, characterized in that, The step of inputting the feature data into a large visual language model for processing to obtain the target mapping information includes: The visual language big model performs correction processing on the feature data to obtain the data to be processed. During the process of the visual language big model recognizing the data to be processed, if the position information of the first box in the data to be processed cannot be recognized, the predicted position information of the first box is generated based on the position information of the other boxes. The visual language big model outputs the target mapping information based on the position information of the other boxes and the predicted position information of the first box.

4. The display screen connection method as described in claim 3, characterized in that, After the visual language large model corrects the feature data to obtain the data to be processed, the method further includes: The visual language big model identifies the data to be processed and obtains the initial mapping information corresponding to the data to be processed. When the visual language model detects that the data to be processed is incomplete, it generates predicted position information of the third box based on the position information of the second box present in the data to be processed; the incomplete data is used to describe that the data to be processed is not a complete picture of the display screen; the third box refers to the box that does not exist in the data to be processed. The visual language big model performs a completion operation on the initial mapping information based on the predicted position information of the third box, so as to output the target mapping information.

5. The display screen interconnection method as described in claim 3, characterized in that, After the visual language large model corrects the feature data to obtain the data to be processed, the method further includes: When the visual language big model detects that the multi-model data is a video type, it sequentially identifies each frame of video data in the data to be processed to obtain the position information corresponding to each frame of video data. The visual language big model combines and processes the various location information to output the target mapping information.

6. The display screen interconnection method as described in claim 5, characterized in that, The process of combining the various location information includes: The location information is concatenated to obtain a two-dimensional matrix structure. If different location information corresponding to different target video data is detected to be the same location information of the same box, then the blurring degree of the target video data in each frame is detected. The location information corresponding to the target video data with the lowest degree of image blur is determined as the target location information of the same box. The target location information is used to update the two-dimensional matrix structure information to obtain the target mapping information.

7. A method for connecting multiple displays, characterized in that, Applied to a user terminal, the method includes: When a user's data collection command is detected, multimodal data of the display screen is acquired; the multimodal data includes the position information of each cabinet of the display screen; The multimodal data is sent to the multimodal recognition module; The system receives target mapping information sent by the multimodal recognition module and outputs the target mapping information; the target mapping information is obtained by the multimodal recognition module processing the multimodal data. After receiving confirmation from the user, the target mapping information is sent to the display screen system so that the display screen system can perform a screen connection operation on the display screen based on the target mapping information.

8. The display screen interconnection method as described in claim 7, characterized in that, The acquisition of multimodal data of the display screen includes: The data acquisition method is determined based on the data acquisition command; the data acquisition method includes photo capture and video capture. Based on the data acquisition method, a data acquisition operation is performed on the display screen to obtain initial data; The initial data is augmented to obtain the multimodal data.

9. A method for connecting multiple displays, characterized in that, Applied to a display system, the method includes: After the display screen is powered on, control the display screen to output the position information of each cabinet; The system receives target mapping information sent by a user terminal. The target mapping information is obtained by the user terminal sending multimodal data containing the position information of each box to a multimodal recognition module, which is obtained by the user terminal through data acquisition of the display screen. The multimodal recognition module processes the multimodal data. Control parameters are generated based on the target mapping information; The display screen is controlled based on the control parameters to achieve screen-to-screen operation.

10. A display screen interconnection system, characterized in that, This includes a multimodal recognition module, a user terminal, and a display screen system; The multimodal recognition module is connected to the user terminal and is used to execute the display screen connection method as described in any one of claims 1-6; The user terminal is connected to the display screen system and is used to execute the display screen connection method as described in any one of claims 7-8; The display screen system is used to perform the display screen interconnection method as described in claim 9.

Images