Multi-user interaction method and system based on picture transmission and multimedia resources

Abstract

The application discloses a multi-user interaction method and system based on picture transmission and multimedia resources. The multi-user interaction method comprises the following steps: obtaining an interaction base map, and distributing and presenting the interaction base map to clients of multiple users; accepting a multimedia operation, and synchronizing the multimedia operation to the clients of the multiple users; and updating and presenting the interaction base map after the multimedia operation to the multiple users. The multi-user interaction method provided by the application does not need to rely on streaming media to continuously transmit images or audio, avoids the disadvantages of relying on video streaming and / or audio streaming, and can still realize relatively smoother remote interaction in a weak network environment. Compared with the screen recording or audio recording of the existing interaction mode, the multi-user interaction method provided by the application can more conveniently record the segmented interaction process, can call and import the segmented interaction process in the next interaction, and can make multiple interactions have excellent continuity.

Description

Technical Field

[0001] This invention relates to the field of remote interaction technology, and in particular to a multi-user interaction method and system based on image transmission and multimedia resources. Background Technology

[0002] With the continuous development of Internet technology, remote interaction methods such as live streaming and remote conferencing have gradually become widely used.

[0003] In existing technologies, remote interaction systems are typically based on video and audio streams. However, this approach requires a very stable network connection. Poor network conditions can lead to communication interruptions, interaction stutters, and other anomalies, significantly impacting the interactive experience. Furthermore, current remote interaction systems cannot easily save previous communication records, let alone ensure continuity between interactions. Therefore, ensuring normal operation in weak network environments and / or maintaining continuity across multiple interactions are pressing issues that need to be addressed. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a multi-user interaction method and system based on image transmission and multimedia resources.

[0005] To achieve the aforementioned objectives, the technical solution adopted by this invention includes:

[0006] In a first aspect, the present invention provides a multi-user interaction method based on image transmission and multimedia resources, comprising:

[0007] Obtain the interaction base map, and distribute and present the interaction base map to the clients of multiple users;

[0008] Accept multimedia operations and synchronize the multimedia operations to multiple users' clients;

[0009] The interactive background image after the multimedia operation is updated and presented to multiple users.

[0010] Secondly, the present invention also provides a multi-user interactive system based on image transmission and multimedia resources, comprising:

[0011] The base map acquisition module is used to acquire the interactive base map and distribute and present the interactive base map to the clients of multiple users;

[0012] An operation synchronization module is used to accept multimedia operations and synchronize the multimedia operations to multiple users' clients;

[0013] The interactive presentation module is used to update and present the interactive background image after the multimedia operation to multiple users.

[0014] In some preferred embodiments, the interactive system further includes:

[0015] The knowledge archiving module is used to archive the interactive base map and multimedia operations to form a knowledge manual, which can be called back by the client.

[0016] Thirdly, the present invention also provides a multi-user interactive web page platform based on image transmission and multimedia resources, comprising:

[0017] The initiating unit is used to accept the initiating operation from the initiating device, establish an interactive room in the web service platform and generate a connection address corresponding to the interactive room, and the receiving device enters the interactive room through the connection address;

[0018] The base map unit is used to accept base map upload operations from the initiating device and / or the receiving device, and to distribute the obtained interactive base map to the initiating device and the receiving device.

[0019] A multimedia operation unit is configured to receive multimedia operations from an initiating device and / or a receiving device, and to synchronize the multimedia operations to the initiating device and the receiving device. The multimedia operations are used to update the content presented on the initiating device and the receiving device.

[0020] In some preferred embodiments, the web service platform further includes an archiving unit for accepting archiving operations from the initiating device and / or the receiving device, and for archiving and recording the interactive background map and multimedia operations generated in the interactive room.

[0021] Based on the above technical solution, compared with the prior art, the beneficial effects of the present invention include at least the following:

[0022] The multi-user interaction method provided by this invention updates the presented content by transmitting the interactive base map and combining multimedia operations. The content transmission is segmented and does not rely on continuous transmission of images or audio through streaming media. This avoids the drawbacks of existing live streaming or remote conferencing that rely on video and / or audio streams. It can still achieve relatively smoother remote interaction in weak network environments.

[0023] Furthermore, in some more preferred embodiments of the present invention, compared with existing interactive methods such as screen recording or audio recording, the multi-user interaction method provided by the present invention can more conveniently record the segmented interaction process, which is very convenient for review or debriefing. In addition, in conjunction with the above-mentioned interaction method, this form of recording can be recalled and imported in the next interaction, so that multiple interactions can have excellent continuity.

[0024] The above description is merely an overview of the technical solution of the present invention. In order to enable those skilled in the art to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described below in conjunction with detailed drawings. Attached Figure Description

[0025] Figure 1 This is an example diagram of the interface of a multi-user interactive system based on image transmission and multimedia resources, provided in a typical embodiment of the present invention. Detailed Implementation

[0026] In view of the shortcomings of the prior art, the inventors of this invention, through long-term research and extensive practice, have proposed the technical solution of this invention. The following will further explain and illustrate this technical solution, its implementation process, and its principles.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0028] This invention provides a multi-user interaction method based on image transmission and multimedia resources, which includes the following steps:

[0029] The process involves acquiring an interactive background image and distributing and presenting it to multiple users' clients. Specifically, this step may involve acquiring an image from the device's camera, local storage, or cloud storage, and using this image as the interactive background image.

[0030] In this invention, the interactive background image refers to an image that can express related meanings through multimedia operations such as drawing, marking, adding text, audio and video, etc. It can also be called a background image. For example, it can be a photograph of a device, with circles or marks drawn on the operation panel and accompanied by text or voice descriptions. Or it can be a temporarily drawn wireframe diagram of the device structure, with text / audio and video descriptions added to the transmission structure, etc. Any form of obtaining the corresponding image and using it in conjunction with the following operations to achieve the relevant meaning expression is within the scope of this invention.

[0031] The system accepts multimedia operations and synchronizes these operations to multiple user clients. As mentioned above, a multimedia operation refers to a process that, after being performed, presents altered or additional content based on the interactive background image. This includes not only additional doodles, markings, text, audio, and video, but also operations such as zooming in, deleting, rotating, and translating parts of the interactive background image. In other words, any operation that changes the content presented against the interactive background image and expresses a corresponding meaning falls under the category of multimedia operations.

[0032] The interactive background image after the multimedia operation is updated and presented to multiple users.

[0033] By adopting the above implementation scheme, the present invention can achieve relatively smooth multi-user interaction in weak network environments. Unlike the media streaming interaction methods in the prior art, which rely on continuous and stable network bandwidth for continuous transmission of streaming media, the data information transmission for multi-user interaction in the technical solution provided by the present invention relies on a segmented transmission mode. For example, the transmission of the background image only needs to be transmitted once at the beginning of the interaction or when the background image is changed, and the background image can be continuously displayed on the interactive device terminal. The various multimedia operations on the background image are also transmitted once in the form of packets, and after restoration, the corresponding presentation operations are formed, thereby realizing smooth interaction in weak network environments.

[0034] The weak network environment mentioned above refers to situations where the network bandwidth cannot meet the requirements of live streaming, or the network bandwidth is unstable and fluctuates, resulting in intermittent and buffering issues during traditional live streaming.

[0035] In some implementations, the multi-user interaction method may further include the following steps:

[0036] The interactive base map and multimedia operations are archived to form a knowledge manual, which can be called back by the client.

[0037] In some implementations, the multi-user interaction method may specifically include the following sub-steps:

[0038] The client's save operation is accepted, and the archive is formed into the knowledge manual.

[0039] The knowledge manual is pushed to multiple clients.

[0040] In some implementations, the knowledge manual is pushed out via a manual link.

[0041] In some implementations, the multi-user interaction method may further include the following sub-steps:

[0042] When multi-user interaction occurs again, the knowledge manual is invoked, and the current multi-user interaction is started based on the newly invoked knowledge manual.

[0043] Therefore, in addition to enabling relatively smooth multi-user interaction in weak network environments, the technical solution adopted in this invention also produces a unique effect different from existing streaming media interaction methods. Because the transmission of data and information is segmented, it eliminates the need for continuous screen recording or audio recording to save the interaction process, as is required by existing interaction recording methods. The segmented data information can be directly stored to preserve the interaction process, making it very convenient for subsequent review and analysis. Furthermore, this storage method is completely consistent with the interaction method at the time of interaction. This means that in the next interaction, the saved knowledge manual from this interaction can be directly imported to continue the content of this interaction, which is something that existing streaming media interaction methods cannot achieve.

[0044] Furthermore, due to the inherent advantages of this solution, the saved knowledge manuals can be accessed and read via web pages, facilitating integration into any other platform and strengthening the accumulation of the company's existing knowledge base.

[0045] Furthermore, compared to conventional video solutions where users joining midway cannot view previous content, resulting in the loss of some information, the implementation scheme provided by this invention allows users to view previously recorded content regardless of when they join the interaction, ensuring the continuity of information.

[0046] In some implementations, the client includes an initiating device and a receiving device; the initiating device and the receiving device interact with each other through a web service platform.

[0047] In some implementations, the multi-user interaction method may specifically include:

[0048] Accept the initiation operation from the initiating device, establish an interactive room in the web service platform, and generate the corresponding connection address for the interactive room.

[0049] The receiving device enters the interactive room via the connection address.

[0050] In some implementations, the connection address may include, for example, a URL and / or a graphic code.

[0051] In some implementations, the multimedia operations include any one or a combination of two or more of the following operations: marking and drawing on the interactive base map, and adding pictures, manuals, audio and / or video media.

[0052] In some implementations, the multi-user interaction method may specifically include the following steps:

[0053] The multimedia operations are converted and encapsulated into data packets.

[0054] The data packets are sent synchronously to the clients of multiple users.

[0055] The data packet is parsed to reconstruct the multimedia operation represented by the data packet.

[0056] Based on the restored multimedia operations, the content presented on the client is updated.

[0057] In some implementations, multiple multimedia operations are received from multiple different clients, and the received multimedia operations are simultaneously presented on the clients.

[0058] In a further preferred embodiment of the present invention, the multimedia operations performed by different users can be marked during specific operations. For example, a user icon or user number can be displayed next to the underline / doodle, and user information can be added to the attached video. This is to mark the operations performed by different users during the interaction. Correspondingly, when saving the interaction process, the multimedia operations of different users should also be marked and saved separately, which makes it easier to review and continue the interaction.

[0059] Alternatively, the interactive system may divide users into different levels, with different levels being able to perform different multimedia operations. For example, Level 1 users can perform all operations, Level 2 users can only doodle or mark and add voice, Level 3 users can only add text or mark simple multimedia operations, and Level 4 users cannot perform multimedia operations, and so on.

[0060] Furthermore, specific to the particular interaction method provided by this invention, the levels of different users can be non-fixed, for example, they can be floating and automatically adjusted. Specifically, when a level 1 user performs a complex multimedia operation such as adding voice or video, the current user's level is downgraded, and the levels of other users are correspondingly upgraded by one level. When they also perform the aforementioned complex multimedia operation, their level is downgraded by one level. When a room is first created, except for the initiator, the initial levels of all other users are the lowest level. After the initiator publishes video or audio, the other users gradually begin to gain level upgrades. In subsequent communication, new users are all at the lowest level by default.

[0061] The above-mentioned rules for changing the speaking level are optimization measures adopted to more effectively adapt to the interaction method provided by this invention. Since the information transmission of this invention is segmented rather than continuous streaming media, adjusting the speaking level is an important means of maintaining communication efficiency.

[0062] Of course, the above-mentioned level rules are not completely absolute. For example, similar to existing live streaming methods, level permissions can be changed manually by the initiator or administrator. This operation can be performed independently of the above-mentioned level change rules as an additional supplement. The same communication effect can be achieved by manually specifying / adjusting the level.

[0063] Furthermore, based on the specific interaction method provided by this invention, different users can be distinguished by different colors. During the interaction, all subsequent operations are differentiated by the assigned user identifier (color), facilitating identity differentiation during the interaction process. Additionally, if a user suddenly disconnects and re-enters, the currently assigned user identifier will be retained and inherited.

[0064] As some typical application examples of the above technical solutions, in specific implementation, the hardware of the interactive system is divided into client and server. The client communicates based on web page images and related multimedia information. For example, the conference interaction can accommodate 2 to 20 clients to exchange information at the same time. Of course, a larger number of interactive users can also be achieved. This is just a convenient example.

[0065] The client is divided into initiating devices and receiving devices. Initiating devices access the preparation interface via a web link, where they can create the rooms needed for remote interaction. The initiating device can share the created room with the receiving devices via a URL link or QR code. Receiving devices can enter the room by entering the link or scanning the QR code. Once inside the room, the initiating device can take a photo using its camera or import a local image, then use this image as a template to add multimedia operations such as drawing, arrows, attaching videos, and attaching images. When receiving devices enter the room, if the initiating device performs any of the following operations—drawing, drawing arrows, attaching multimedia resources such as videos and images, playing videos, or playing images—these operations will be encapsulated into data packets and sent to all receiving devices in the room via the server. All receiving devices in the room, upon receiving the data packets synchronized from the server, parse the packets and recreate all the operations encapsulated within them on their own devices, including but not limited to drawing, drawing arrows, attaching multimedia resources such as videos and images, playing videos, and playing images. This ensures that all devices in the room see the same screen.

[0066] In addition to the initiating device, all other receiving devices in this room can also operate on the web page synchronized to the receiving device, including but not limited to: drawing doodles, drawing arrows, attaching video and image multimedia resources, playing videos, playing images, etc. The above operations will also be encapsulated into data packets and sent to all other devices in this room through the server, including the initiating device and the receiving device. Similarly, these data packets will be parsed into the same operations by their respective devices and displayed on their respective device interfaces.

[0067] In a more specific implementation, all operations performed by all devices in this room are synchronized. For example, if devices A, B, and C all enter the same room, and if device A draws a line on the base map, and device B adds a video to the base map at the same time, then the interfaces of devices A, B, and C will all show that a line has been added to the base map and a video has been added.

[0068] Regarding the continuity of the aforementioned multi-user interaction, this embodiment of the invention provides further examples of knowledge storage and retrieval:

[0069] like Figure 1 As shown, by saving the operation, the results of all operations on all devices in the room can be preserved, including but not limited to: all background maps, graffiti, arrows, videos, and images, as well as other multimedia resources. Ultimately, these results can be saved by the initiating device as a knowledge manual containing all the aforementioned operation results. This knowledge manual can be saved and retrieved.

[0070] In the next interaction, the saved knowledge manual can also be retrieved when the room is created again and synchronized to all devices entering the room. This allows for efficient and convenient continuity of multi-user interaction. In contrast, existing media stream-based multi-user interaction methods cannot achieve such convenient continuity. This is because saving the previous interaction is very difficult with media streams, requiring screen recording or audio recording. Furthermore, this type of data is often difficult to retrieve and import in the next interaction, and it is also very difficult to review. For example, in a remote meeting, if the entire meeting lasts 30 minutes, continuing the previous meeting requires reviewing the 30-minute video recording to recall the previous communication and proceed with the current communication. In contrast, the technical solution provided by this invention allows for direct one-click import of the previously saved segmented data to start the next communication, which is obviously more efficient and convenient.

[0071] Therefore, it can be clearly seen that, based on the above-mentioned multi-user interaction method, the present invention solves the problem of remote interaction only being possible under extremely difficult network conditions, and the data of the entire interaction process can be saved in the form of a webpage for later review.

[0072] For the corresponding multi-user interaction methods mentioned above, see [link to relevant documentation]. Figure 1 As shown, this embodiment of the invention also provides a multi-user interactive system based on image transmission and multimedia resources, which includes:

[0073] The base map acquisition module is used to acquire the interactive base map and distribute and present the interactive base map to the clients of multiple users.

[0074] The operation synchronization module is used to accept multimedia operations and synchronize the multimedia operations to the clients of multiple users.

[0075] The interactive presentation module is used to update and present the interactive background image after the multimedia operation to multiple users.

[0076] In some implementations, the multi-user interaction system may further include:

[0077] The knowledge archiving module is used to archive the interactive base map and multimedia operations to form a knowledge manual, which can be called back by the client.

[0078] As a specific implementation of the above technical solution, this embodiment of the invention also provides an exemplary multi-user interactive webpage platform based on image transmission and multimedia resources. This webpage platform may, for example, belong to the aforementioned multi-user interactive system, and includes:

[0079] The initiating unit is used to accept the initiating operation from the initiating device, establish an interactive room in the web service platform and generate a connection address corresponding to the interactive room, and the receiving device enters the interactive room through the connection address.

[0080] The base map unit is used to accept base map upload operations from the initiating device and / or the receiving device, and to distribute the obtained interactive base map to the initiating device and the receiving device.

[0081] A multimedia operation unit is configured to accept multimedia operations from an initiating device and / or a receiving device, and to synchronize the multimedia operations to the initiating device and the receiving device. The multimedia operations are used to update the content presented on the initiating device and the receiving device.

[0082] In some implementations, the multi-user interactive web platform may further include an archiving unit for receiving archiving operations from the initiating device and / or the receiving device, and for archiving and recording the interactive background map and multimedia operations generated in the interactive room.

[0083] As a very specific implementation example of the present invention, the specific multi-user interaction operation steps can be as follows:

[0084] Step S1: Connect to the user system. After logging in, the user will enter their own live stream record page.

[0085] Step S2: Users can create a live streaming room using the creation function.

[0086] Step S3: Share the live stream link with the people who need to participate in the meeting using the sharing function.

[0087] Step S4: After the participants open the connection, all participants' devices will enter the same live broadcast room.

[0088] Step S5: Anyone can select the image to be discussed from the camera or the local file system by clicking the "Create Image" button.

[0089] Step S6: The system uploads the selected images to the server and distributes them to the interfaces of all people entering the live broadcast room.

[0090] Step S7: All users in the live stream can use features such as drawing, arrows, uploading pictures, videos, manuals, and playing videos and pictures. Every action taken by a user is synchronized to the interface of all users who have joined the live stream via the server.

[0091] Step S8: The initiator can save all the information by clicking the publish button and push it to the server. The server will combine the data into a manual and return the manual link to the initiator.

[0092] Step S9: Anyone who opens the manual link retrieves the data saved at that time from the server, displays it on the page for viewing and use, and can choose to directly import the manual for further interaction in the next interaction.

[0093] Based on the above implementation examples, it is clear that the multi-user interaction method provided by the embodiments of the present invention updates the presented content by transmitting the interactive base map and combining multimedia operations. The content transmission is segmented and does not rely on streaming media to continuously transmit images or audio. This avoids the drawbacks of existing live broadcasts or remote conferences that rely on video streams and / or audio streams. Even in a weak network environment, it can still achieve relatively smoother remote interaction.

[0094] Furthermore, in some more preferred embodiments of the present invention, compared with existing interactive methods such as screen recording or audio recording, the multi-user interaction method provided by the present invention can more conveniently record the segmented interaction process, which is very convenient for review or debriefing. In addition, in conjunction with the above-mentioned interaction method, this form of recording can be recalled and imported in the next interaction, so that multiple interactions can have excellent continuity.

[0095] It should be understood that the above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A multi-user interaction method based on picture transmission and multimedia resources, characterized in that, include: Obtain the interaction base map, and distribute and present the interaction base map to the clients of multiple users, the clients including initiating devices and receiving devices; Accept the initiation operation from the initiating device, establish an interactive room in the web service platform, and generate the connection address corresponding to the interactive room; The receiving device enters the interactive room via a connection address and performs multi-user interaction through a web service platform. Accept multimedia operations and synchronize the multimedia operations to the clients of multiple users. The multimedia operations include any one or a combination of two or more of the following operations: marking and drawing on the interactive base map, and adding pictures, manuals, audio and / or video media. The interactive background image after the multimedia operation is updated and presented to multiple users; Accept the client's save operation and archive the interactive base map and multimedia operations to form a knowledge manual; The knowledge manual is pushed to multiple clients. When multi-user interaction is performed again, the knowledge manual is invoked, and the current multi-user interaction is started based on the newly invoked knowledge manual. Users are divided into different levels: Level 1 users can perform all the multimedia operations, Level 2 users can only doodle or mark and add audio, Level 3 users can only add marks, and Level 4 users cannot perform multimedia operations. When any user at level 1 performs a multimedia operation involving additional voice or video, that user's level is downgraded, and the levels of the remaining users are upgraded by one level accordingly. Except for the initiator, all other users start at the lowest level, and new users join during the interaction process are also at the lowest level by default.

2. The multi-user interaction method of claim 1, wherein, The knowledge manual is pushed out via a link to the manual.

3. The multi-user interaction method of claim 1, wherein, The connection address includes a URL and / or a graphic code.

4. The multi-user interaction method according to claim 1, characterized in that, Specifically, it includes: The multimedia operations are converted and encapsulated into data packets; The data packets are simultaneously sent to the clients of multiple users; The data packet is parsed to reconstruct the multimedia operation represented by the data packet; Based on the restored multimedia operations, the content presented on the client is updated.

5. The multi-user interaction method according to claim 1, characterized in that, Multiple multimedia operations are received from multiple different clients, and the received multimedia operations are simultaneously presented on the clients.

6. A multi-user interaction system based on image transmission and multimedia resources, used to execute the multi-user interaction method according to any one of claims 1-5, characterized in that, include: The base map acquisition module is used to acquire the interactive base map and distribute and present the interactive base map to the clients of multiple users; An operation synchronization module is used to accept multimedia operations and synchronize the multimedia operations to multiple users' clients; The interactive presentation module is used to update and present the interactive background image after the multimedia operation to multiple users. The knowledge archiving module is used to archive the interactive base map and multimedia operations to form a knowledge manual, which can be called back by the client.

7. A multi-user interactive webpage platform based on image transmission and multimedia resources, used to implement the multi-user interaction method according to any one of claims 1-5, characterized in that, include: The initiating unit is used to accept the initiating operation from the initiating device, establish an interactive room in the web service platform and generate a connection address corresponding to the interactive room, and the receiving device enters the interactive room through the connection address; The base map unit is used to accept base map upload operations from the initiating device and / or the receiving device, and to distribute the obtained interactive base map to the initiating device and the receiving device. A multimedia operation unit is configured to receive multimedia operations from an initiating device and / or a receiving device, and to synchronize the multimedia operations to the initiating device and the receiving device. The multimedia operations are used to update the content presented on the initiating device and the receiving device. The archiving unit is used to accept archiving operations from the initiating device and / or the receiving device, and to archive and record the interactive base map and multimedia operations generated in the interactive room.

Images