Live broadcast interaction method and device based on augmented reality, equipment and storage medium
By using augmented reality technology in the live broadcast system, and updating virtual objects based on live broadcast progress and audience interaction data, the problem of insufficient interactivity in traditional live broadcast systems is solved, and personalized interactive experience and high participation are achieved.
Patent Information
- Application Number
- CN202411086446.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-08
AI Technical Summary
Traditional live broadcast systems lack real-time interaction capabilities with the audience, and cannot meet users' needs for a variety of interactive content and personalized experiences, resulting in low audience participation and satisfaction.
By responding to user access instructions, obtaining live streaming data and calling virtual object models in the augmented reality database based on live streaming progress data and audience interaction data, updating virtual objects in combination with device sensor data, superimposing them at the target position, providing personalized Interactive experience.
It greatly improves the audience's sense of participation and interaction, allowing the audience not only to be passive viewers, but also to have an impact on the live broadcast content through interaction, which enhances the interactiveness and user stickiness of the live broadcast.
Smart Images

Figure CN119011894A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of augmented reality technology, and in particular to a live interactive method, device, equipment and storage medium based on augmented reality. Background Art
[0002] Traditional live broadcast systems mainly focus on recording and playing video and audio content, and usually lack the ability to interact with the audience in real time. With the growing demand for remote participation and high-quality interactive experience in education, training, and entertainment, traditional live broadcast systems are insufficient in terms of interactivity. Simple recording and playback functions can no longer meet users' pursuit of rich and diverse interactive content and personalized experience.
[0003] In scenarios such as distance education, corporate training, and online meetings, viewers not only want to watch high-quality video and audio content, but also want to participate in real-time through mobile devices, such as interacting with content, asking questions, and participating in voting. Existing technologies have obvious deficiencies in providing these interactive functions, resulting in low audience participation and satisfaction.
[0004] In summary, the problems existing in the prior art need to be solved urgently. Summary of the invention
[0005] The present invention provides a live interactive method, device, equipment and storage medium based on augmented reality, which are used to solve the defects in the prior art and improve the interactive experience of users.
[0006] The present invention provides a live interactive method based on augmented reality, comprising:
[0007] In response to a user access instruction, acquiring live streaming data, wherein the live streaming data includes live streaming progress data;
[0008] According to the live broadcast progress data and the audience interaction data, calling a virtual object model in an augmented reality database, wherein the augmented reality database is used to store the virtual object model;
[0009] Get device sensor data;
[0010] According to the audience interaction data and the device sensor data, the virtual object model is updated to obtain a live virtual object;
[0011] The live virtual object is superimposed at the target position.
[0012] According to a live interactive method based on augmented reality provided by the present invention, after the step of superimposing the live virtual object at the target position, the method further includes:
[0013] Obtain audience preference data;
[0014] A customized virtual object is generated according to the audience preference data, wherein the customized virtual object includes language subtitles and virtual object annotations.
[0015] According to a live interactive method based on augmented reality provided by the present invention, the step of calling a virtual object model in an augmented reality database according to the live progress data and the audience interaction data specifically includes:
[0016] Determining a model type of the virtual object model according to the live broadcast progress data and the audience interaction data;
[0017] According to the model type, matching is performed in the augmented reality database to obtain a corresponding virtual object model.
[0018] According to a live interactive method based on augmented reality provided by the present invention, the step of updating the live virtual object according to the audience interaction data and the device sensor data specifically includes:
[0019] Determining attribute information of the live broadcast virtual object according to the audience interaction data;
[0020] Determine the location information of the live virtual object according to the device sensor data;
[0021] The live virtual object is updated according to the attribute information and position information of the live virtual object.
[0022] According to a live interactive method based on augmented reality provided by the present invention, the device sensor data also includes ambient lighting data, and after the step of determining the attribute information of the live virtual object according to the audience interaction data, the method further includes:
[0023] Determining the brightness and contrast of the live virtual object according to the ambient lighting data;
[0024] The brightness and contrast of the current device screen are determined according to the ambient light data.
[0025] According to a live interactive method based on augmented reality provided by the present invention, the device sensor data includes rotation rate data, acceleration data and static direction data; the step of determining the position information of the live virtual object according to the device sensor data specifically includes:
[0026] According to the rotation rate data, the acceleration data and the static direction data, the posture of the current device is estimated to determine the motion state of the current device;
[0027] Determining the virtual three-dimensional coordinates of the live virtual object according to the motion state;
[0028] The virtual three-dimensional coordinates are transformed to obtain the two-dimensional coordinates of the device.
[0029] According to a live interactive method based on augmented reality provided by the present invention, the step of superimposing the live virtual object at the target position specifically includes:
[0030] Inputting the live streaming data into an environment understanding model to obtain the target position, wherein the target position is used to overlay the live virtual object;
[0031] Determining the stacking order of the live virtual objects;
[0032] The live virtual object is superimposed on the target position according to the superposition order and the two-dimensional coordinates of the device.
[0033] The present invention also provides a live interactive device based on augmented reality, comprising:
[0034] A data acquisition module, configured to acquire live streaming data in response to a user access instruction, wherein the live streaming data includes live streaming progress data;
[0035] A model calling module, used to call a virtual object model in an augmented reality database according to the live broadcast progress data and the audience interaction data, wherein the augmented reality database is used to store the virtual object model;
[0036] Sensor module, used to obtain device sensor data;
[0037] A model updating module, used for updating the virtual object model according to the audience interaction data and the device sensor data to obtain a live virtual object;
[0038] The model overlay module is used to overlay the live virtual object at the target position.
[0039] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a live interactive method based on augmented reality as described above is implemented.
[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described augmented reality-based live interactive methods.
[0041] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned augmented reality-based live interactive methods.
[0042] The live interactive method, device, equipment and storage medium based on augmented reality provided by the present invention obtain live streaming data by responding to user access instructions, and the live streaming data includes live progress data; according to the live progress data and audience interaction data, call the virtual object model in the augmented reality database, and the augmented reality database is used to store the virtual object model; obtain device sensor data; according to the audience interaction data and the device sensor data, update the virtual object model to obtain a live virtual object; superimpose the live virtual object at the target position. By obtaining live streaming data in response to user access instructions and dynamically adjusting virtual objects according to live progress data and audience interaction data, the audience can obtain a personalized interactive experience during the live broadcast. This augmented reality interactive method greatly improves the audience's sense of participation and interactivity, so that the audience is not only a passive viewer, but can also influence the live content through interaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0044] Figure 1 It is a flow chart of the live interactive method based on augmented reality provided by the present invention;
[0045] Figure 2 It is a structural schematic diagram of a live interactive device based on augmented reality provided by the present invention;
[0046] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] In order to solve the problems in the prior art, the present invention proposes a live interactive method based on augmented reality to improve the interactive experience of users watching live broadcasts. The live interactive method based on augmented reality is described below. Figure 1 As shown, including but not limited to the following steps:
[0049] Step 110: In response to a user access instruction, obtain live streaming data, where the live streaming data includes live streaming progress data.
[0050] In step 110, when a user accesses the live broadcast platform through a mobile device or computer, the system will first receive the user's access instruction. This instruction is usually triggered by the user clicking the "Watch Live" button or scanning the QR code to enter the live broadcast room. After successfully processing the user access, the system obtains the current live broadcast stream data from the live broadcast server or content distribution network (CDN). The live broadcast stream data includes video streams, audio streams, and real-time data related to live broadcasts, such as live broadcast progress information, interactive data, audience statistics, etc. After obtaining the live broadcast stream data, the system further parses the live broadcast progress data therein. The live broadcast progress data includes the time progress of the current live broadcast content, the node information of the live broadcast activity (such as ongoing topic discussions or speech sessions), and related interactive content (such as barrages, real-time voting results, etc.).
[0051] By analyzing the live broadcast progress data, the system can determine the stage of the current live broadcast content and provide users with appropriate synchronization information, such as displaying the live broadcast progress bar, the current topic of discussion, or reminding users whether there is any interactive content that has not been viewed.
[0052] Step 120: Call the virtual object model in the augmented reality database according to the live broadcast progress data and the audience interaction data, and the augmented reality database is used to store the virtual object model.
[0053] In step 120, after obtaining the live streaming data and live progress data, the system will parse these data to determine the theme, progress, important nodes (such as key issues to be discussed, and the interactive session that is about to begin) and other information of the current live content. For example, if the live broadcast is entering an interactive session, such as a question-and-answer session or a voting activity, the system will identify this node and prepare the corresponding virtual object model to enhance the audience's interactive experience. At the same time, the system will collect the audience's interactive data in real time. These data include various forms of audience participation in live broadcast interaction through mobile devices or computers, such as questions raised, voting choices, and barrage messages sent. By analyzing these interactive data, the system can understand the audience's interests and focus, and thus decide which virtual object models need to be called and displayed to the audience. Based on the parsed live progress data and audience interaction data, the system will search and call the corresponding virtual object model in the augmented reality (AR) database. The database stores multiple virtual object models related to different scenes and interaction types, such as 3D charts, virtual characters, prompt boxes, animation effects, etc. For example, in a question-and-answer session, the system can call a virtual object model from the AR database to display the audience's questions and their answers, or display a real-time updated voting results chart in a voting activity. The system will dynamically select the most appropriate virtual object model based on the current live content and the specific situation of audience interaction.
[0054] Step 130: Obtain device sensor data.
[0055] In step 130, the system obtains data from various sensors of the user device, which may include but are not limited to:
[0056] Gyroscope: Provides the device's rotation angle information and is used to detect the device's tilt, rotation, and orientation changes.
[0057] Accelerometer: Measures the linear acceleration of the device and is used to detect the direction and speed of the device's movement.
[0058] Magnetometer: Used to detect the orientation of the device relative to the Earth's magnetic field, thereby determining the orientation of the device.
[0059] Camera: used to capture image data of the device's current environment and provide visual reference for the accurate positioning of virtual objects.
[0060] Light sensor: Detects the light intensity of the environment and is used to adjust the brightness and shadow effects of virtual objects.
[0061] These data will be collected in real time and pre-processed by the sensor data processing module inside the system, such as data denoising, filtering and calibration. The system will synchronize the data from different sensors to ensure that each data point corresponds to the device status at the same time. The synchronized data will be fused to obtain the complete attitude information (position, orientation and motion status) of the device and environmental feature information (lighting, distance, etc.).
[0062] Step 140: Update the virtual object model according to the audience interaction data and the device sensor data to obtain a live virtual object.
[0063] In step 140, after receiving the audience's interactive data, the system analyzes the data to determine the content of the virtual object model that needs to be updated. For example, if the audience asks a new question or initiates a specific interactive request during the live broadcast (such as clicking on a certain area to view detailed information), the system will mark the data as the basis for updating the virtual object. At the same time, the display position, angle and effect of the virtual object model are adjusted in real time using the data obtained from the device sensor. After combining the audience interaction data and sensor data, the system will update the content of the virtual object model. For example, if the audience participates in the vote during the live broadcast, the system will update the display content of the virtual object, such as charts or text information, according to the voting results. In addition, the system may trigger specific animation effects or interactive feedback based on the audience's operation. For example, after the audience clicks a virtual button, the system may display the corresponding extended information or start a new interactive scene. The updated virtual object model will be rendered as a "live virtual object", which combines real-time audience interaction data and device sensor data.
[0064] Through the above implementation methods, the system can flexibly update the virtual object model in real time based on the audience's interaction data and device sensor data, ensuring that the audience is provided with a personalized and highly interactive augmented reality experience during the live broadcast.
[0065] Step 150: superimpose the live virtual object at the target position.
[0066] In step 150, it is first necessary to determine the target position of the live virtual object in the user's view. The determination of the target position depends on multiple factors: device sensor data, real environment characteristics, live progress data, and audience interaction data. After determining the target position, the virtual object needs to be spatially mapped to ensure that its display in the user's view is accurate. If the target position is a dynamically changing point in the user's mobile device view (such as the user's perspective changes as the device moves), the system will adjust the mapping of the virtual object in real time to keep it accurately aligned with the target position. The processed and rendered live virtual object will be superimposed on the display screen of the user's device. The audience can see the combination of the virtual object and the real environment through the device screen, and interact or adjust the perspective according to their needs.
[0067] Through the above steps, the system successfully superimposes the live virtual object accurately at the target position, ensuring that the display of the virtual object in the user's view is closely integrated with the real environment, thereby providing the audience with a natural and interactive augmented reality experience.
[0068] The live interactive method, device, equipment and storage medium based on augmented reality provided by the present invention obtain live streaming data by responding to user access instructions, and the live streaming data includes live progress data; according to the live progress data and audience interaction data, call the virtual object model in the augmented reality database, and the augmented reality database is used to store the virtual object model; obtain device sensor data; according to the audience interaction data and the device sensor data, update the virtual object model to obtain a live virtual object; superimpose the live virtual object at the target position. By obtaining live streaming data in response to user access instructions and dynamically adjusting virtual objects according to live progress data and audience interaction data, the audience can obtain a personalized interactive experience during the live broadcast. This augmented reality interactive method greatly improves the audience's sense of participation and interactivity, so that the audience is not only a passive viewer, but can also influence the live content through interaction.
[0069] As a further optional embodiment, after the step of superimposing the live virtual object at the target position, the method further includes:
[0070] Obtain audience preference data;
[0071] A customized virtual object is generated according to the audience preference data, wherein the customized virtual object includes language subtitles and virtual object annotations.
[0072] In this embodiment, the personalized preference data of the audience is obtained from the audience's device or account. These data include the audience's selected language, viewing habits, subtitle preferences, interest in specific content (such as preferring technical details or concise summaries), and previous interaction records. The audience preference data can be actively set by the audience during the live broadcast or automatically learned and recorded by the system. For example, the system can derive the audience's preferences by analyzing the audience's past interactive behavior or questionnaire feedback. Based on the audience preference data, the system generates customized virtual objects to meet the specific needs of the audience. For example: Based on the audience preference data, the system generates customized virtual objects to meet the specific needs of the audience. For specific content or details that the audience is interested in, the system can generate virtual objects with annotations. These annotations may include text explanations, background information, related links, or multimedia resources so that the audience can get more information when watching the live broadcast. For example, in an educational live broadcast, the audience may see annotations superimposed next to the explanation content, providing more background knowledge or examples. During the live broadcast, the audience's preferences may change, and the system will monitor these changes in real time and make corresponding adjustments to the customized virtual objects.
[0073] By introducing the steps of obtaining audience preference data and generating customized virtual objects, the present invention further enhances the personalized experience of augmented reality interaction, so that each audience can obtain live broadcast content that is highly consistent with their needs and interests, thereby enhancing the interactivity and user stickiness of the live broadcast.
[0074] As a further optional embodiment, the step of calling the virtual object model in the augmented reality database according to the live broadcast progress data and the audience interaction data specifically includes:
[0075] Determining a model type of the virtual object model according to the live broadcast progress data and the audience interaction data;
[0076] According to the model type, matching is performed in the augmented reality database to obtain a corresponding virtual object model.
[0077] In this embodiment, the type of virtual object model to be called is first determined based on the live broadcast progress data and the audience interaction data. This includes the theme of the live broadcast content, the currently playing content segment, scene changes, etc. For example, in a teaching live broadcast, the appropriate virtual object model type, such as a chart, 3D model, or text label, etc., can be determined based on the chapter or topic currently being explained. This includes the audience's interactive behavior during the live broadcast, such as questions raised, votes participated in, or options selected. For example, if the audience shows special concern for a certain issue, a more detailed virtual object model type may be selected, such as a model with explanatory text or multimedia.
[0078] According to the determined model type, the system matches in the augmented reality database to find the virtual object model corresponding to this type. Various types of virtual object models are pre-stored in the database, including but not limited to 3D models, animations, charts, text labels, video clips, etc. Each model type has multiple instances, which can be selected according to the specific live content or interactive needs. Accurate matching will be performed based on the specific requirements of the model type, such as the size, display effect, and interactive capabilities of the model, to ensure that the selected virtual object model is consistent with the current live content and audience interaction needs. For example, if the live progress data indicates that the current content is an explanation of a complex mechanical structure, the system will match the corresponding 3D mechanical model.
[0079] As a further optional embodiment, the step of updating the live virtual object according to the audience interaction data and the device sensor data specifically includes:
[0080] Determining attribute information of the live broadcast virtual object according to the audience interaction data;
[0081] Determine the location information of the live virtual object according to the device sensor data;
[0082] The live virtual object is updated according to the attribute information and position information of the live virtual object.
[0083] In this embodiment, the audience interaction data is first analyzed to determine the attribute information that needs to be updated for the live virtual object. The audience interaction data may include the following:
[0084] Audience click behavior: If the audience clicks on a virtual object, the system needs to update the display effect or status of the object, such as displaying additional information or triggering animation effects.
[0085] Viewer Choice: Preferences expressed by viewers through voting or selection features can affect the properties of virtual objects, such as adjusting the content’s level of detail or switching view modes.
[0086] Interaction requests: Questions or requests raised by viewers may require marking on virtual objects or providing relevant instructions, such as adding text annotations or links to virtual objects.
[0087] Attribute information: Attribute information can include the color, size, animation status, display text, interactive functions, etc. of the virtual object. For example, the audience may request to change the color of the virtual object or add new annotation information.
[0088] In addition, the exact position of the virtual object in the user's view is determined through the device's sensor data (such as gyroscope, accelerometer, camera, etc.). The information provided by the device sensor data includes:
[0089] Device posture: By obtaining the rotation angle and motion state of the device through the gyroscope and accelerometer, the system can determine the relative position of the virtual object in the three-dimensional space.
[0090] View and orientation: The camera’s image of the environment and the device’s viewing orientation can help the system determine the precise position and angle of virtual objects.
[0091] Environmental changes: Device sensors can also monitor lighting changes, spatial transformations, etc. to ensure that the display of virtual objects remains consistent in the real environment.
[0092] Based on the attribute information and location information obtained in the above steps, the system updates the live virtual object in real time to ensure that its display effect is consistent with the audience's interactive needs and the actual situation of the device.
[0093] As a further optional embodiment, the device sensor data also includes ambient lighting data, and after the step of determining the attribute information of the live virtual object according to the audience interaction data, the step further includes:
[0094] Determining the brightness and contrast of the live virtual object according to the ambient lighting data;
[0095] The brightness and contrast of the current device screen are determined according to the ambient light data.
[0096] In this embodiment, the ambient light data in the device sensor data is used to measure the light conditions of the environment the audience is in. The ambient light data is usually provided by a light sensor of the device, reflecting the light intensity and light changes of the current environment.
[0097] Based on the ambient lighting data, the brightness of virtual objects can be adjusted to ensure that they are clearly visible in various lighting conditions. If the ambient light intensity is high, the system will increase the brightness of the virtual objects; if the ambient light is low, the brightness will be reduced to maintain the visibility of the virtual objects.
[0098] Adjust the brightness of the current device screen based on the ambient lighting data to match the ambient lighting conditions. Doing so ensures that virtual objects displayed on the screen have appropriate brightness and maintain visual comfort under different ambient lighting conditions.
[0099] It is understandable that the contrast can also be adjusted. After determining the brightness and contrast of the virtual object and the brightness and contrast of the device screen, the system will update the display settings of the virtual object. By introducing the processing of ambient lighting data, this embodiment further enhances the visual adaptability and display quality of the virtual object, so that the virtual object can maintain a clear and natural display under various lighting conditions. This method effectively improves the overall quality of the augmented reality experience and enhances the audience's sense of immersion in different environments.
[0100] As a further optional embodiment, the device sensor data includes rotation rate data, acceleration data and static direction data; the step of determining the position information of the live virtual object according to the device sensor data specifically includes:
[0101] According to the rotation rate data, the acceleration data and the static direction data, the posture of the current device is estimated to determine the motion state of the current device;
[0102] Determining the virtual three-dimensional coordinates of the live virtual object according to the motion state;
[0103] The virtual three-dimensional coordinates are transformed to obtain the two-dimensional coordinates of the device.
[0104] In this embodiment, posture estimation is performed based on the rotation rate data, acceleration data, and static direction data to determine the motion state of the device. This includes: using the rotation rate data to calculate the rotation angle of the device on each axis; using the acceleration data to calculate the speed change and acceleration state of the device; and determining the overall direction of the device in combination with the static direction data. Based on the motion state of the device, the system will determine the virtual coordinates of the live virtual object in three-dimensional space. The virtual three-dimensional coordinates represent the position of the virtual object in the device view, and usually include three coordinate values of X, Y, and Z. The device posture estimation results are applied to the position calculation of the virtual object to ensure that the coordinates of the virtual object are consistent with the actual motion state and direction of the device. For example, when the device moves forward, the position of the virtual object is adjusted accordingly.
[0105] Next, the virtual 3D coordinates are converted to 2D coordinates on the device screen. When performing the coordinate transformation, the actual position of the virtual object on the screen is calculated, taking into account the perspective effect and the device's viewing angle. The perspective transformation takes into account the device's viewing angle and distance, ensuring that the display position of the virtual object on the screen matches its position in 3D space. The resulting 2D device coordinates are applied to the display settings of the virtual object, ensuring that the position of the virtual object on the device screen accurately reflects its position in 3D space.
[0106] Through these steps, the system can accurately calculate the 3D position of virtual objects based on device sensor data and accurately map them to 2D coordinates on the device screen. This method ensures that the position and display of virtual objects in the augmented reality environment are consistent with the audience's actual perspective and the motion state of the device, enhancing the accuracy and immersion of the interactive experience.
[0107] As a further optional embodiment, the step of superimposing the live virtual object at the target position specifically includes:
[0108] Inputting the live streaming data into an environment understanding model to obtain the target position, wherein the target position is used to overlay the live virtual object;
[0109] Determining the stacking order of the live virtual objects;
[0110] The live virtual object is superimposed on the target position according to the superposition order and the two-dimensional coordinates of the device.
[0111] In this embodiment, the data obtained from the live stream is input into the environment understanding model. The environment understanding model can be an image processing model based on deep learning, which is used to analyze the environmental information in the live content. The environment understanding model processes the live stream data to identify and determine the target position. The target position refers to the specific position used to overlay the virtual object in the live screen. For example, the model can identify a specific area or object in the live content as the overlay target position of the virtual object. Then, according to the preset rules or the requirements of the live content, the overlay order of the virtual objects is determined. For example, the priority of the overlay can be arranged according to the importance, interactivity or logical order of the virtual object. An overlay priority is assigned to each virtual object to ensure that the virtual objects are displayed in the correct order during the live broadcast. This helps to optimize the audience's viewing experience and avoid visual conflicts between virtual objects. According to the determined overlay order, the virtual objects are overlaid one by one on the target position. Each virtual object is overlaid at the target position according to its priority order to ensure that it is correctly displayed in the live content.
[0112] The following is a description of the live interactive device based on augmented reality provided by the present invention. Figure 2 As shown, the live interactive device based on augmented reality described below and the live interactive method based on augmented reality described above can refer to each other.
[0113] A live interactive device based on augmented reality, comprising:
[0114] The data acquisition module 210 is used to acquire live streaming data in response to a user access instruction, wherein the live streaming data includes live streaming progress data;
[0115] A model calling module 220, configured to call a virtual object model in an augmented reality database according to the live broadcast progress data and the audience interaction data, wherein the augmented reality database is configured to store the virtual object model;
[0116] Sensor module 230, used to obtain device sensor data;
[0117] A model updating module 240, configured to update the virtual object model according to the audience interaction data and the device sensor data to obtain a live virtual object;
[0118] The model overlay module 250 is used to overlay the live virtual object at the target position.
[0119] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the live interactive method based on augmented reality, and the method includes:
[0120] In response to a user access instruction, acquiring live streaming data, wherein the live streaming data includes live streaming progress data;
[0121] According to the live broadcast progress data and the audience interaction data, calling a virtual object model in an augmented reality database, wherein the augmented reality database is used to store the virtual object model;
[0122] Get device sensor data;
[0123] According to the audience interaction data and the device sensor data, the virtual object model is updated to obtain a live virtual object;
[0124] The live virtual object is superimposed at the target position.
[0125] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-On l yMemory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.
[0126] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the live interactive method based on augmented reality provided by the above methods, the method includes:
[0127] In response to a user access instruction, acquiring live streaming data, wherein the live streaming data includes live streaming progress data;
[0128] According to the live broadcast progress data and the audience interaction data, calling a virtual object model in an augmented reality database, wherein the augmented reality database is used to store the virtual object model;
[0129] Get device sensor data;
[0130] According to the audience interaction data and the device sensor data, the virtual object model is updated to obtain a live virtual object;
[0131] The live virtual object is superimposed at the target position.
[0132] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the live interactive method based on augmented reality provided by the above methods, the method comprising:
[0133] In response to a user access instruction, acquiring live streaming data, wherein the live streaming data includes live streaming progress data;
[0134] According to the live broadcast progress data and the audience interaction data, calling a virtual object model in an augmented reality database, wherein the augmented reality database is used to store the virtual object model;
[0135] Get device sensor data;
[0136] According to the audience interaction data and the device sensor data, the virtual object model is updated to obtain a live virtual object;
[0137] The live virtual object is superimposed at the target position.
[0138] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0139] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A live interactive method based on augmented reality, characterized in that: include: In response to a user access instruction, acquiring live streaming data, wherein the live streaming data includes live streaming progress data; According to the live broadcast progress data and the audience interaction data, calling a virtual object model in an augmented reality database, wherein the augmented reality database is used to store the virtual object model; Get device sensor data; According to the audience interaction data and the device sensor data, the virtual object model is updated to obtain a live virtual object; The live virtual object is superimposed at the target position.
2. The live interactive method based on augmented reality according to claim 1, characterized in that: After the step of superimposing the live virtual object at the target position, the method further includes: Obtain audience preference data; A customized virtual object is generated according to the audience preference data, wherein the customized virtual object includes language subtitles and virtual object annotations.
3. The live interactive method based on augmented reality according to claim 1, characterized in that: The step of calling the virtual object model in the augmented reality database according to the live broadcast progress data and the audience interaction data specifically includes: Determining a model type of the virtual object model according to the live broadcast progress data and the audience interaction data; According to the model type, matching is performed in the augmented reality database to obtain a corresponding virtual object model.
4. The live interactive method based on augmented reality according to claim 1, characterized in that: The step of updating the live virtual object according to the audience interaction data and the device sensor data specifically includes: Determining attribute information of the live broadcast virtual object according to the audience interaction data; Determine the location information of the live virtual object according to the device sensor data; The live virtual object is updated according to the attribute information and position information of the live virtual object.
5. The live interactive method based on augmented reality according to claim 4, characterized in that: The device sensor data also includes ambient lighting data. After the step of determining the attribute information of the live virtual object according to the audience interaction data, the method further includes: Determining the brightness and contrast of the live virtual object according to the ambient lighting data; The brightness and contrast of the current device screen are determined according to the ambient light data.
6. The live interactive method based on augmented reality according to claim 4, characterized in that: The device sensor data includes rotation rate data, acceleration data and static direction data; the step of determining the position information of the live virtual object according to the device sensor data specifically includes: According to the rotation rate data, the acceleration data and the static direction data, the posture of the current device is estimated to determine the motion state of the current device; Determining the virtual three-dimensional coordinates of the live virtual object according to the motion state; The virtual three-dimensional coordinates are transformed to obtain the two-dimensional coordinates of the device.
7. The live interactive method based on augmented reality according to claim 1, characterized in that: The step of superimposing the live virtual object at the target position specifically includes: Inputting the live streaming data into an environment understanding model to obtain the target position, wherein the target position is used to overlay the live virtual object; Determining the stacking order of the live virtual objects; The live virtual object is superimposed on the target position according to the superposition order and the two-dimensional coordinates of the device.
8. A live interactive device based on augmented reality, characterized in that: include: A data acquisition module, configured to acquire live streaming data in response to a user access instruction, wherein the live streaming data includes live streaming progress data; A model calling module, used to call a virtual object model in an augmented reality database according to the live broadcast progress data and the audience interaction data, wherein the augmented reality database is used to store the virtual object model; Sensor module, used to obtain device sensor data; A model updating module, used for updating the virtual object model according to the audience interaction data and the device sensor data to obtain a live virtual object; The model overlay module is used to overlay the live virtual object at the target position.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the live interactive method based on augmented reality is implemented as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the live interactive method based on augmented reality is implemented as described in any one of claims 1 to 7.
Citation Information
Patent Citations
AR-augmented game live method and system
CN109195020A
Live broadcast interaction method and device, electronic equipment and storage medium
CN110519611A
Virtual live broadcast system and method
CN112291576A
Live broadcast control method and device, computer equipment and storage medium
CN114401415A
Live broadcast interaction method and device, equipment and storage medium
CN117082285A
Cited By
Operation guidance method, device and equipment for self-service equipment
CN121305738A
