Panoramic screen projection control method and device, computer equipment and storage medium

Abstract

The invention relates to a panoramic screen projection control method and device, computer equipment and a storage medium. The method is applied to camera equipment in a screen projection system and comprises the steps that in response to detected state notification information, a simple object access protocol message which is sent by terminal equipment and contains panoramic control function parameter information is received, and the state notification information represents that connection is established among the terminal equipment, the camera equipment and display equipment; analyzing the simple object access protocol message, and determining a panoramic control function and a panoramic control parameter indicated by the panoramic control function parameter information; and determining target panoramic data displayed by the display device, and adjusting the target panoramic data based on the panoramic control function and the panoramic control parameter. By adopting the method, when the panoramic view angle is operated through the terminal equipment, equipment connection details do not need to be concerned, the process can automatically trigger instruction transmission and picture adjustment after connection confirmation, finally, the adjustment result is displayed on the display equipment in real time, and the camera equipment does not need to be used again for adjustment.

Description

Technical Field

[0001] This disclosure relates to the field of screen projection technology, and in particular to a panoramic screen projection control method, device, computer equipment, and storage medium. Background Technology

[0002] With the development of screen projection technology, more and more application scenarios will use panoramic screen projection technology. Traditional panoramic screen projection technology has many limitations. When users view panoramic images, they usually adjust the viewing angle through the camera and the terminal device adjusts the panoramic image to be displayed, resulting in complicated operation and poor experience. Summary of the Invention

[0003] Therefore, it is necessary to provide a panoramic projection control method, device, computer equipment, and storage medium to address the aforementioned technical problems.

[0004] Firstly, this disclosure provides a panoramic projection control method. The method applies to a camera device in a projection system, which further includes a terminal device and a display device.

[0005] In response to the detection of status notification information, a Simple Object Access Protocol message containing panoramic control function parameter information sent by the terminal device is received, wherein the status notification information indicates that a connection has been established between the terminal device, the camera device, and the display device;

[0006] Parse the Simple Object Access Protocol message to determine the panoramic control function and panoramic control parameters indicated by the panoramic control function parameter information;

[0007] The target panoramic data to be displayed on the display device is determined, and the target panoramic data is adjusted based on the panoramic control function and panoramic control parameters. The adjusted target panoramic data is then used for display on the display device.

[0008] In one embodiment, the method further includes:

[0009] The device description document of the camera device is determined, and for each predetermined preset function, definition parameters of the preset function are added to the device description document. The definition parameters include: function name, input parameters, parameter type and format.

[0010] In one embodiment, before receiving the Simple Object Access Protocol (SOP) message containing panoramic control function parameter information sent by the terminal device, the method further includes: sending the device description document to the terminal device to instruct the terminal device to generate a SOP message containing panoramic control function parameter information based on the defined parameters in the device description document;

[0011] The panoramic control function includes: rotating the viewing angle; the panoramic control parameters include: rotation angle and rotation direction; the rotation direction includes: longitude direction, latitude direction, and planar rotation direction; adjusting the target panoramic data based on the panoramic control function and panoramic control parameters includes:

[0012] Determine the camera coordinate system of the camera device;

[0013] The camera coordinate system is adjusted based on one or more of the longitude direction, latitude direction, and planar rotation direction, and according to the corresponding rotation angle;

[0014] The target panoramic data is projected according to the adjusted camera coordinate system.

[0015] In one embodiment, the method further includes:

[0016] In response to an audio recording command, the device receives audio data recorded during the response process of the audio recording command, which is sent by the terminal device via the HTTP protocol.

[0017] The display device outputs an adjustment command to control the video playback progress of the target panoramic data during the display process;

[0018] The target panoramic video is adjusted based on the adjustment instructions, and the audio data is time-axis aligned with the adjusted target panoramic data to generate a video recording.

[0019] Secondly, this disclosure also provides a panoramic projection control method, applied to a terminal device in a projection system, wherein the projection system further includes: a camera device and a display device, and the method includes:

[0020] Output a search request, and in response to receiving a status notification information that matches the search request and in response to screen touch information input by the terminal device, determine the panoramic control function parameter information;

[0021] A Simple Object Access Protocol (SOP) message carrying the panoramic control function parameter information is sent to the camera device. The SOP message is used to instruct the camera device to determine the target panoramic data, panoramic control function, and panoramic control parameters, and to adjust the target panoramic data according to the panoramic control function and panoramic control parameters.

[0022] Obtain metadata of the adjusted target panoramic data from the camera device, generate a display link based on the metadata, and use the display link to instruct the display device to display the adjusted target panoramic data.

[0023] In one embodiment, the method further includes:

[0024] In response to an audio recording command, audio data is captured during the response process of the audio recording command using the microphone in the terminal device;

[0025] The audio data is sent to the camera device via the HTTP protocol.

[0026] Thirdly, this disclosure also provides a panoramic projection control device. The device is used in a projection system with a camera, and the projection system further includes a terminal device and a display device.

[0027] The information receiving module is used to receive a Simple Object Access Protocol message containing panoramic control function parameter information sent by the terminal device in response to the detection of status notification information. The status notification information indicates that a connection has been established between the terminal device, the camera device, and the display device.

[0028] The information parsing module is used to parse the Simple Object Access Protocol message and determine the panoramic control function and panoramic control parameters indicated by the panoramic control function parameter information.

[0029] The data adjustment module is used to determine the target panoramic data to be displayed by the display device, and adjust the target panoramic data based on the panoramic control function and panoramic control parameters. The adjusted target panoramic data is used for display by the display device.

[0030] Fourthly, this disclosure also provides a panoramic projection control device. This device is applied to a terminal device in a projection system, which further includes a camera device and a display device.

[0031] The information determination module is used to output a search request, and in response to receiving status notification information that matches the search request and in response to screen touch information input by the terminal device, determine the panoramic control function parameter information.

[0032] The information sending module is used to send a Simple Object Access Protocol (SOP) message carrying the panoramic control function parameter information to the camera device. The SOP message is used to instruct the camera device to determine the target panoramic data, panoramic control function and panoramic control parameters, and to adjust the target panoramic data according to the panoramic control function and panoramic control parameters.

[0033] A link generation module is used to obtain metadata of the adjusted target panoramic data from the camera device, and generate a display link based on the metadata. The display link is used to instruct the display device to display the adjusted target panoramic data.

[0034] Fifthly, this disclosure also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in any of the above method embodiments.

[0035] Sixthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in any of the above method embodiments.

[0036] In a seventh aspect, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in any of the method embodiments described above.

[0037] In the above embodiments, subsequent interactions are triggered by "status notification information" to ensure that panoramic control command transmission only begins after the terminal device, camera device, and display device have successfully established a connection, avoiding command loss or execution failure due to incomplete connection. This "connection confirmed before operation" mechanism ensures the reasonable timing of interactions between devices and reduces invalid communication. When users operate the panoramic view through terminal devices (such as mobile phones), they do not need to pay attention to device connection details. The process automatically triggers command transmission and image adjustment after "connection confirmation," and finally presents the adjustment results in real time on the display device, eliminating the need for further adjustments using the camera device. This "end-to-end" automated processing reduces the complexity of user operations, making panoramic control more intuitive and convenient. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a panoramic projection control method in one embodiment;

[0040] Figure 2 This is a schematic diagram of the camera spherical coordinate system in one embodiment;

[0041] Figure 3 This is a flowchart illustrating step S106 in one embodiment;

[0042] Figure 4 This is a flowchart illustrating the audio and video recording steps in one embodiment;

[0043] Figure 5 This is a flowchart illustrating another panoramic projection control method in one embodiment;

[0044] Figure 6 This is a schematic block diagram of the panoramic projection control device in one embodiment;

[0045] Figure 7 This is a schematic block diagram of another panoramic projection control device in one embodiment;

[0046] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0048] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings herein are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0049] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0050] In one embodiment, such as Figure 1 As shown, a panoramic projection control method is provided. This method is applied to a camera device in a projection system. Besides the camera device, the projection system also includes a terminal device and a display device. The method includes:

[0051] S102, in response to detecting status notification information, receive a Simple Object Access Protocol message containing panoramic control function parameter information sent by the terminal device, wherein the status notification information indicates that a connection has been established between the terminal device, the camera device and the display device.

[0052] Status notification information typically refers to notification messages generated when a connection is established between devices. These messages inform other devices that "the terminal device, camera device, and display device have successfully established a network connection," serving as a prerequisite for triggering subsequent interactions. For example, status notification information can be a NOTIFY message, a notification message used to proactively push information or state changes, commonly found in protocols such as UPnP (Universal Plug and Play), used by devices or services to proactively announce their status, events, or existence to other nodes in the network. The terminal device can be a mobile phone, tablet, or other smart device, acting as a control point responsible for discovering devices and sending commands. The camera device can provide a video source (accessing video files stored in the camera), receive audio streams and control commands from the terminal device, process them, and transmit the video stream to the display device. The display device, acting as a media rendering device (DMR), receives and displays the video stream transmitted by the camera device and responds to playback control commands (such as pause and volume adjustment). Panoramic control function parameter information typically refers to information or data for adjusting the panoramic video, such as adjusting the viewing angle. Simple Object Access Protocol (SOAP) messages are typically SOAP messages. SOAP is an XML-based communication protocol used to exchange structured data between different applications on a network. It defines the data format, transmission method, and interaction rules, enabling applications running on different operating systems and using different programming languages ​​to communicate with each other.

[0053] Specifically, after the terminal device, camera device, and display device complete device discovery and establish a connection via the SSDP protocol, a "status notification message" (such as a NOTIFY message or connection confirmation signal sent between devices) is generated. Upon detecting this status notification message, the terminal device confirms that the three devices have successfully connected and then generates a SOAP message containing panoramic control function parameters (for example, after the user rotates the viewpoint on the terminal device, the terminal device encapsulates parameters such as rotation angle and rotation axis into SOAP format). Subsequently, the terminal device sends this SOAP message to the camera device via the HTTP protocol.

[0054] S104, parse the Simple Object Access Protocol message to determine the panoramic control function and panoramic control parameters indicated by the panoramic control function parameter information.

[0055] Panorama control functions typically include viewpoint zoom and rotation. Panorama control parameters may include zoom level and rotation angle. Target panorama data usually refers to the panorama media data specified in the SOAP message that requires control operations; this data is typically stored in the camera device.

[0056] Specifically, the camera device can call the corresponding processing function to parse the Simple Object Access Protocol (SOP) message, obtain the XML structure in the SOP message, and thus obtain the panoramic control function parameter information.

[0057] S106, determine the target panoramic data to be displayed by the display device, adjust the target panoramic data based on the panoramic control function and panoramic control parameters, and use the adjusted target panoramic data for display by the display device.

[0058] Specifically, the camera device can determine the panoramic data currently stored in the panorama data that the display device is currently displaying; this panoramic data can be the target panoramic data. Based on the parsed panoramic control functions and parameters, the camera device adjusts the angle of view, zoom level, and other parameters of the target panoramic video to obtain the adjusted target panoramic data. The display device can then read the adjusted target panoramic data from the camera device, thereby achieving the effect of controlling the panoramic data on the display device.

[0059] In the aforementioned panoramic projection control method, subsequent interactions are triggered by "status notification information," ensuring that panoramic control command transmission only begins after the terminal device, camera device, and display device have successfully established a connection. This avoids command loss or execution failure due to incomplete connection. This "connection confirmed before operation" mechanism ensures the logical timing of interactions between devices and reduces invalid communication. When users operate the panoramic view through terminal devices (such as mobile phones), they do not need to pay attention to device connection details. The process automatically triggers command transmission and image adjustment after "connection confirmation," ultimately presenting the adjustment results in real time on the display device, eliminating the need for further adjustments using the camera device. This end-to-end automated processing reduces user operational complexity, making panoramic control more intuitive and convenient.

[0060] In one embodiment, the method further includes:

[0061] The device description document of the camera device is determined, and for each predetermined preset function, definition parameters of the preset function are added to the device description document. The definition parameters include: function name, input parameters, parameter type and format.

[0062] The device description document (XML device description file) serves as an "identity statement" for the camera device, showcasing its capabilities to other devices on the network (such as terminal devices and display devices). It includes basic device information, supported standard services, and preset function entries (such as rotation view and zoom). Preset functions may include rotation view and zoom. Defined parameters are typically the core information describing the preset functions. The function name can be a unique identifier (e.g., "view rotation" corresponds to SetViewOrientation); input parameters are usually the specific parameters required by the function (e.g., TargetAngle can represent "rotation angle," and Axis can represent "rotation axis"); parameter types and formats typically specify the data type (e.g., TargetAngle is a floating-point number) and format specifications (e.g., angle range, optional axis values).

[0063] Specifically, the camera device first needs to prepare a basic XML device description document, including its identity information as a standard DMS (Digital Media Server) (such as device name and model), supported standard UPnP / DLNA services (such as ContentDirectorY service), and service-related URLs (control URLs, event URLs, etc.). The preset functions that the camera device needs to support should be clearly defined, including two parts: standard functions: such as media file browsing and transfer (already defined in the basic XML document, these functions do not need to be added or adjusted); custom functions: such as non-standard operations like "view rotation," "start / stop recording," and "zoom," these are functions that need to be added. For each custom preset function, a corresponding function entry should be added to the device description document, and the parameters should be defined according to the specifications: for example, for the "view rotation" function, define: Function name: SetViewOrientation; Input parameters: TargetAngle (rotation angle), Axis (rotation axis); Parameter type and format: TargetAngle is a floating-point number (e.g., -180° to 180°), Axis is a string (optional values ​​"longitude," "latitude," "plane rotation"). Ensure these definitions conform to the UPnP protocol specification so that the terminal device can recognize and understand these functions when parsing the document, thereby assigning corresponding parameters to these functions. The modified device description document is stored by the camera device and made available externally (e.g., for retrieval by other devices via an HTTP endpoint). When the terminal device discovers the camera device, it retrieves the document via HTTP, parses out the definition parameters of all preset functions, and then generates SOAP instructions that conform to the specified format (such as view rotation instructions containing TargetAngle and Axis), enabling precise control of the camera device.

[0064] In this embodiment, by uniformly adding the definition parameters (function name, input parameters, parameter type, and format) of preset functions to the device description document, the camera device can clearly "declare" to the terminal device all the functions it supports and the calling rules. For example, the "view rotation" function is defined as SetViewOrientation in the document, with input parameters TargetAngle (floating-point number) and Axis (string, optional "longitude," etc.). This ensures that the terminal device can accurately identify these functions when parsing the document, avoiding misunderstandings or call failures caused by ambiguous function descriptions, and providing a "common language" for cross-device interaction. When the camera device needs to add a preset function (such as "image cropping," "filter switching," etc.), it only needs to add the definition parameters of the function in a uniform format to the device description document, and the terminal device can automatically identify the new function and adapt to the call by parsing the document. This document-based function declaration method does not require modification of the underlying protocol framework, making the expansion of device functions more flexible. It ensures the standardization of control commands and reduces interaction errors. The explicit "parameter type and format" in the definition parameters (such as limiting TargetAngle to floating-point number and Axis to optional values) can constrain the control commands sent by the terminal device to conform to the standard. For example, if the terminal device mistakenly passes the "longitude" parameter as an integer, the camera device can reject or correct the error based on the document definition, avoiding abnormal function execution due to parameter errors (such as invalid view rotation angle), thus improving the stability and reliability of the interaction.

[0065] In one embodiment, before receiving the Simple Object Access Protocol (SAP) message containing panoramic control function parameter information sent by the terminal device, the method further includes: sending the device description document to the terminal device to instruct the terminal device to generate a SAP message containing panoramic control function parameter information based on the defined parameters in the device description document.

[0066] Specifically, after a camera device starts up or connects to the network, it broadcasts its presence via the SSDP protocol (e.g., by sending a NOTIFY message). When a terminal device discovers the camera device via an M-SEARCH request, it sends an HTTP request to the camera device to obtain a device description document. The camera device responds to this request by sending an XML-formatted device description document containing preset function definition parameters (such as the function name for "view rotation," input parameters, etc.) to the terminal device via HTTP. When a user performs panoramic control operations on the terminal device (such as sliding to adjust the view), the terminal device, based on the parsed definition parameters, converts the user's operation into formatted panoramic control function parameter information (such as a rotation angle of 30.5 degrees and a rotation axis of "longitude"), and encapsulates these parameters into a SOAP message (XML format) according to the SOAP protocol specification, containing the corresponding function name and parameter values. The terminal device then sends the generated SOAP message to the corresponding service control endpoint of the camera device to control the panoramic data.

[0067] The panoramic control function includes: rotating the viewing angle; the panoramic control parameters include: rotation angle and rotation direction; the rotation direction includes: longitude direction, latitude direction, and planar rotation direction, such as... Figure 2 The diagram shows the camera's spherical coordinate system OXYZ. Longitude: This corresponds to the rotation direction around the Y-axis in the camera coordinate system (OXYZ). The rotation angle λ around the Y-axis can be used as the longitude, with a value range of [-180°, 180°]. When the camera receives a rotation command in the "longitude" direction, it will rotate the Y-axis of the camera coordinate system in the opposite direction to adjust the horizontal direction of the field of view (e.g., rotating the view left or right), ensuring that the Z-axis points to the center of the user's field of view after the operation. Latitude: This corresponds to the rotation direction around the Z-axis in the camera coordinate system. The rotation angle Φ around the Z-axis is the latitude, with a value range of [-90°, 90°]. When the camera receives a rotation command in the "latitude" direction, it will rotate the Z-axis of the camera coordinate system in the opposite direction to adjust the vertical direction of the field of view (e.g., rotating the view up or down), ensuring that the Z-axis is always aligned with the center of the user's field of view after the operation. Planar rotation direction: This corresponds to the rotation direction along the Z-axis in the camera coordinate system. When the camera receives the "plane rotation" command, it will rotate the camera coordinate system along the Z-axis at a specified angle (similar to clockwise / counterclockwise rotation in a plane), so that the image rotates as a whole on the horizontal plane, and finally remaps the projected image according to the adjusted coordinate system.

[0068] like Figure 3 As shown, adjusting the target panoramic data based on the panoramic control function and panoramic control parameters includes:

[0069] S202, Determine the camera coordinate system of the camera device.

[0070] S204, adjust the camera coordinate system based on one or more of the longitude direction, latitude direction and plane rotation direction, and according to the corresponding rotation angle.

[0071] S206 Project the target panoramic data according to the adjusted camera coordinate system.

[0072] Specifically, the camera device initializes and defines its own camera coordinate system—typically with the origin as the camera's center of symmetry, the Y-axis as the axis of symmetry, and the initial Z-axis pointing to the default field of view center (e.g., directly in front), forming a spatial reference to determine the camera device's coordinate system. The camera device receives SOAP messages from the terminal device and parses out the direction to be adjusted (longitude, latitude, or planar rotation, one or more) and the corresponding rotation angle (e.g., "rotate 20° in longitude" or "rotate 15° in planar rotation"). If longitude is involved: the camera coordinate system's Y-axis is rotated in the opposite direction according to the parsed rotation angle (e.g., if the user command is "increase longitude by 20°", the coordinate system rotates 20° in the opposite direction around the Y-axis), making the Z-axis point to the new horizontal field of view center. If latitude is involved: the camera coordinate system's Z-axis is rotated in the opposite direction according to the rotation angle (e.g., if the user command is "increase latitude by 10°", the coordinate system rotates 10° in the opposite direction around the Z-axis), making the Z-axis point to the new vertical field of view center. If planar rotation is involved: the camera rotates around the coordinate system's Z-axis (initially directly in front) according to the rotation angle, adjusting the orientation of the image in the plane (e.g., rotate 15° clockwise). If multiple directional combinations exist (such as adjusting longitude and latitude simultaneously), the rotation operations are performed sequentially to update the coordinate system. The camera device performs spatial projection calculations on the original target panoramic data (the panoramic video stream captured by the camera) according to the adjusted camera coordinate system, that is, adjusts the target panoramic data according to the coordinate axis directions of the new coordinate system.

[0073] In this embodiment, the abstract operations of the user on "longitude (left and right), latitude (up and down), and plane rotation (image turning)" are transformed into quantitative adjustments to the camera coordinate system (such as rotating 20° in the opposite direction around the Y-axis and 15° around the Z-axis) through the process of "first determining the coordinate system → then adjusting by direction and angle → finally projecting the data." This coordinate system-based "precise calculation + data projection" mode avoids problems such as viewpoint shift and image stretching. Compared to "pixel-level processing" such as cropping and rotating the panoramic video stream, the method of adjusting the camera coordinate system and then projecting the data only requires spatial mapping calculation based on coordinate system parameters (coordinate axis direction and rotation angle) to extract the target field of view from the original panoramic data. This "coordinate system-centric" processing logic does not require repeated modification of the video stream itself, reducing the amount of data computation (such as avoiding pixel-by-pixel rotation of multiple frames).

[0074] In one embodiment, such as Figure 4 As shown, the method further includes:

[0075] S302, in response to the recording instruction, receive audio data recorded during the recording instruction response process sent by the terminal device, wherein the audio data is sent by the terminal device via the HTTP protocol.

[0076] S304, Obtain the adjustment command output by the display device during the process of displaying the target panoramic data to control the video playback progress of the target panoramic data.

[0077] S306, adjust the target panoramic video based on the adjustment instruction, and align the audio data with the adjusted target panoramic data on the timeline to generate a video recording.

[0078] Among them, audio and video recording commands refer to the control commands sent by the terminal device to the camera device to trigger the recording function. Timeline alignment is usually based on the timestamps synchronized by the NTP protocol, which precisely matches the playback time points of the audio data with the adjusted target panoramic data to ensure the synchronization of the picture and sound.

[0079] Specifically, the camera device can receive "audio / video recording instructions" (such as a SOAP "start recording" instruction) sent by the terminal device, and then begin preparations for audio and video recording and compositing. After sending the audio / video recording instruction, the terminal device captures audio in real time through its microphone and generates audio data; simultaneously, the terminal device can also continuously send this audio data to the camera device via HTTP POST or PUT requests, and the camera device continuously receives and caches this audio data. When the display device plays the target panoramic data transmitted by the camera device, if the terminal device sends standard playback control instructions (such as Pause, Seek, etc. SOAP messages) to the display device, the display device will perform corresponding operations (such as pausing playback, skipping progress), and simultaneously output "adjustment instructions" (such as "pause current playback progress" or "skip progress to 1 minute 30 seconds"); the camera device obtains these adjustment instructions in real time by listening to the event notifications from the display device (based on the UPnP event subscription mechanism), avoiding the loss of playback control operation records during recording. Then, based on the received adjustment instructions, the camera device updates the playback progress of its cached target panoramic data (e.g., if a "fast forward 10 seconds" instruction is received, the playback time of the target panoramic video is moved forward by 10 seconds), ensuring that the adjusted video progress matches the actual playback progress of the display device. The camera device extracts the timestamps of the audio data and the adjusted target panoramic data, and precisely aligns the two along the timeline (e.g., matching the "00:01:05" time point of the audio data with the "00:01:05" frame of the video data), avoiding audio-visual asynchrony. The camera device encodes the timeline-aligned audio data and the adjusted target panoramic data (including the complete video stream containing playback control operation records) (e.g., compressing it into a standard video format), ultimately generating a "recorded video" containing the complete video and synchronized audio, completing the entire audio and video recording process.

[0080] In this embodiment, adjustment instructions (control data) and audio / video data (media data) are processed separately, and audio is transmitted using a separate HTTP protocol. This ensures the speed of audio data transmission, reduces interference between different data transmissions, and also reduces the complexity of data interaction between devices and network bandwidth usage.

[0081] In one embodiment, this disclosure also provides a panoramic projection control method applied to a terminal device in a projection system, wherein the projection system further includes a camera device and a display device, such as... Figure 5 As shown, the method includes:

[0082] S402, output a search request, and in response to receiving a status notification information matching the search request and in response to the screen touch information input by the terminal device, determine the panoramic control function parameter information.

[0083] Search requests typically refer to messages sent by terminal devices to discover other devices (such as camera devices and display devices) in the network. These requests can be implemented based on the SSDP (Simple Service Discovery Protocol) in the UPnP protocol and are usually broadcast as M-SEARCH messages to probe information such as device type and service capabilities. Screen touch information typically refers to user interactions on the terminal device screen (such as swiping left and right, up and down, and rotation gestures) to trigger adjustments to the panoramic view (such as "rotate the view to the left" or "zoom in").

[0084] Specifically, after the terminal device starts up, it broadcasts a search request (M-SEARCH message) to the local area network via the SSDP protocol. This message contains the target device type (e.g., "camera device," "display device") or service type (e.g., DLNA's DMS, DMR services) to discover interactive devices on the network. Upon receiving the search request, if the camera and display devices match the request, they send a status notification to the terminal device via an SSDP NOTIFY message, containing key information such as their device description document URL, service type, and network address. The terminal device receives and parses these notifications, confirming that the camera and display devices have been discovered and a connection can be established (completing the device discovery process). When the user interacts with the terminal device's screen (e.g., two-finger swipe to adjust the viewing angle, one-finger rotation of the screen), the terminal device's operating system captures this touch information, identifying the operation type (e.g., horizontal swipe corresponds to rotation in the longitude direction, vertical swipe corresponds to rotation in the latitude direction) and the operation amplitude (e.g., swipe distance mapped to rotation angle). According to the preset mapping rules, the terminal device converts the screen touch information into standardized panoramic control function parameters: for example, when the user slides the screen to the right (horizontal direction), it is recognized as the "longitude direction rotation" function, with the sliding distance corresponding to TargetAngle=30° (rotation angle) and Axis=longitude (rotation axis).

[0085] S404, a Simple Object Access Protocol (SOP) message carrying the panoramic control function parameter information is sent to the camera device. The SOP message is used to instruct the camera device to determine the target panoramic data, panoramic control function, and panoramic control parameters, and to adjust the target panoramic data according to the panoramic control function and panoramic control parameters.

[0086] For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0087] S406, Obtain metadata of the adjusted target panoramic data from the camera device, and generate a display link based on the metadata. The display link is used to instruct the display device to display the adjusted target panoramic data.

[0088] Metadata typically consists of structured information describing the attributes of the adjusted target panoramic data. In a document context, this usually includes: media file name, storage path, format (e.g., MP4), resolution, duration, access URL (Uniform Resource Locator), timestamp, etc., used to identify and locate the panoramic data. Display links are usually standardized access identifiers (usually in URL form) generated based on metadata, pointing to the adjusted target panoramic data. They contain information such as the network address and resource path required for the display device to obtain the data, and are the key "index" that triggers playback on the display device.

[0089] Specifically, the terminal device calls the camera device's ContentDirectorY service to send a Browse or Search operation request, specifying the query conditions (such as data identifier ID) for "obtaining adjusted target panoramic data." The camera device responds to the request, returning the metadata of the data to the terminal device via the HTTP protocol. The terminal device extracts the core information from the metadata, namely the "access URL of the adjusted target panoramic data," and encapsulates it into a display link according to the DLNA / UPnP protocol specification. For example, the HTTP URL in the metadata can be directly used as the display link to ensure that the display device can directly access the target data through this link. The terminal device calls the display device's AVTransport service, sends a SetAVTransportURI operation command, and passes the display link as the "media URI" parameter, informing the display device of the target panoramic data address to be played; then it sends a Play operation, triggering the display device to obtain and display the adjusted target panoramic data from the camera device through the display link. The ContentDirectorY service is an important service defined in the DLNA (Digital Living Network Alliance) protocol, part of the UPnP AV (Audio / Video) framework, mainly used for managing and sharing media content between devices within a local area network. Browse and Search requests are two core data query commands initiated by terminal devices to media servers (such as cameras storing panoramic data or media library devices) to obtain information about target resources. The AVTransport service is one of the core media control services, primarily used to manage and control the playback status of audio, video, and panoramic media resources between playback devices (such as display devices, audio systems, and televisions) and media servers (such as cameras, computers, and NAS). It is a key component for cross-device media collaboration. SetAVTransportURI is the core operation command of the AVTransport service in the UPnP protocol. Its main function is to transmit the "address of the media resource to be played" and associated metadata to the playback device (such as a display device or television), allowing the playback device to clarify "what resource to play" and "the basic information of that resource," serving as a "prerequisite" for cross-device media playback.

[0090] In this embodiment, the process, from the terminal device sending a search request to discover devices, to receiving status notifications to confirm the connection, and then to generating control parameters, sending SOAP commands, and generating a display link to trigger the display, forms a complete closed loop of "device linkage - command transmission - screen presentation". This avoids fragmented interactions between devices (such as individual device discovery operations or manual input of control commands). Relying on standardized UPnP / DLNA protocols (such as SSDP discovery, SOAP commands, and AVTransport services), the terminal device acts as a "central hub" to seamlessly connect the camera device (processing data) and the display device (presenting the screen), reducing manual intervention steps by the user and ensuring a smooth process from "user touch operation" to "display device screen update", thus improving the fluency of multi-device collaboration.

[0091] In one embodiment, the method further includes:

[0092] In response to an audio recording command, audio data is captured during the response process of the audio recording command using the microphone in the terminal device;

[0093] The audio data is sent to the camera device via the HTTP protocol.

[0094] For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0095] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0096] Based on the same inventive concept, this disclosure also provides a panoramic projection control device for implementing the panoramic projection control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more panoramic projection control device embodiments provided below can be found in the limitations of the panoramic projection control method described above, and will not be repeated here.

[0097] In one embodiment, such as Figure 6As shown, a panoramic projection control device 500 is provided, applied to a camera device in a projection system. The projection system further includes a terminal device and a display device, comprising: an information receiving module 502, an information parsing module 504, and a data adjustment module 506, wherein:

[0098] Information receiving module 502 is used to receive a Simple Object Access Protocol message containing panoramic control function parameter information sent by the terminal device in response to the detection of status notification information, wherein the status notification information indicates that a connection has been established between the terminal device, the camera device and the display device;

[0099] Information parsing module 504 is used to parse the Simple Object Access Protocol message and determine the panoramic control function and panoramic control parameters indicated by the panoramic control function parameter information.

[0100] The data adjustment module 506 is used to determine the target panoramic data to be displayed by the display device, and adjust the target panoramic data based on the panoramic control function and panoramic control parameters. The adjusted target panoramic data is used for display by the display device.

[0101] In one embodiment of the device, the device further includes:

[0102] The function addition module is used to determine the device description document of the camera device, and add definition parameters of the preset function to the device description document based on each preset function. The definition parameters include: function name, input parameters, parameter type and format.

[0103] In one embodiment of the device, the device further includes: a document sending module, configured to send the device description document to the terminal device, to instruct the terminal device to generate a Simple Object Access Protocol (SAP) message containing panoramic control function parameter information based on the defined parameters in the device description document. The panoramic control function includes: a rotation viewpoint, and the panoramic control parameters include: a rotation angle and a rotation direction, the rotation direction including: longitude direction, latitude direction, and planar rotation direction. The data adjustment module 506 includes:

[0104] A coordinate system determination module is used to determine the camera coordinate system of the camera device;

[0105] The coordinate system adjustment module is used to adjust the camera coordinate system based on one or more of the longitude direction, latitude direction, and planar rotation direction, and according to the corresponding rotation angle;

[0106] The data projection module is used to project the target panoramic data according to the adjusted camera coordinate system.

[0107] In one embodiment of the device, the device further includes:

[0108] An audio data receiving module is used to receive audio data recorded during the recording instruction response process sent by the terminal device in response to the recording instruction, wherein the audio data is sent by the terminal device via the HTTP protocol;

[0109] The adjustment instruction generation module is used to obtain the adjustment instruction output by the display device during the display of the target panoramic data to control the video playback progress of the target panoramic data;

[0110] The video recording generation module is used to adjust the target panoramic video based on the adjustment instructions, and to align the audio data with the adjusted target panoramic data on the timeline to generate a video recording.

[0111] In one embodiment, such as Figure 7 As shown in the figure, this disclosure also provides a panoramic projection control device 600, applied to a terminal device in a projection system. The projection system further includes: a camera device and a display device. The device includes:

[0112] The parameter information determination module 602 is used to output a search request, and in response to receiving status notification information that matches the search request and in response to screen touch information input by the terminal device, determine the panoramic control function parameter information.

[0113] The target panoramic data determination module 604 is used to send a Simple Object Access Protocol (SAP) message carrying the panoramic control function parameter information to the camera device. The SAP message is used to instruct the camera device to determine the target panoramic data, panoramic control function, and panoramic control parameters, and to adjust the target panoramic data according to the panoramic control function and panoramic control parameters.

[0114] Display link generation module 606 is used to obtain metadata of the adjusted target panoramic data in the camera device, generate display links based on the metadata, and instruct the display device to display the adjusted target panoramic data.

[0115] In one embodiment of the device, the method further includes:

[0116] An audio data transmission module is used to respond to a recording command by capturing audio data during the response process of the recording command using the microphone in the terminal device; and to send the audio data to the camera device via the HTTP protocol.

[0117] Each module in the aforementioned panoramic projection control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0118] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a panoramic projection control method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0119] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the computer device to which the present disclosure is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0120] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0121] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0122] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0123] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this disclosure may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this disclosure may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the appended claims.

Claims

1. A panoramic screen projection control method, characterized in that, A camera device applied to a screen projection system, the screen projection system further comprising a terminal device and a display device, the method comprising: in response to detecting state notification information, receiving a simple object access protocol message containing panorama control function parameter information sent by the terminal device, the state notification information indicating that a connection is established between the terminal device, the camera device and the display device; parsing the simple object access protocol message to determine the panorama control function and the panorama control parameter indicated by the panorama control function parameter information; determining target panorama data displayed by the display device, and adjusting the target panorama data based on the panorama control function and the panorama control parameter, the adjusted target panorama data being used for display by the display device.

2. The method of claim 1, wherein, The method further comprises: determining a device description document of the camera device, and adding definition parameters of each predetermined preset function in the device description document based on the preset function, the definition parameters including function name, input parameter, parameter type and format.

3. The method of claim 2, wherein, Before the receiving the simple object access protocol message containing panorama control function parameter information sent by the terminal device, the method further comprises: sending the device description document to the terminal device to instruct the terminal device to generate a simple object access protocol message containing panorama control function parameter information based on the definition parameters in the device description document; the panorama control function includes rotating a viewing angle, the panorama control parameter includes a rotation angle and a rotation direction, the rotation direction includes a longitude direction, a latitude direction and a plane rotation direction; and the adjusting the target panorama data based on the panorama control function and the panorama control parameter comprises: determining a camera coordinate system of the camera device; adjusting the camera coordinate system according to the corresponding rotation angle based on one or more of the longitude direction, the latitude direction and the plane rotation direction; projecting the target panorama data according to the adjusted camera coordinate system.

4. The method of claim 1, wherein, The method further comprises: in response to a recording instruction, receiving audio data recorded during a response process of the recording instruction sent by the terminal device, the audio data being sent by the terminal device through an HTTP protocol; obtaining an adjustment instruction for controlling a video playback progress of the target panorama data output by the display device during display of the target panorama data; adjusting the target panorama video based on the adjustment instruction, time-shaft aligning the audio data with the adjusted target panorama data, and generating a video recording.

5. A panoramic screen projection control method, characterized by, A terminal device applied to a screen projection system, the screen projection system further comprising a camera device and a display device, the method comprising: outputting a search request, determining panorama control function parameter information in response to receiving state notification information matching the search request, and in response to screen touch information input by the terminal device; send a simple object access protocol message carrying the panoramic control function parameter information to the camera device, the simple object access protocol message being used to instruct the camera device to determine target panoramic data, panoramic control function and panoramic control parameter, and adjust the target panoramic data according to the panoramic control function and the panoramic control parameter; obtain metadata of the adjusted target panoramic data in the camera device, and generate a display link based on the metadata, the display link being used to instruct the display device to display the adjusted target panoramic data.

6. The method of claim 5, wherein, The method further comprises: in response to the audio / video recording instruction, capturing audio data in a response process of the audio / video recording instruction based on a microphone in the terminal device; sending the audio data to the camera device through an HTTP protocol.

7. A panoramic screen projection control device, characterized by, The camera device applied to the screen projection system further comprises a terminal device and a display device, and the device comprises: an information receiving module, configured to receive a simple object access protocol message containing panoramic control function parameter information sent by the terminal device in response to detection of state notification information, the state notification information representing establishment of a connection among the terminal device, the camera device and the display device; an information analyzing module, configured to analyze the simple object access protocol message to determine panoramic control function and panoramic control parameter indicated by the panoramic control function parameter information; a data adjusting module, configured to determine target panoramic data displayed by the display device, and adjust the target panoramic data based on the panoramic control function and the panoramic control parameter, the adjusted target panoramic data being used to be displayed by the display device.

8. A panoramic screen projection control device, characterized by, The terminal device applied to the screen projection system further comprises a camera device and a display device, and the device comprises: an information determining module, configured to output a search request, determine panoramic control function parameter information in response to reception of state notification information matching the search request, and in response to screen touch information input by the terminal device; an information sending module, configured to send a simple object access protocol message carrying the panoramic control function parameter information to the camera device, the simple object access protocol message being used to instruct the camera device to determine target panoramic data, panoramic control function and panoramic control parameter, and adjust the target panoramic data according to the panoramic control function and the panoramic control parameter; a link generating module, configured to obtain metadata of the adjusted target panoramic data in the camera device, and generate a display link based on the metadata, the display link being used to instruct the display device to display the adjusted target panoramic data. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 4 or any one of claims 5 or 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 4 or any one of claims 5 or 6.

Images