High-speed low-consumption cooperative transmission method and system between AI glasses and smart phone

Abstract

The invention discloses a high-speed low-consumption cooperative transmission method and system between AI glasses and a smart phone, and relates to the technical field of data transmission, and the method comprises the steps: constructing a first scene background model based on a first scene data packet through an AI glasses terminal; performing differential identification on a real-time scenarized semantic data packet collected by the AI glasses terminal to obtain a first differential transmission data packet corresponding to the foreground change area; extracting a user visual focus area of the AI glasses terminal, and performing hierarchical collaborative coding on the first differential transmission data packet to obtain a second differential transmission data packet; and transmitting the second differential transmission data packet to the smart phone terminal for storage and updating. The technical problems that in the prior art, core information concerned by a user cannot be intelligently distinguished and preferentially transmitted, the data transmission efficiency is low, and the power consumption is high are solved, and the technical effects that the transmission data volume, the equipment power consumption and the transmission delay are reduced, and the transmission quality of key information and the user experience are guaranteed are achieved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of data transmission, and in particular to a high-speed and low-consumption collaborative transmission method and system between AI glasses and a smart phone. BACKGROUND

[0002] AI smart glasses provide users with an immersive experience of seamlessly integrating the real world and digital information. The core capability thereof depends on real-time perception of the surrounding environment and information superposition. However, AI glasses have significant hardware limitations in computing power, storage space and battery endurance. AI glasses usually work in collaboration with a smart phone to provide high-quality and low-latency AR services. The AI glasses serve as a lightweight perception and display front end, responsible for collecting high-dimensional environmental data. The smart phone serves as a powerful back-end processing unit, responsible for complex scene understanding, model reasoning and data fusion. How to achieve high-speed and low-consumption data transmission between the AI glasses and the smart phone becomes a key issue. Traditional continuous video stream transmission has a huge amount of data, which can quickly consume the limited battery of the AI glasses and occupy a large amount of wireless bandwidth, resulting in high transmission delay and user experience lag. Moreover, motion detection or simple inter-frame difference cannot effectively distinguish meaningful foreground changes from meaningless light changes, leaf swaying and other disturbances, resulting in transmission of excessive redundant data. In addition, the user's intention is not taken into account.

[0003] Therefore, in the related art, there is a technical problem that the core information of user interest cannot be intelligently distinguished and preferentially transmitted, and the data transmission efficiency is low and the power consumption is high. SUMMARY

[0004] The application provides a high-speed and low-consumption collaborative transmission method and system between AI glasses and a smart phone, which solves the technical problem in the prior art that the core information of user interest cannot be intelligently distinguished and preferentially transmitted, and the data transmission efficiency is low and the power consumption is high, and achieves the technical effects of reducing the amount of transmission data, device power consumption and transmission delay, and guaranteeing the transmission quality of key information and user experience.

[0005] The application provides a high-speed and low-consumption collaborative transmission method between AI glasses and a smart phone, which comprises the following steps: constructing a first scene background model based on a first scene data packet through an AI glasses terminal; performing differential identification on real-time scene semantic data packets collected by the AI glasses terminal based on the first scene background model, to obtain a first differential transmission data packet corresponding to a foreground change area; extracting a user visual focus area of the AI glasses terminal, and performing hierarchical collaborative coding on the first differential transmission data packet according to the user visual focus area, to obtain a second differential transmission data packet; and transmitting the second differential transmission data packet to the smart phone terminal for storage and update.

[0006] In a possible implementation, the high-speed and low-consumption collaborative transmission method between the AI glasses and the smart phone further performs the following processing: collecting an overlap change area and a non-overlap change area of the user visual focus area and the foreground change area; configuring a first quality parameter and a second quality parameter, wherein the encoding quality of the first quality parameter is greater than the encoding quality of the second quality parameter; encoding the overlap change area based on the first quality parameter to obtain a first encoded data packet; encoding the non-overlap change area based on the second quality parameter to obtain a second encoded data packet; and updating the first differential transmission data packet by using the first encoded data packet and the second encoded data packet to obtain a second differential transmission data packet.

[0007] In a possible implementation, the high-speed and low-consumption collaborative transmission method between the AI glasses and the smart phone further performs the following processing: collecting an overlap change area and a non-overlap change area of the user visual focus area and the foreground change area; encoding the overlap change area based on a first quality parameter to obtain a first encoded data packet; identifying semantic change information of the non-overlap change area to obtain a semantic data packet, wherein the semantic data packet includes object category identification, spatial position coordinates, and action state description; and updating the first differential transmission data packet by using the first encoded data packet and the semantic data packet to obtain a second differential transmission data packet.

[0008] In a possible implementation, the high-speed and low-consumption collaborative transmission method between the AI glasses and the smart phone further performs the following processing: defining a quality parameter item, wherein the quality parameter includes resolution, code rate target, frame rate, and quantization parameter; defining a first quality parameter solution space and a second quality parameter solution space according to the quality parameter item; selecting a first initial quality parameter and a second initial quality parameter in the first quality parameter solution space and the second quality parameter solution space, and sending the first initial quality parameter and the second initial quality parameter to an encoding quality scoring model; performing iterative analysis of encoding quality scoring with a difference constraint condition that the encoding quality of the first quality parameter is greater than the encoding quality of the second quality parameter to obtain a first quality parameter constraint and a second quality parameter constraint; and performing Bayesian optimization search according to the first quality parameter constraint and the second quality parameter constraint to obtain a first quality parameter and a second quality parameter.

[0009] In a possible implementation, the high-speed, low-power collaborative transmission method between the AI ​​glasses and the smartphone further performs the following processing: acquiring a training dataset, which includes encoded data packet samples obtained by encoding scene data packet samples using different quality parameters based on the first quality parameter solution space and the second quality parameter solution space; calculating the peak signal-to-noise ratio, encoding efficiency index, and decoding accuracy for each encoded data packet sample, and calculating the encoding quality score label of the encoded data packet sample by weighted information entropy; and training a regression model using the encoding quality score labels of the corresponding encoded data packets under different quality parameters to obtain a converged encoding quality score model.

[0010] In a possible implementation, the high-speed, low-power collaborative transmission method between the AI ​​glasses and the smartphone further performs the following processing: obtaining a predicted coding quality score for the first initial quality parameter and the second initial quality parameter based on the coding quality scoring model; if the predicted coding quality score does not meet a preset coding quality score threshold, calculating the expected gain based on the first quality parameter constraint and the second quality parameter constraint, and using the expected gain factor to identify the selection probability of the first quality parameter solution space and the second quality parameter solution space; obtaining the first candidate quality parameter and the second candidate quality parameter again according to the selection probability identification, and so on, until the number of iterations reaches a preset threshold and the search stops, thus obtaining the first quality parameter and the second quality parameter.

[0011] In a possible implementation, the high-speed, low-power collaborative transmission method between the AI ​​glasses and the smartphone further performs the following processing: A transmission channel is set up between the AI ​​glasses terminal and the smartphone terminal. The transmission channel includes a first transmission channel and a second transmission channel. The first transmission channel is configured using encoding compatibility parameters, and the second transmission channel is configured using semantic compatibility parameters. After the first differential transmission data packet is hierarchically co-coded according to the user's visual focus area, the transmission channel is used to transmit the second differential transmission data packet in parallel. The first transmission channel is used for transmitting encoded data packets, and the second transmission channel is used for transmitting semantic data packets.

[0012] In a possible implementation, the high-speed, low-power collaborative transmission method between the AI ​​glasses and the smartphone further performs the following processing: identifying semantic change information of the overlapping change region according to a first semantic granularity to obtain a first semantic data packet; identifying semantic change information of the non-overlapping change region according to a second semantic granularity to obtain a second semantic data packet, wherein the first semantic granularity is smaller than the second semantic granularity; updating the first differential transmission data packet based on the first encoded data packet, the first semantic data packet, and the second semantic data packet to obtain a second differential transmission data packet.

[0013] This application also provides a high-speed, low-power collaborative transmission system between AI glasses and a smartphone. The system includes: a scene construction module, used to construct a first scene background model based on a first scene-specific data packet through the AI ​​glasses terminal; a differential recognition module, used to perform differential recognition on the real-time scene-specific semantic data packet collected by the AI ​​glasses terminal based on the first scene background model, and obtain a first differential transmission data packet corresponding to the foreground change region; a hierarchical co-coding module, used to extract the user's visual focus region of the AI ​​glasses terminal, and perform hierarchical co-coding on the first differential transmission data packet according to the user's visual focus region to obtain a second differential transmission data packet; and a storage update module, used to transmit the second differential transmission data packet to the smartphone terminal for storage update.

[0014] This application proposes a high-speed, low-power collaborative transmission method and system between AI glasses and a smartphone. The method involves: constructing a first scene background model based on a first scene-specific data packet using the AI ​​glasses terminal; performing differential recognition on real-time scene-specific semantic data packets collected by the AI ​​glasses terminal to obtain a first differential transmission data packet corresponding to the foreground change region; extracting the user's visual focus region from the AI ​​glasses terminal; performing hierarchical collaborative encoding on the first differential transmission data packet to obtain a second differential transmission data packet; and transmitting the second differential transmission data packet to the smartphone terminal for storage updates. This solves the technical problems of existing technologies, such as the inability to intelligently distinguish and prioritize the transmission of core information of interest to the user, low data transmission efficiency, and high power consumption. It achieves the technical effect of reducing the amount of transmitted data, device power consumption, and transmission latency, while ensuring the transmission quality of key information and improving user experience. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0016] Figure 1 This is a schematic diagram of the high-speed, low-power collaborative transmission method between AI glasses and a smartphone provided in an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of the high-speed, low-power collaborative transmission system between AI glasses and a smartphone provided in an embodiment of this application.

[0018] Figure labeling: Scene construction module 10, differential recognition module 20, hierarchical co-coding module 30, storage update module 40. Detailed Implementation

[0019] To further illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose, the following detailed description is provided in conjunction with the accompanying drawings and preferred embodiments, based on the specific implementation methods, structures, features and effects of the present invention.

[0020] This application provides a method for high-speed, low-power collaborative transmission between AI glasses and smartphones, such as... Figure 1 As shown, the method includes: Step S100: Construct a first scene background model based on the first scene-based data package through the AI ​​glasses terminal.

[0021] Preferably, the AI ​​glasses terminal acquires and processes initial static scene data through its image sensor, inertial measurement unit (IMU), and processor to obtain a first scene-based data package. This package contains semantic and structural information about the scene, which may include, but is not limited to, one or more frames of RGB images, pixel-level depth information obtained through sensors such as stereo vision or ToF, image key points and their descriptive vectors extracted using algorithms such as ORB and SIFT, and static semantic segmentation maps identifying the sky, buildings, roads, etc., through preliminary segmentation and labeling of the image using a lightweight neural network. Then, a first scene background model is constructed based on the first scene-based data package. That is, the AI ​​smart glasses' processor utilizes the static scene data in the first scene-based data package... Invariant element data representation may include using a Gaussian mixture model to probabilistically model the color value of each pixel in the image to describe the most likely background color for that point across multiple frames of observation, thereby distinguishing rare foreground color values; representing the scene as a stable set of feature points and their three-dimensional spatial locations using a feature-based model; dividing the scene into multiple grids using a semantic grid model that combines depth information and semantic segmentation, and assigning each grid a static background label and three-dimensional coordinates; and then generating a first scene background model as a benchmark for efficient comparison and differential calculation with real-time data frames to quickly identify foreground regions that have changed relative to this benchmark, thereby achieving a significant reduction in the amount of data transmitted.

[0022] Step S200: Based on the first scene background model, perform differential recognition on the real-time scene-based semantic data packets collected by the AI ​​glasses terminal to obtain the first differential transmission data packets corresponding to the foreground change region.

[0023] Preferably, AI glasses are used to continuously collect scene data from the latest frame or a time period to determine real-time scene-based semantic data packages containing preliminary semantic information, such as RGB image frames with timestamps and depth maps, semantic feature maps extracted by a lightweight neural network, and object detection boxes or preliminary classification labels. Then, differential recognition is performed on the real-time scene-based semantic data packages based on a first scene background model. This involves comparing the real-time scene data with the scene background model to identify regions with significant differences. Specifically, if the background model is a Gaussian mixture model, differential recognition determines whether the color value of each pixel in the real-time frame conforms to the probability distribution of the background model, and identifies pixels with probabilities below a set threshold as foreground. If the background model is a feature-based point cloud model, differential recognition compares newly extracted feature points in the real-time frame with features in the background model. The system performs point matching, identifying newly emerging feature points that fail to match as foreground change regions. If the background model is a semantic grid model, differential recognition checks whether objects enter the grid region marked as static background in the semantic segmentation results of the real-time frame. Then, objects or parts of the scene that are dynamic, newly appearing, or moving relative to the background, identified through differential recognition, are determined as foreground change regions, such as walking people, passing vehicles, or a user picking up a cup. Finally, the first differential transmission data packet corresponding to the foreground change region is obtained, containing only structured data different from the scene background model. This includes at least the image pixel block corresponding to the foreground change region extracted from the real-time frame, the coordinate information of the image pixel block in the foreground region, and the timestamp and sequence number. This ensures that all static background information in the scene is filtered out, and only the dynamically changing parts are extracted.

[0024] Step S300: Extract the user's visual focus area of ​​the AI ​​glasses terminal, and perform hierarchical collaborative coding on the first differential transmission data packet according to the user's visual focus area to obtain the second differential transmission data packet.

[0025] Step S300 further includes step S310, acquiring overlapping and non-overlapping change areas between the user's visual focus area and the foreground change area; step S320, configuring a first quality parameter and a second quality parameter, wherein the encoding quality of the first quality parameter is greater than the encoding quality of the second quality parameter; step S330, encoding the overlapping change area based on the first quality parameter to obtain a first encoded data packet; step S340, encoding the non-overlapping change area based on the second quality parameter to obtain a second encoded data packet; and step S350, updating the first differential transmission data packet with the first encoded data packet and the second encoded data packet to obtain a second differential transmission data packet.

[0026] Preferably, the user's visual focus area of ​​the AI ​​glasses terminal is obtained and the first differential transmission data packet is hierarchically and collaboratively encoded. Specifically, through the eye-tracking sensor built into the AI ​​glasses, based on the infrared camera and the pupil-corneal reflection algorithm, the area in the screen or real space where the user's gaze is focused at the current moment is obtained, and the user's visual focus area is determined. It is usually an elliptical or polygonal area centered on the gaze point. Then, the spatial intersection of the user's visual focus area and the foreground change area is calculated to obtain the overlapping change area and the non-overlapping change area. The overlapping change area refers to the part that is both within the user's visual focus area and belongs to the foreground change area, representing the change part that the user is looking at, and has the highest importance. The non-overlapping change area refers to the part that is within the foreground change area but outside the user's visual focus area, representing the change part that the user sees in the peripheral vision or has not noticed, and has lower importance. Then configure the first quality parameter and the second quality parameter. The quality parameter refers to the key configuration items used by the digital video and image encoder to control the compression quality and file size, including resolution, bitrate target, frame rate, and quantization parameter. The encoding quality under the first quality parameter must be greater than the encoding quality under the second quality parameter. The first quality parameter is set to high encoding quality, such as high resolution, high bitrate, and low quantization parameter value, for overlapping variation areas. The second quality parameter is set to low encoding quality, such as low resolution, low bitrate, and high quantization parameter value, for non-overlapping variation areas.

[0027] Preferably, image data from overlapping and varying regions is input into a standard video / image encoder and compressed using a first quality parameter, outputting a first encoded data packet. This first encoded data packet is relatively large in size but has high fidelity. Similarly, image data from non-overlapping and varying regions is input into a standard video / image encoder and compressed using a second quality parameter, outputting a second encoded data packet. This second encoded data packet is significantly compressed but suffers from some distortion. Then, the corresponding parts of the first differential transmission data packet are replaced and updated with the first and second encoded data packets. This is lossy compression based on human visual characteristics, ultimately generating a second differential transmission data packet with visual attention. Its internal data structure may remain unchanged, but the encoding quality and data volume have been optimized and reorganized according to the user's focus area. By significantly compressing non-focus areas, a second sharp reduction in data transmission volume is achieved with minimal impact on the user's subjective perception, thereby achieving the goal of high-speed, low-power collaborative transmission.

[0028] Furthermore, step S300 also includes: collecting overlapping and non-overlapping change regions between the user's visual focus region and the foreground change region; encoding the overlapping change region based on a first quality parameter to obtain a first encoded data packet; identifying semantic change information of the non-overlapping change region to obtain a semantic data packet, wherein the semantic data packet includes an object category identifier, spatial location coordinates, and an action state description; and updating the first differential transmission data packet with the first encoded data packet and the semantic data packet to obtain a second differential transmission data packet.

[0029] Preferably, after the foreground region is segmented, a more aggressive and efficient implementation method is adopted for hierarchical collaborative coding. The core is to transmit pixel streams to the focal region and information streams to the non-focal region. Specifically, overlapping and non-overlapping changing regions between the user's visual focal region and the foreground changing region are collected. Using an image / video encoder and based on a first quality parameter with high resolution, high bit rate, and low quantization values, the visual pixel information of the overlapping changing region is compressed into a first encoded data packet. This first encoded data packet contains binary blocks of compressed visual data, used to reconstruct the image of the overlapping changing region at the smartphone receiver. Advanced semantic understanding and structured data extraction are performed on the non-overlapping changing region, including lightweight object detection. AI models for image classification and behavior recognition analyze image data from non-overlapping change areas, extracting key structured semantic information and encapsulating it into semantic data packets. These packets include object category identifiers, spatial coordinates, and action state descriptions, such as object category labels or IDs, spatial coordinates, and action state descriptions. The original first differential transmission data packet is then disassembled. The overlapping change area portion is replaced with a high-quality first encoded data packet, and the non-overlapping change area portion is replaced with a semantic data packet describing its semantic information. This process ultimately generates a second differential transmission data packet, which contains both encoded data for reconstructing the focus area and pure semantic data for describing events in non-focus areas.

[0030] Furthermore, step S300 also includes: identifying semantic change information of the overlapping change region according to a first semantic granularity to obtain a first semantic data packet; identifying semantic change information of the non-overlapping change region according to a second semantic granularity to obtain a second semantic data packet, wherein the first semantic granularity is smaller than the second semantic granularity; updating the first differential transmission data packet based on the first encoded data packet, the first semantic data packet, and the second semantic data packet to obtain a second differential transmission data packet.

[0031] Preferably, after encoding the overlapping change region based on the first quality parameter and obtaining the first encoded data packet, multi-granularity semantics is introduced to achieve fine-grained control over the description of scene information. Semantic granularity refers to the level of detail in describing objects / events in the scene; fine granularity indicates a specific and detailed description, while coarse granularity indicates a general and general description. Then, semantic change information of the overlapping change region is identified according to the first semantic granularity, i.e., a fine-grained identification model is used to analyze the overlapping change region and obtain the first semantic data packet, which provides extremely rich semantic details for the user's focus area. Next, semantic change information of the non-overlapping change region is identified according to the second semantic granularity, i.e., a lighter... A coarse-grained recognition model of scale analyzes non-overlapping change regions to obtain a second semantic data packet, which is used to provide coarse-grained semantics for non-focal regions that meet basic context awareness requirements. Finally, the first differential transmission data packet is replaced and updated using the first encoded data packet, the first semantic data packet, and the second semantic data packet to determine the second differential transmission data packet. The first encoded data packet provides visual pixel information of the focal region, the first semantic data packet provides detailed semantic information of the focal region, and the second semantic data packet provides summary semantic information of the non-focal region, thereby ensuring that the most comprehensive and valuable scene information is provided to the smartphone backend with minimal data transmission and computational overhead.

[0032] Further, step S320 also includes step S321, defining quality parameter items, the quality parameters including resolution, target bit rate, frame rate, and quantization parameters; step S322, defining a first quality parameter solution space and a second quality parameter solution space based on the quality parameter items; step S323, selecting a first initial quality parameter and a second initial quality parameter from the first and second quality parameter solution spaces, and sending the first initial quality parameter and the second initial quality parameter to the coding quality scoring model; step S324, performing iterative analysis of coding quality scoring with the coding quality of the first quality parameter being greater than that of the second quality parameter as a difference constraint condition, to obtain the first quality parameter constraint and the second quality parameter constraint; step S325, performing Bayesian optimization search based on the first quality parameter constraint and the second quality parameter constraint to obtain the first quality parameter and the second quality parameter.

[0033] Preferably, quality parameters are defined, including resolution, target bitrate, frame rate, and quantization parameters. Resolution is the pixel size of the output image, bitrate is the number of bits of data processed per second, frame rate is the number of frames encoded per second, and the quantization parameter directly controls the compression distortion; a higher value results in greater compression and lower quality. Then, based on the defined quality parameters, multiple parameter combinations within a high-quality range are randomly set for overlapping variation regions. For example, resolutions range from 720p to 1080p, and quantization parameter values ​​range from 18 to 28, forming a first quality parameter solution space. Then, multiple parameter combinations within a low-quality range are randomly set for non-overlapping variation regions. For example, resolutions range from 240p to 480p, and quantization parameter values ​​range from 30 to 40, forming a second quality parameter solution space. A set of parameters is randomly selected from the first and second quality parameter solution spaces to determine the first and second initial quality parameters. The first and second initial quality parameters are then sent to the coding quality scoring model. The coding quality scoring model is pre-trained based on a regression model and can predict the comprehensive quality score obtained after encoding the scene content using the input set of coding quality parameters, thereby obtaining the first coding quality score and the second coding quality score.

[0034] Preferably, the coding quality scoring iterative analysis is performed with the coding quality of the first quality parameter being greater than that of the second quality parameter as a difference constraint. If the coding quality of the first quality parameter is not greater than that of the second quality parameter, meaning the quality score of the focal region is lower than that of the non-focal region, it indicates that the initial quality parameter setting is unreasonable, and this set of quality parameters is rejected. Then, relative quality relationship constraints based on model prediction are obtained, namely the first quality parameter constraint and the second quality parameter constraint. Bayesian optimization search is performed based on the first and second quality parameter constraints, i.e., the search is performed simultaneously in the solution space of the first and second quality parameters, with each iteration... A pair of candidate quality parameters, namely the first candidate quality parameter and the second candidate quality parameter, is extracted and input into the coding quality scoring model. It is checked whether the coding quality of the first candidate quality parameter is greater than that of the second candidate quality parameter. If so, the expected gain is calculated. Through multiple iterations, regions that do not meet the constraints or have little foreground are skipped. Finally, the coding configuration with the best overall transmission efficiency under the premise of satisfying visual attention priority is automatically and efficiently determined, and the optimal pair of quality parameters is output to obtain the first quality parameter and the second quality parameter. Thus, the best balance point is found between visual quality, data volume and computational complexity, ensuring high-speed and low-power collaborative transmission.

[0035] Furthermore, step S323 also includes obtaining a training dataset, which includes encoded data packet samples obtained by encoding scene data packet samples using different quality parameters based on the first quality parameter solution space and the second quality parameter solution space; calculating the peak signal-to-noise ratio, encoding efficiency index, and decoding accuracy for each encoded data packet sample; calculating the encoding quality score label of the encoded data packet sample by weighted information entropy; and training a regression model using the encoding quality score labels of the corresponding encoded data packets under different quality parameters to obtain a converged encoding quality score model.

[0036] Preferably, representative real image patches and video clips are acquired using AI glasses as scene data packet samples. Then, multiple sets of different quality parameters are randomly selected from the first and second quality parameter solution spaces defined for the focus and non-focus regions. Each set of selected quality parameters is used to encode each scene data packet sample. Each encoding generates an encoded data packet sample and its corresponding set of quality parameters, thus obtaining a training dataset. The encoded data packet sample corresponds to the set of quality parameters. Peak signal-to-noise ratio, encoding efficiency index, and decoding accuracy are calculated for each encoded data packet sample.

[0037] Preferably, peak signal-to-noise ratio (PSNR) is an objective indicator that measures the pixel-level error between the encoded image and the original image. A higher PSNR indicates higher fidelity. Coding efficiency refers to the compression ratio or the size of the encoded file, which is directly related to the goal of low-cost transmission. Under the same quality, the smaller the file, the higher the efficiency. Decoding accuracy is a semantic-level indicator. An object detector or classifier is used to analyze the original uncoded image and the decoded reconstructed image, respectively. The detected object categories and bounding box positions in the two analysis results are compared, and the proportion of objects in the original image that can be correctly identified and located in the reconstructed image is calculated.

[0038] Preferably, the peak signal-to-noise ratio (PSNR), coding efficiency index, and decoding accuracy are calculated using weighted information entropy. Information entropy is used for weighting; the higher the information entropy, the more dispersed the index is across all samples, and the more discriminative information it contains, thus assigning it a higher weight in the overall score. Specifically, the PSNR, coding efficiency index, and decoding accuracy are normalized across all training data. The information entropy of each index is calculated separately, and a corresponding weight is assigned based on the information entropy value. The three normalized indices for each training data point are then weighted and summed to obtain the coding quality score label for the encoded data packet. An initial scoring model is constructed using gradient boosting decision trees or support vector regression. The regression model is trained using the coding quality score labels of the corresponding encoded data packet samples under different quality parameters. Gradient descent is used to optimize and adjust the internal parameters of the initial scoring model, making its predicted scores as close as possible to the true coding quality score labels. When the model performance no longer significantly improves, a converged coding quality scoring model is determined and used to predict coding quality scores under different quality parameters.

[0039] Step S325 further includes obtaining a predicted coding quality score for the first initial quality parameter and the second initial quality parameter based on the coding quality scoring model; if the predicted coding quality score does not meet a preset coding quality score threshold, calculating the expected gain based on the first quality parameter constraint and the second quality parameter constraint, and using the expected gain factor to identify the selection probability of the first quality parameter solution space and the second quality parameter solution space; obtaining the first candidate quality parameter and the second candidate quality parameter again according to the selection probability identification, and so on, until the number of iterations reaches a preset threshold and the search stops, thus obtaining the first quality parameter and the second quality parameter.

[0040] Preferably, the first and second initial quality parameters are input into the coding quality scoring model for predictive analysis, and the corresponding predicted coding quality scores are output, including the first and second predicted coding quality scores. A preset coding quality score threshold is configured according to data transmission requirements. For example, if the difference between the initial coding quality score of the focal region and the initial coding quality score of the non-focal region is greater than 20, and the predicted coding quality score does not meet the preset coding quality score threshold, it indicates that the performance of the initial quality parameter combination is insufficient. Then, the expected gain is calculated based on the constraints of the first and second quality parameters. That is, for countless potential candidate points in the solution spaces of the first and second quality parameters, their expected gain factors are calculated respectively. This represents the possible improvement in the coding quality score relative to the currently known best score, based on the weighted average of the probability distribution. Candidate points with high expected gain factors are characterized as high-score points or points with uncertainty. At a high point, the selection probability is then marked on the solution spaces of the first and second quality parameters based on the expected gain factor. Regions with higher expected gain factors have a greater selection probability, ensuring the search is directed towards regions with potential coding quality parameters. Then, based on the constraints of the first and second quality parameters, the first and second candidate quality parameters are re-obtained according to the selection probability markings. These are then input into the coding quality scoring model for evaluation. The evaluation results update the Bayesian optimization probability model, and the expected gain factor and selection probability are recalculated to obtain the next set of candidate quality parameters. This process continues until the number of iterations reaches a preset threshold, at which point the search stops. The first and second quality parameters that satisfy the constraints and have the highest quality scores are obtained, determining the coding configuration scheme that achieves the best balance between visual quality and compression efficiency, thereby dynamically realizing optimal collaborative transmission between AI glasses and mobile phones.

[0041] Step S400: The second differential transmission data packet is transmitted to the smartphone terminal for storage update.

[0042] Preferably, the second differential transmission data packet is transmitted from the AI ​​glasses terminal to the smartphone terminal via a wireless communication channel. After receiving the second differential transmission data packet, the smartphone terminal caches it in its storage space to update the scene model maintained on the smartphone. Specifically, the smartphone terminal uses a corresponding decoder to decode the encoded data packet in the second differential transmission data packet, restore the image pixel blocks of the focal area and non-focal area, parse the received semantic data packet, and integrate the decoded and reconstructed pixel blocks and the parsed semantic information into the scene model of the smartphone terminal. Through storage updates, the scene representation of the smartphone terminal is synchronized with the real-world view of the AI ​​smart glasses, thereby ensuring the continuous, stable, and efficient operation of the entire system, while greatly reducing the transmission burden and improving transmission efficiency.

[0043] Furthermore, the high-speed, low-power collaborative transmission method between AI glasses and smartphones also includes setting up transmission channels between the AI ​​glasses terminal and the smartphone terminal. The transmission channels include a first transmission channel and a second transmission channel. The first transmission channel is configured using encoding compatibility parameters, and the second transmission channel is configured using semantic compatibility parameters. After the first differential transmission data packet is hierarchically co-coded according to the user's visual focus area, the transmission channels are used to transmit the second differential transmission data packet in parallel. Specifically, the first transmission channel is used for transmitting encoded data packets, and the second transmission channel is used for transmitting semantic data packets.

[0044] Preferably, a wireless communication link is set up between the AI ​​glasses terminal and the smartphone terminal as a transmission channel, such as different socket ports or data streams under the same Wi-Fi connection, or simultaneous use of Wi-Fi and Bluetooth. The transmission channel includes a first transmission channel and a second transmission channel. The first transmission channel is used for transmitting encoded data packets, and the second transmission channel is used for transmitting semantic data packets. The first transmission channel is configured with encoding compatibility parameters, that is, the parameters of the transmission channel are optimized for streaming media transmission, such as high bandwidth, allowing a certain packet loss rate, and high transmission speed. The second transmission channel is configured with semantic compatibility parameters, that is, the parameters of the transmission channel are optimized for critical instruction transmission, such as high reliability, low latency, and low bandwidth. After the first differential transmission data packet is hierarchically co-coded according to the user's visual focus area, the transmission channel is used to transmit the second differential transmission data packet in parallel. That is, the encoded data packet is sent through the first transmission channel, where the encoded data packet contains high-quality encoded packets of the focus area and low-quality encoded packets of the non-focus area. At the same time, the semantic data packet is sent through the second transmission channel, where the semantic data packet contains fine-grained semantics of the focus area and coarse-grained semantics of the non-focus area, thereby reducing the overall transmission latency and improving the reliability of data transmission.

[0045] In the above text, refer to Figure 1 A high-speed, low-power collaborative transmission method between AI glasses and a smartphone according to embodiments of the present invention is described in detail. Next, reference will be made to... Figure 2 A high-speed, low-power collaborative transmission system between AI glasses and a smartphone according to an embodiment of the present invention is described.

[0046] The high-speed, low-power collaborative transmission system between AI glasses and smartphones according to embodiments of the present invention addresses the technical problems in existing technologies, such as the inability to intelligently distinguish and prioritize the transmission of core information of interest to the user, low data transmission efficiency, and high power consumption. It achieves the technical effect of reducing the amount of transmitted data, device power consumption, and transmission latency, while ensuring the transmission quality of critical information and improving user experience. Figure 2As shown, the high-speed, low-power collaborative transmission system between AI glasses and smartphones includes: a scene construction module 10, a differential recognition module 20, a hierarchical collaborative coding module 30, and a storage update module 40.

[0047] The scene construction module 10 is used to construct a first scene background model based on a first scene-based data packet through the AI ​​glasses terminal; the differential recognition module 20 is used to perform differential recognition on the real-time scene-based semantic data packet collected by the AI ​​glasses terminal based on the first scene background model to obtain the first differential transmission data packet corresponding to the foreground change region; the hierarchical co-coding module 30 is used to extract the user's visual focus region of the AI ​​glasses terminal and perform hierarchical co-coding on the first differential transmission data packet according to the user's visual focus region to obtain a second differential transmission data packet; the storage update module 40 is used to transmit the second differential transmission data packet to the smartphone terminal for storage update.

[0048] The specific configuration of the hierarchical cooperative coding module 30 will be described in detail below. The hierarchical cooperative coding module 30 further includes: acquiring overlapping and non-overlapping change regions between the user's visual focus region and the foreground change region; configuring a first quality parameter and a second quality parameter, wherein the encoding quality of the first quality parameter is greater than the encoding quality of the second quality parameter; encoding the overlapping change region based on the first quality parameter to obtain a first encoded data packet; encoding the non-overlapping change region based on the second quality parameter to obtain a second encoded data packet; and updating the first differential transmission data packet with the first encoded data packet and the second encoded data packet to obtain a second differential transmission data packet.

[0049] The specific configuration of the hierarchical cooperative coding module 30 will be described in detail below. The hierarchical cooperative coding module 30 further includes: acquiring overlapping and non-overlapping change regions between the user's visual focus region and the foreground change region; encoding the overlapping change regions based on a first quality parameter to obtain a first encoded data packet; identifying semantic change information of the non-overlapping change regions to obtain a semantic data packet, wherein the semantic data packet includes an object category identifier, spatial location coordinates, and an action state description; and updating the first differential transmission data packet with the first encoded data packet and the semantic data packet to obtain a second differential transmission data packet.

[0050] The specific configuration of the hierarchical cooperative coding module 30 will be described in detail below. The hierarchical cooperative coding module 30 further includes: defining quality parameter items, including resolution, target bit rate, frame rate, and quantization parameters; defining a first quality parameter solution space and a second quality parameter solution space based on the quality parameter items; selecting a first initial quality parameter and a second initial quality parameter from the first and second quality parameter solution spaces, and sending the first initial quality parameter and the second initial quality parameter to the coding quality scoring model; performing iterative analysis of coding quality scoring with the coding quality of the first quality parameter being greater than the coding quality of the second quality parameter as a difference constraint condition to obtain the first quality parameter constraint and the second quality parameter constraint; and performing Bayesian optimization search based on the first quality parameter constraint and the second quality parameter constraint to obtain the first quality parameter and the second quality parameter.

[0051] The specific configuration of the hierarchical cooperative coding module 30 will be described in detail below. The hierarchical cooperative coding module 30 further includes: acquiring a training dataset, which includes coded data packet samples obtained by encoding scene data packet samples using different quality parameters under the first and second quality parameter solution spaces; calculating the peak signal-to-noise ratio, coding efficiency index, and decoding accuracy for each coded data packet sample; calculating the coding quality score label of the coded data packet sample using weighted information entropy; and training a regression model using the coding quality score labels of the corresponding coded data packets under different quality parameters to obtain a converged coding quality score model.

[0052] The specific configuration of the hierarchical cooperative coding module 30 will be described in detail below. The hierarchical cooperative coding module 30 further includes: obtaining predicted coding quality scores for the first initial quality parameter and the second initial quality parameter based on a coding quality scoring model; if the predicted coding quality score does not meet a preset coding quality score threshold, calculating the expected gain based on the first quality parameter constraint and the second quality parameter constraint, and using the expected gain factor to assign a selection probability label to the solution space of the first quality parameter and the solution space of the second quality parameter; obtaining the first candidate quality parameter and the second candidate quality parameter again according to the selection probability label, and so on, until the number of iterations reaches a preset threshold and the search stops, thus obtaining the first quality parameter and the second quality parameter.

[0053] The following describes in detail the specific configuration of the high-speed, low-power collaborative transmission system between AI glasses and smartphones, including: setting up transmission channels between the AI ​​glasses terminal and the smartphone terminal, the transmission channels including a first transmission channel and a second transmission channel, the first transmission channel being configured with encoding compatibility parameters and the second transmission channel being configured with semantic compatibility parameters; after the first differential transmission data packet is hierarchically collaboratively encoded according to the user's visual focus area, the transmission channels are used to transmit the second differential transmission data packet in parallel, wherein the first transmission channel is used for transmitting encoded data packets and the second transmission channel is used for transmitting semantic data packets.

[0054] The specific configuration of the hierarchical cooperative coding module 30 will be described in detail below. The hierarchical cooperative coding module 30 further includes: identifying semantic change information of the overlapping change region according to a first semantic granularity to obtain a first semantic data packet; identifying semantic change information of the non-overlapping change region according to a second semantic granularity to obtain a second semantic data packet, wherein the first semantic granularity is smaller than the second semantic granularity; updating the first differential transmission data packet based on the first coded data packet, the first semantic data packet, and the second semantic data packet to obtain a second differential transmission data packet.

[0055] The high-speed, low-power collaborative transmission system between AI glasses and smartphones provided in this embodiment of the invention can execute the high-speed, low-power collaborative transmission method between AI glasses and smartphones provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A high-speed, low-power collaborative transmission method between AI glasses and a smartphone, characterized in that: The method includes: A first scene background model based on a first scene-specific data package is constructed using AI glasses terminals; Based on the first scene background model, differential recognition is performed on the real-time scene-based semantic data packets collected by the AI ​​glasses terminal to obtain the first differential transmission data packets corresponding to the foreground change area; Extract the user's visual focus area of ​​the AI ​​glasses terminal, and perform hierarchical collaborative coding on the first differential transmission data packet according to the user's visual focus area to obtain the second differential transmission data packet; The second differential transmission data packet is transmitted to the smartphone terminal for storage update.

2. The high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in claim 1, characterized in that, The method involves performing hierarchical co-coding on the first differential transmission data packet according to the user's visual focus area to obtain the second differential transmission data packet, including: Collect the overlapping and non-overlapping change areas between the user's visual focus area and the foreground change area; Configure a first quality parameter and a second quality parameter, wherein the encoding quality of the first quality parameter is greater than the encoding quality of the second quality parameter; The overlapping variation region is encoded based on a first quality parameter to obtain a first encoded data packet; The non-overlapping variation region is encoded based on the second quality parameter to obtain a second encoded data packet; The first differential transmission data packet is updated by the first encoded data packet and the second encoded data packet to obtain the second differential transmission data packet.

3. The high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in claim 1, characterized in that, The method further includes performing hierarchical co-coding on the first differential transmission data packet according to the user's visual focus area to obtain the second differential transmission data packet. Collect the overlapping and non-overlapping change areas between the user's visual focus area and the foreground change area; The overlapping variation region is encoded based on a first quality parameter to obtain a first encoded data packet; Identify the semantic change information of the non-overlapping change region and obtain a semantic data packet, wherein the semantic data packet includes an object category identifier, spatial location coordinates, and an action state description; The first differential transmission data packet is updated by the first encoded data packet and the semantic data packet to obtain the second differential transmission data packet.

4. The high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in claim 2, characterized in that, The methods for configuring the first and second mass parameters include: Define quality parameters, including resolution, bitrate target, frame rate, and quantization parameters; Based on the aforementioned mass parameter terms, define a first mass parameter solution space and a second mass parameter solution space; Select a first initial quality parameter and a second initial quality parameter from the first quality parameter solution space and the second quality parameter solution space, and send the first initial quality parameter and the second initial quality parameter to the coding quality scoring model; Using the coding quality of the first quality parameter being greater than that of the second quality parameter as a difference constraint, an iterative analysis of coding quality scoring is performed to obtain the first quality parameter constraint and the second quality parameter constraint. A Bayesian optimization search is performed based on the first quality parameter constraint and the second quality parameter constraint to obtain the first quality parameter and the second quality parameter.

5. The high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in claim 4, characterized in that, Methods for constructing coding quality scoring models include: Obtain a training dataset, which includes encoded data packet samples obtained by encoding scene data packet samples using different quality parameters based on the first quality parameter solution space and the second quality parameter solution space; Peak signal-to-noise ratio, coding efficiency index, and decoding accuracy are calculated for each encoded data packet sample. The coding quality score label of the encoded data packet sample is obtained by calculating the weighted information entropy. By training a regression model using the coding quality score labels of the corresponding encoded data packets under different quality parameters, a converged coding quality score model is obtained.

6. The high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in claim 4, characterized in that, A Bayesian optimization search is performed based on the first quality parameter constraint and the second quality parameter constraint to obtain the first quality parameter and the second quality parameter. The method includes: The predicted coding quality scores for the first initial quality parameter and the second initial quality parameter are obtained based on the coding quality scoring model. If the predicted coding quality score does not meet the preset coding quality score threshold, the expected gain is calculated based on the first quality parameter constraint and the second quality parameter constraint, and the expected gain factor is used to identify the selection probability of the first quality parameter solution space and the second quality parameter solution space. The first and second candidate quality parameters are obtained again based on the selected probability identifier, and so on, until the number of iterations reaches a preset threshold and the search stops, thus obtaining the first and second quality parameters.

7. The high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in claim 1, characterized in that, The method includes: A transmission channel is set up between the AI ​​glasses terminal and the smartphone terminal. The transmission channel includes a first transmission channel and a second transmission channel. The first transmission channel is configured through encoding compatibility parameters, and the second transmission channel is configured through semantic compatibility parameters. After the first differential transmission data packet is hierarchically co-coded according to the user's visual focus area, the transmission channel is used to transmit the second differential transmission data packet in parallel, wherein the first transmission channel is used to transmit coded data packets and the second transmission channel is used to transmit semantic data packets.

8. The high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in claim 3, characterized in that, After encoding the overlapping variation region based on the first quality parameter and obtaining the first encoded data packet, the method further includes: Identify the semantic change information of the overlapping change region according to the first semantic granularity, and obtain the first semantic data packet; Identify the semantic change information of the non-overlapping change region according to the second semantic granularity, and obtain the second semantic data packet, wherein the first semantic granularity is smaller than the second semantic granularity; The first differential transmission data packet is updated based on the first encoded data packet, the first semantic data packet, and the second semantic data packet to obtain the second differential transmission data packet.

9. A high-speed, low-power collaborative transmission system between AI glasses and a smartphone, characterized in that: The system is used to implement the high-speed, low-power collaborative transmission method between AI glasses and a smartphone as described in any one of claims 1 to 8, the system comprising: The scene construction module is used to construct a first scene background model based on the first scene-based data package through the AI ​​glasses terminal; The differential recognition module is used to perform differential recognition on the real-time contextualized semantic data packets collected by the AI ​​glasses terminal based on the first scene background model, and obtain the first differential transmission data packets corresponding to the foreground change area. The hierarchical co-coding module is used to extract the user's visual focus area of ​​the AI ​​glasses terminal, and perform hierarchical co-coding on the first differential transmission data packet according to the user's visual focus area to obtain the second differential transmission data packet. The storage update module is used to transmit the second differential transmission data packet to the smartphone terminal for storage update.

Images