Data processing system, glasses and mobile terminal

Abstract

The embodiment of the invention discloses a data processing system, glasses and a mobile terminal. Wherein the glasses comprise a camera sensor, a camera processor and a first communication unit, and the camera processor is used for carrying out preprocessing such as automatic focusing, automatic exposure and / or automatic white balance on original video data and / or original image data shot by the camera sensor to obtain preprocessed data; the mobile terminal comprises a terminal processor and a second communication unit, and the terminal processor is configured to carry out image processing on the preprocessed data received by the second communication unit, obtain a target video and / or a target image and send the target video and / or the target image to the mobile terminal; the preprocessed data comprises preprocessed RAW domain video data and / or RAW domain image data. Therefore, according to the glasses provided by the embodiment of the invention, the RAW domain original data can be directly sent to the mobile terminal, and corresponding image processing operation is carried out through the mobile terminal, so that the power consumption and the hardware cost of the glasses are reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a data processing system, glasses, and a mobile terminal. Background Art

[0002] Smart glasses are usually used in cooperation with a mobile terminal, and can perform data communication with the mobile terminal through communication methods such as Bluetooth and Wi-Fi. The smart glasses are relatively small in volume and weight, and are sensitive to power consumption and heat. Therefore, how to effectively reduce the power consumption of the smart glasses and ensure the user experience is crucial. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data processing system, glasses, and a mobile terminal. Among them, the glasses can directly send RAW domain original data to the mobile terminal, and perform corresponding image processing operations through the mobile terminal, which reduces the power consumption and hardware cost of the glasses.

[0004] In a first aspect, an embodiment of the present invention provides a data processing system, the system includes:

[0005] Glasses, including a camera sensor, a camera processor, and a first communication unit. The camera sensor is configured to capture original data, the camera processor is configured to preprocess the original data to obtain preprocessed data, and the first communication unit is configured to send the preprocessed data. The original data includes original video data and / or original image data, the preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data, and the preprocessing includes at least one of autofocus, autoexposure, and automatic white balance;

[0006] A mobile terminal, including a terminal processor and a second communication unit. The second communication unit is configured to receive the preprocessed data, and the terminal processor is configured to perform image processing on the preprocessed data to obtain a target video and / or a target image.

[0007] Further, the image processing is a combination of one or more of the following: lens correction processing, color conversion processing, noise reduction processing, sharpening processing, contrast adjustment, and brightness adjustment.

[0008] Further, the glasses further include a host controller and a storage device corresponding to the host controller. The camera processor is connected to the host controller through an SPI interface or a USB interface, and the camera processor stores the preprocessed data in the storage device through the SPI interface or the USB interface.

[0009] Further, the glasses are configured to transmit the preprocessed data stored in the storage device to the mobile terminal through the first communication unit in response to receiving a data synchronization instruction from the mobile terminal.

[0010] Further, the mobile terminal is configured to send a data deletion instruction to the glasses through the second communication unit in response to completion of data synchronization, and the glasses are configured to delete the preprocessed data in the storage device through the host controller in response to receiving the data deletion instruction.

[0011] Further, the glasses further include an audio sensor and a motion sensor. The audio sensor is configured to acquire audio data, the motion sensor is configured to acquire attitude data of the glasses, and the first communication unit is further configured to transmit the audio data and the attitude data to the mobile terminal respectively.

[0012] Further, the terminal processor is provided with an audio-video encoder. The terminal processor encodes information of the video frames, the audio data, and the attitude data after image processing through the audio-video encoder to generate a target video.

[0013] Further, the system further includes:

[0014] A cloud server, configured to receive the target video and / or target image synchronized from the mobile terminal.

[0015] In a second aspect, an embodiment of the present invention provides a pair of glasses, which includes:

[0016] A camera sensor, configured to capture raw data, where the raw data includes raw video data and / or raw image data;

[0017] A camera processor, configured to preprocess the raw data to obtain preprocessed data, where the preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data, and the preprocessing includes at least one of autofocus, automatic exposure, and automatic white balance;

[0018] A first communication unit, configured to send the preprocessed data to a mobile terminal so that the mobile terminal performs image processing on the preprocessed data to obtain a target video and / or a target image.

[0019] In a third aspect, an embodiment of the present invention provides a mobile terminal, which includes:

[0020] A second communication unit, configured to receive preprocessed data from the glasses, where the preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data, and the preprocessing includes at least one of autofocus, autoexposure, and automatic white balance;

[0021] A terminal processor, configured to perform image processing on the preprocessed data to obtain a target video and / or a target image.

[0022] In an embodiment of the present invention, the glasses include a camera sensor, a camera processor, and a first communication unit. The camera processor is used to perform preprocessing such as autofocus, autoexposure, and / or automatic white balance on the original video data and / or original image data captured by the camera sensor to obtain preprocessed data, and send the preprocessed data to the mobile terminal through the first communication unit. The mobile terminal includes a terminal processor and a second communication unit. The terminal processor is configured to perform image processing on the preprocessed data received through the second communication unit to obtain a target video and / or a target image. The preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data. Thus, the glasses in this embodiment can directly send the RAW domain original data to the mobile terminal and perform corresponding image processing operations through the mobile terminal, which reduces the power consumption and hardware cost of the glasses. Description of the Drawings

[0023] Through the following description of the embodiments of the present invention with reference to the drawings, the above and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0024] Figure 1 is a schematic diagram of a data processing system according to an embodiment of the present invention;

[0025] Figure 2 is a flowchart of a data processing method according to an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of another data processing system according to an embodiment of the present invention;

[0027] Figure 4 is a flowchart of another data processing method according to an embodiment of the present invention;

[0028] Figure 5 is a schematic diagram of yet another data processing system according to an embodiment of the present invention;

[0029] Figure 6 is a flowchart of yet another data processing method according to an embodiment of the present invention;

[0030] Figure 7 is a schematic diagram of yet another data processing system according to an embodiment of the present invention. Detailed Embodiments

[0031] The present application will be described based on embodiments, but the present application is not limited to these embodiments. In the following detailed description of the present application, some specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. In order to avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0032] In addition, those of ordinary skill in the art should understand that the drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0033] Unless the context clearly requires otherwise, words such as "including" and "comprising" in the entire application document should be construed in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to".

[0034] In the description of the present application, it should be understood that terms such as "first" and "second" are only configured for descriptive purposes and cannot be construed as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0035] For the solutions described in this specification and the embodiments, if they involve personal information processing, they will be processed on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for performing a contract, etc.), and will only be processed within the specified or agreed scope. If the user refuses to process personal information other than the necessary information required for the basic functions, it will not affect the user's use of the basic functions.

[0036] Taking pictures and recording videos are usage scenarios in the comparison scenarios of smart glasses. In the relevant comparative examples, usually, an ISP (Image Signal Process) chip is configured in the smart glasses to perform operations such as lens correction, color conversion, noise reduction, and sharpening on the RAW domain raw data output by the camera sensor to improve the quality of photos or videos. However, performing various image processing operations on the RAW domain raw data inside the smart glasses greatly increases the load of the smart glasses, resulting in a high power consumption of the smart glasses, easy heating, and thus may require frequent charging, affecting the user experience. Therefore, the embodiments of the present invention provide a data processing system, glasses, and a mobile terminal. Among them, the glasses can directly send the RAW domain raw data to the mobile terminal and perform corresponding image processing operations through the mobile terminal, which reduces the power consumption and hardware cost of the glasses.

[0037] Figure 1 is a schematic diagram of the data processing system of the present invention. As Figure 1As shown in the figure, the data processing system of this embodiment includes glasses 10 and a mobile terminal 20. Among them, the mobile terminal 20 can be a mobile phone, a tablet, a smart watch or other mobile terminals, and this embodiment does not limit the device type of the mobile terminal 20.

[0038] Among them, the glasses 10 include a camera sensor 11, a camera processor 12, and a first communication unit 13. Among them, the camera sensor 11 is used to capture raw data. The raw data includes raw video data and / or raw image data. Optionally, the camera sensor 11 can be controlled by a corresponding shooting instruction to record video to obtain raw video data, or controlled by a corresponding shooting instruction to capture raw image data.

[0039] In an alternative implementation, the shooting instruction can be triggered by a gesture operation on the glasses 10 (such as clicking or continuously clicking multiple times within a predetermined range of the glasses 10, sliding a predetermined gesture within a predetermined range of the glasses 10, etc.), can also be triggered by voice, or can also be triggered by the mobile terminal 20 communicatively connected thereto. It should be understood that this embodiment does not limit how to trigger the camera sensor 11 in the glasses 10 to execute the shooting action, and it can be set based on actual needs.

[0040] The camera processor 12 is configured to preprocess the raw data captured by the camera sensor 11 to obtain preprocessed data. Among them, the preprocessing includes at least one of autofocus (AF), automatic exposure (AE), and automatic white balance (AWB). The preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data. Since the camera sensor 11 requires 3A (that is, AF, AE, AWB) when taking pictures and recording videos, especially after AE is stabilized, subsequent image data transmission and processing can be performed. Therefore, the camera sensor 11 of this embodiment needs to be configured with a camera processor 12 that can perform AF, AE, and / or AWB processing.

[0041] In this embodiment, the raw data captured by the camera sensor 11 is RAW domain data, and the preprocessed data after the camera processor 12 performs 3A processing is still RAW domain data. Among them, RAW domain data refers to the raw data directly obtained from the camera sensor without image processing or compression. Different from formats such as JPEG that have undergone certain image processing and compression, the RAW file saves all the information captured by the camera sensor and provides the greatest flexibility for subsequent image processing.

[0042] It can be seen that the camera processor 12 of this embodiment only performs certain 3A processing on the raw data, and the power consumption and heat generated are very low. Further, the camera processor 12 can be embedded in the camera sensor 11 or can be set outside the camera sensor 11, and this embodiment does not limit its setting position.

[0043] Furthermore, the camera processor 12 of this embodiment can adopt the ISP chip configured by existing camera sensors, and turn off the processing functions such as image processing and compression in the ISP chip, so that it only performs 3A processing to reduce the power consumption of the ISP chip. This can implement the solution of this embodiment without changing the configuration of the existing glasses, reducing the cost of implementing the solution. In another alternative implementation, this embodiment can configure a camera processor with only 3A processing function in the glasses, which reduces the hardware cost of the camera processor on the basis of reducing the power consumption of the glasses, and further reduces the manufacturing cost of the glasses.

[0044] The first communication unit 13 is configured to send the preprocessed data in the RAW domain to the mobile terminal 20. In an alternative implementation, the first communication unit 13 can be a Bluetooth device, a wifi device, or a device of other communication types, which can be based on the specific configuration of the glasses 10. Further optionally, the first communication unit 13 can include a Bluetooth device and a wifi device. When the glasses 10 are connected to the mobile terminal 20 via Bluetooth, data can be transmitted through the Bluetooth device. When the glasses 10 are not connected to the mobile terminal 20 via Bluetooth and are connected to the corresponding routing device via a wifi device, or when the amount of data to be transmitted is large and the glasses 10 are connected to the corresponding routing device via a wifi device, the glasses 10 can transmit data through the wifi communication network.

[0045] Furthermore, the mobile terminal 20 includes a terminal processor 21 and a second communication unit 22. The second communication unit is configured to receive the preprocessed data in the RAW domain from the glasses 10. The mobile terminal 20 is communicatively connected to the glasses 10 through the second communication unit 22 and the first communication unit 13. Optionally, the second communication unit 22 can be a Bluetooth device, a wifi device, or a device of other communication types, which can be based on the specific configuration of the mobile terminal 20. Further optionally, the second communication unit 22 can include a Bluetooth device and a wifi device. When the glasses 10 are connected to the mobile terminal 20 via Bluetooth, data can be transmitted through the Bluetooth device. When the glasses 10 are not connected to the mobile terminal 20 via Bluetooth and are connected to the corresponding routing device via a wifi device, or when the amount of data to be transmitted is large and the mobile terminal 20 is connected to the corresponding routing device via a wifi device, the mobile terminal 20 can transmit data with the glasses 10 through the wifi communication network. It should be understood that this embodiment does not limit the data transmission method between the glasses 10 and the mobile terminal 20.

[0046] Further, the terminal processor 21 is configured to perform image processing on the RAW domain preprocessed data received by the second communication unit 22 to obtain a target video and / or a target image. The image processing includes one or a combination of the following operations: lens correction processing, color conversion processing, noise reduction processing, sharpening processing, contrast adjustment, brightness adjustment, encoding, and other processing operations. The image processing of the terminal processor 21 in this embodiment is used to optimize and enhance the domain processor data in the RAW domain to obtain a target video or image that meets the user's requirements in terms of clarity, aesthetics, etc.

[0047] Further, the terminal processor 21 is configured to perform operations such as BPC (Bad Pixel Correction), LTM (Local Tone Mapping), DM (Demosaic), Denoise (noise reduction), GAMMA (gamma correction), COLOR (color correction), HFG (High Frequency Gain), TCC (Texture Contrast Control), encoding and decoding on the RAW domain preprocessed data to obtain a target video or image.

[0048] Figure 2 is a flowchart of a data processing method according to an embodiment of the present invention. The flowchart of the data processing method in this embodiment is applicable to a data processing system as shown in Figure 1 shown, and as shown in Figure 2 shown, the data processing method according to the embodiment of the present invention includes the following steps:

[0049] Step S21, capturing raw data. The glasses 10 in this embodiment capture raw data through the camera sensor 11. The raw data includes raw video data and / or raw image data. Optionally, the camera sensor 11 can be controlled by a corresponding shooting instruction to record video to obtain raw video data, or controlled by a corresponding shooting instruction to capture to obtain raw image data.

[0050] Step S22, performing 3A preprocessing on the raw data to obtain RAW domain preprocessed data. The glasses 10 in this embodiment perform preprocessing on the raw data captured by the camera sensor 11 through the camera processor 12 to obtain preprocessed data. Among them, the 3A preprocessing includes at least one of autofocus (AF), automatic exposure (AE), and automatic white balance (AWB).

[0051] Step S23, sending the RAW domain preprocessed data. The glasses 10 in this embodiment send the RAW domain preprocessed data to the mobile terminal 20 through the first communication unit 13.

[0052] Step S24: Perform image processing on the preprocessed RAW domain data to obtain a target video and / or a target image. In this embodiment, the mobile terminal 20 receives the preprocessed RAW domain data through the second communication unit 22, and the terminal processor 21 performs image processing on the received preprocessed RAW domain data to obtain a target video and / or a target image. Among them, the image processing includes one or a combination of the following processing operations: lens correction processing, color conversion processing, noise reduction processing, sharpening processing, contrast adjustment, brightness adjustment, encoding, etc. The image processing of the terminal processor 21 in this embodiment is used to optimize and enhance the domain processor data in the RAW domain to obtain a target video or image that meets the user's requirements in terms of clarity, aesthetics, etc.

[0053] In an embodiment of the present invention, the glasses include a camera sensor, a camera processor, and a first communication unit. The camera processor is used to perform preprocessing such as autofocus, autoexposure, and / or auto white balance on the original video data and / or original image data captured by the camera sensor to obtain preprocessed data, and send the preprocessed data to the mobile terminal through the first communication unit. The mobile terminal includes a terminal processor and a second communication unit. The terminal processor is configured to perform image processing on the preprocessed data received through the second communication unit to obtain a target video and / or a target image. The preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data. Thus, the glasses in this embodiment can directly send the RAW domain original data to the mobile terminal and perform corresponding image processing operations through the mobile terminal, which reduces the power consumption and hardware cost of the glasses.

[0054] Figure 3 is a schematic diagram of another data processing system according to an embodiment of the present invention. As Figure 3 shown, in the data processing system of this embodiment, in addition to the camera sensor 11, the camera processor 12, and the first communication unit 13, the glasses 10 further include a host controller 14 and a storage device 15. It should be understood that in this embodiment, the camera sensor 11, the camera processor 12, and the first communication unit 13 are similar to those in the Figure 1 described embodiment and will not be described in detail here.

[0055] In an optional implementation manner, the host controller 14 in the glasses 10 can be implemented by an MCU (Microcontroller Unit). It should be understood that the host controller 14 of the glasses 10 in this embodiment can also be implemented by other types of controllers or processors as long as they can implement the corresponding control functions and data processing, and will not be exemplified one by one here.

[0056] Further, the storage device 15 can be an eMMC (Embedded Multi-Media Card). Among them, eMMC is an embedded non-volatile storage that integrates NAND flash memory and a host controller. It can provide a standardized interface, simplify storage management, and improve the reliability and performance of the system. It should be understood that this embodiment does not limit the storage device 13, and it can also be a UFS (Universal Flash Storage) storage, an NVMe SSD (Non-Volatile Memory Express Solid State Drive), etc., which can be configured based on specific requirements and costs, and will not be exemplified one by one here.

[0057] Further optionally, in the glasses 10 of this embodiment, the host controller 14 and the storage device 15 are integrated together to provide a large-capacity and reliable storage solution for the host controller 14. In other alternative implementation manners, the host controller 14 and the storage device 15 may not be integrated, and this embodiment does not limit the installation positions and integration manners of the host controller 14 and the storage device 15.

[0058] Further, in this embodiment, the camera processor 12 and the host controller 14 can be connected through an SPI interface or a USB interface. It should be understood that this embodiment does not limit the connection manner between the camera processor 12 and the host controller 14, as long as it can achieve data transmission and control.

[0059] Further, in this embodiment, the camera sensor 11 captures raw data, and the camera processor 12 performs preprocessing of at least one of autofocus, autoexposure, and automatic white balance on the raw data to obtain RAW domain preprocessed data. Further, the camera processor 12 stores the obtained RAW domain preprocessed data in the storage device 15 through an SPI interface or a USB interface, waiting for the user to perform data synchronization.

[0060] Further, in response to receiving a data synchronization instruction, the mobile terminal 20 sends a data synchronization instruction to the glasses 10 through the second communication unit 22 and the first communication unit 13. After receiving the data synchronization instruction, the host controller 14 in the glasses 10 transmits the preprocessed data in the RAW domain in the storage device 15 to the mobile terminal 20 through the first communication unit 13 and the second communication unit 22. Further, in response to the completion of data synchronization, the mobile terminal 20 sends a data deletion instruction to the glasses 10 through the second communication unit 22 and the first communication unit 13. In response to receiving the data deletion instruction, the glasses 10 delete the preprocessed data in the storage device 15 through the host controller 14 to release storage resources.

[0061] Figure 4It is a flowchart of another data processing method according to an embodiment of the present invention. The flowchart of the data processing method in this embodiment is applicable to a data processing system as shown in Figure 3 shown, and as shown in Figure 4 the data processing method according to an embodiment of the present invention includes the following steps:

[0062] Step S41, capturing raw data. The glasses 10 in this embodiment capture raw data through the camera sensor 11. The raw data includes raw video data and / or raw image data. Optionally, the camera sensor 11 can be controlled by a corresponding shooting instruction to record video to obtain raw video data, or controlled by a corresponding shooting instruction to capture to obtain raw image data.

[0063] Step S42, performing 3A preprocessing on the raw data to obtain preprocessed data in the RAW domain. The glasses 10 in this embodiment preprocess the raw data captured by the camera sensor 11 through the camera processor 12 to obtain preprocessed data. Among them, the 3A preprocessing includes at least one of autofocus (AF), automatic exposure (AE), and automatic white balance (AWB).

[0064] Step S43, storing the preprocessed data in the RAW domain in a storage device. The camera processor 12 in this embodiment stores the obtained preprocessed data in the RAW domain in the storage device 15 through the SPI interface or the USB interface, waiting for the user to perform data synchronization.

[0065] Step S44, the mobile terminal 20 sends a data synchronization instruction to the glasses 10.

[0066] Step S45, sending the preprocessed data in the RAW domain. After receiving the data synchronization instruction, the host controller 14 in the glasses 10 transmits the preprocessed data in the RAW domain in the storage device 15 to the mobile terminal 20 through the first communication unit 13 and the second communication unit 22.

[0067] Step S46, in response to the completion of data synchronization, the mobile terminal 20 sends a data deletion instruction to the glasses 10.

[0068] Step S47, in response to receiving the data deletion instruction, the glasses 10 delete the preprocessed data in the storage device 15 through the host controller 14 to release storage resources.

[0069] Step S48: Perform image processing on the preprocessed data in the RAW domain to obtain a target video and / or a target image. In this embodiment, the mobile terminal 20 receives the preprocessed data in the RAW domain through the second communication unit 22, and the terminal processor 21 performs image processing on the received preprocessed data in the RAW domain to obtain a target video and / or a target image. Among them, the image processing includes one or a combination of the following processing operations: lens correction processing, color conversion processing, noise reduction processing, sharpening processing, contrast adjustment, brightness adjustment, encoding, etc. The image processing of the terminal processor 21 in this embodiment is used to optimize and enhance the data of the domain processor in the RAW domain to obtain a target video or image that meets the user's requirements in terms of clarity, aesthetics, etc.

[0070] It should be understood that in this embodiment, step S48 may start to be executed after receiving the preprocessed data in the RAW domain, or may start to be executed after the data synchronization is completed. This embodiment does not limit this.

[0071] It should be understood that the above data synchronization process is controlled by the mobile terminal 20. In other alternative implementation manners, when the glasses 10 and the mobile terminal 20 are in a communication connection state and the user enables the data function, after the glasses 10 obtain the preprocessed data in the RAW domain, they can directly synchronously transmit it to the mobile terminal 20 and delete the preprocessed data in the storage device 15 after the transmission is completed. That is, this embodiment does not limit the control end of the data synchronization process, and it can be configured based on the actual application situation.

[0072] In the embodiment of the present invention, the glasses include a camera sensor, a camera processor, and a first communication unit. The camera processor is used to perform preprocessing such as autofocus, autoexposure, and / or automatic white balance on the original video data and / or original image data captured by the camera sensor to obtain preprocessed data, and send the preprocessed data to the mobile terminal through the first communication unit. The mobile terminal includes a terminal processor and a second communication unit. The terminal processor is configured to perform image processing on the preprocessed data received through the second communication unit to obtain a target video and / or a target image. The preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data. Among them, in this embodiment, the glasses can store the preprocessed data and synchronize the preprocessed data to the mobile terminal for image processing when the glasses and the mobile terminal establish a communication connection to perform data synchronization. Thus, the glasses in this embodiment can directly send the RAW domain original data to the mobile terminal and perform corresponding image processing operations through the mobile terminal, which reduces the power consumption and hardware cost of the glasses.

[0073] Figure 5 is a schematic diagram of another data processing system according to an embodiment of the present invention. As Figure 5As shown, in the data processing system of this embodiment, in addition to the camera sensor 11, the camera processor 12, and the first communication unit 13, the glasses 10 further include an audio sensor 16 and / or a motion sensor 17. Optionally, the glasses 10 of this embodiment may further include a host controller 14 and a storage device 15. It should be understood that in this embodiment, the hardware settings and data processing methods of the camera sensor 11, the camera processor 12, the first communication unit 13, the host controller 14, and the storage device 15 are similar to those of Figure 3 the embodiments described above and will not be described in detail here.

[0074] Furthermore, the camera sensor 11, the first communication unit 13, the audio sensor 16, the motion sensor 17, and the host controller 14 of this embodiment can be connected via a bus.

[0075] Furthermore, the camera sensor 11 of this embodiment captures raw data (such as raw video data and / or image data) through its built-in camera service module (CameraService).

[0076] Furthermore, when the user records sound or video, the audio sensor 16 is required to provide corresponding support. The audio sensor 16 of this embodiment records corresponding audio data through its built-in audio service module (Audio Service).

[0077] Furthermore, if the user is outdoors or in other scenarios with frequent movement, there may be video jitter or distortion when recording video. In this embodiment, the motion sensor 17 can capture attitude data to adjust the position and angle of the video frame based on this, thereby improving the visual stability of the video and reducing the blur and distortion caused by jitter, and thus improving the quality of the subsequent generated video. The motion sensor 17 of this embodiment can be implemented by an IMU (Inertial Measurement Unit), or corresponding sensors such as an accelerometer, a gyroscope, and a magnetometer can be configured based on the actual situation to implement it. This embodiment does not limit this.

[0078] Furthermore, the motion sensor 17 of this embodiment obtains the collected attitude data through its built-in motion service module (SensorService).

[0079] In an alternative implementation, since a large amount of data is generated when the camera sensor 11 records video, for the convenience of RAW data storage and data transmission efficiency, the glasses 10 of this embodiment further include a data processing module 18 to compress the preprocessed data in the RAW domain. Further optionally, the data processing module 18 of this embodiment can store the preprocessed data, audio data, and attitude data in the RAW domain after data compression into the corresponding storage device 15 of the host controller 14 to wait for a data synchronization instruction from the mobile terminal 20 to perform data synchronization. In another alternative implementation, if the glasses 10 and the mobile terminal 20 maintain a data synchronization state, the data processing module 18 can, after compressing the preprocessed data, audio data, and attitude data in the RAW domain, send them to the mobile terminal 20 through the first communication unit 13 and the second communication unit 22.

[0080] Thus, when recording video data in this embodiment, the video frames, audio data, and attitude data involved in the video data are timestamped and then transmitted to the mobile terminal 20 respectively. There is no need to perform video encoding on the glasses 10 side, which further saves the encoder power consumption of video files and image files, that is, further reduces the power consumption of the glasses 10, avoids the heating of the glasses 10, and improves the user experience.

[0081] Further, when the mobile terminal 20 receives the image data, it performs image processing and image encoding on the image data to obtain a target image. When receiving the video frames, audio data, and attitude data, it performs image processing on the video frames based on the attitude data and performs video encoding based on the timestamps of the video frames and audio data to obtain a target video. Optionally, the terminal processor 21 of the mobile terminal 20 includes an audio-video decoder 211 for encoding and processing the image data and video data to obtain a target image and a target video.

[0082] Figure 6 is a flowchart of another data processing method according to an embodiment of the present invention. The flowchart of the data processing method of this embodiment is applicable to a data processing system as shown in Figure 5 shown, as shown in Figure 6 shown, the data processing method of the embodiment of the present invention includes the following steps:

[0083] Step S61, capturing raw data. If the currently captured raw data is video data, it includes video frames captured by the camera sensor 11, and may also include audio data recorded by the audio sensor 16 and / or attitude data collected by the motion sensor.

[0084] Step S62: Perform 3A preprocessing on the original data to obtain preprocessed data in the RAW domain. In this embodiment, the glasses 10 perform preprocessing on the original data captured by the camera sensor 11 through the camera processor 12 to obtain preprocessed data. Among them, the 3A preprocessing includes at least one of autofocus (AF), automatic exposure (AE), and automatic white balance (AWB).

[0085] Step S63: Perform compression processing on the preprocessed data in the RAW domain to reduce the storage space and improve the data transmission efficiency. Further, in this embodiment, compression processing can be performed on the preprocessed data in the RAW domain, or compression processing can be simultaneously performed on the audio data and / or the pose data.

[0086] Step S64: Send the processed preprocessed data in the RAW domain, audio data, and pose data to the mobile terminal 20.

[0087] Step S65: Perform image processing, as well as image encoding or video encoding processing, on the preprocessed data in the RAW domain to obtain a target image or a target video. When the mobile terminal 20 receives the image data, it performs image processing and image encoding on the image data to obtain a target image. When it receives video frames, audio data, and pose data, it performs image processing on the video frames based on the pose data, and performs video encoding based on the timestamps of the video frames and the audio data to obtain a target video.

[0088] In the embodiment of the present invention, the glasses include a camera sensor, a camera processor, and a first communication unit. The camera processor is used to perform preprocessing such as autofocus, automatic exposure, and / or automatic white balance on the original video data and / or original image data captured by the camera sensor to obtain preprocessed data, and send the preprocessed data to the mobile terminal through the first communication unit. The mobile terminal includes a terminal processor and a second communication unit. The terminal processor is configured to perform image processing on the preprocessed data received through the second communication unit to obtain a target video and / or a target image. The preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data. Thus, the glasses in this embodiment can directly send the RAW domain original data to the mobile terminal and perform corresponding image processing operations through the mobile terminal, which reduces the power consumption and hardware cost of the glasses.

[0089] Further, the glasses may further include an audio sensor and / or a motion sensor, and send the RAW domain preprocessed data, the audio sensor, and / or the motion sensor to the mobile terminal respectively, so that after the mobile terminal performs image processing, the image data is encoded to obtain a target image, or after the video frame performs image processing, the video frame and the audio data are video encoded to obtain a target video. Thus, in this embodiment, video encoding is not required on the glasses side, further saving the encoder power consumption of the video file and the image file, that is, further reducing the power consumption of the glasses, avoiding the heating situation of the glasses, and improving the user experience.

[0090] Figure 7 is a schematic diagram of another data processing system according to an embodiment of the present invention. As Figure 7 shown, the data processing system of this embodiment includes glasses 10, a mobile terminal 20, and a cloud server 30. Among them, the hardware architectures and data processing processes of the glasses 10 and the mobile terminal 20 are similar to those of the corresponding embodiments in Figures 1 - 6 and will not be elaborated here.

[0091] In this embodiment, if the user enables cloud synchronization, the mobile terminal 20 may synchronize the obtained target video and / or target image to the cloud server 30. The mobile terminal 20 may save the target video and / or target image, or controllably delete the target video and / or target image to save the storage space of the mobile terminal 20.

[0092] In an embodiment of the present invention, the glasses include a camera sensor, a camera processor, and a first communication unit. The camera processor is configured to perform preprocessing such as autofocus, autoexposure, and / or automatic white balance on the original video data and / or original image data captured by the camera sensor, obtain preprocessed data, and send the preprocessed data to the mobile terminal through the first communication unit. The mobile terminal includes a terminal processor and a second communication unit. The terminal processor is configured to perform image processing on the preprocessed data received through the second communication unit to obtain a target video and / or a target image. The preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data. Thus, the glasses in this embodiment can directly send the RAW domain original data to the mobile terminal and perform corresponding image processing operations through the mobile terminal, which reduces the power consumption and hardware cost of the glasses.

[0093] Another embodiment of the present invention relates to a non-volatile storage medium configured to store a computer-readable program, and the computer-readable program is configured to be executed by a computer to perform the above-mentioned partial or all method embodiments.

[0094] That is, those skilled in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0095] The foregoing are only the preferred embodiments of the present application and are not configured to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A data processing system, characterized in that: The system comprises: The glasses include a camera sensor, a camera processor, and a first communication unit, wherein the camera sensor is configured to capture raw data, the camera processor is configured to preprocess the raw data to obtain preprocessed data, and the first communication unit is configured to send the preprocessed data, wherein the raw data includes raw video data and / or raw image data, the preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data, and the preprocessing includes at least one of auto focus, auto exposure, and auto white balance; The mobile terminal comprises a terminal processor and a second communication unit, wherein the second communication unit is configured to receive the pre-processed data, and the terminal processor is configured to perform image processing on the pre-processed data to obtain a target video and / or a target image.

2. The system according to claim 1, characterized in that The image processing may be a combination of one or more of the following: lens correction processing, color conversion processing, noise reduction processing, sharpening processing, contrast adjustment, and brightness adjustment.

3. The system according to claim 1, characterized in that The glasses also include a host controller and a storage device corresponding to the host controller. The camera processor is connected to the host controller via an SPI interface or a USB interface. The camera processor stores the pre-processed data to the storage device via the SPI interface or the USB interface.

4. The system according to claim 3, characterized in that The glasses are configured to transmit the pre-processed data stored in the storage device to the mobile terminal through the first communication unit in response to receiving a data synchronization instruction from the mobile terminal.

5. The system according to claim 4, characterized in that The mobile terminal is configured to send a data deletion instruction to the glasses through the second communication unit in response to completion of data synchronization, and the glasses are configured to delete the pre-processed data in the storage device through the host controller in response to receiving the data deletion instruction.

6. The system according to claim 1, characterized in that The glasses further include an audio sensor and a motion sensor, the audio sensor is configured to acquire audio data, the motion sensor is configured to acquire posture data of the glasses, and the first communication unit is further configured to transmit the audio data and posture data to the mobile terminal respectively.

7. The system according to claim 6, characterized in that The terminal processor is provided with an audio and video encoder, and the terminal processor encodes information on the video frame after image processing, the audio data and the posture data through the audio and video encoder to generate a target video.

8. The system according to claim 1, characterized in that The system further comprises: The cloud server is configured to receive the target video and / or target image synchronized from the mobile terminal.

9. A pair of glasses, characterized in that: The glasses include: A camera sensor configured to capture raw data, wherein the raw data includes raw video data and / or raw image data; A camera processor is configured to preprocess the raw data to obtain preprocessed data, wherein the preprocessed data includes preprocessed RAW domain video data and / or RAW domain image data, and the preprocessing includes at least one of auto focus, auto exposure and auto white balance; The first communication unit is configured to send the pre-processed data to the mobile terminal, so that the mobile terminal performs image processing on the pre-processed data to obtain a target video and / or a target image.

10. A mobile terminal, characterized in that: The mobile terminal comprises: a second communication unit configured to receive pre-processed data from the glasses, wherein the pre-processed data includes pre-processed RAW domain video data and / or RAW domain image data, wherein the pre-processing includes at least one of auto focus, auto exposure and auto white balance; The terminal processor is configured to perform image processing on the preprocessed data to obtain a target video and / or a target image.

Images