Data processing method and related apparatus

Abstract

A data processing method and a related apparatus, which are applied to the technical field of data processing. The data processing method comprises: acquiring first data and second data (S701); and on the basis of the first data and the second data, generating a first video and / or any one or more image frames in the first video (S702), wherein the first data is used by a display apparatus to display augmented reality data of an environment in which a first vehicle is located, and the second data comprises data that is of the environment in which the first vehicle is located and is detected by a detection apparatus. In the present method, augmented reality data that is of an environment in which a first vehicle is located and is displayed by a display apparatus can be fused with data that is of the environment in which the first vehicle is located and is detected by a detection apparatus, so as to obtain a video and / or any one or more image frames in the video, which provide an augmented reality visual experience, such that augmented reality display content of an AR-HUD can be provided for viewing by a front-seat passenger and rear-seat passengers, and can also be replayed by a user after a trip ends, thereby enriching usage scenarios of the AR-HUD, and greatly improving the user experience.

Description

Data processing methods and related devices

[0001] This application claims priority to Chinese Patent Application No. 202510287664.9, filed on March 11, 2025, entitled “Data Processing Method and Related Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology, and in particular to a data processing method and related apparatus. Background Technology

[0003] With the development of the intelligent vehicle market, intelligent optical displays such as augmented reality head-up displays (AR-HUD) are increasingly becoming core components. AR-HUD combines holographic augmented reality (AR) technology with head-up display functionality (HUD), making digital information richer and more intuitive, thus improving the driving experience and safety. AR-HUD technology is gradually being widely applied in large, medium, and small passenger cars, commercial vehicles, and engineering vehicles, comprehensively enhancing the performance and experience of intelligent cockpits.

[0004] Currently, the augmented reality display content of AR-HUD is only for real-time viewing within a limited field of view (FOV) of the driver's position. Passengers in the front passenger seat and rear seats cannot view it, and users cannot review it after the trip ends. This has a significant limitation on the use cases of AR-HUD and affects the user experience. Summary of the Invention

[0005] This application provides a data processing method and related apparatus that can provide the augmented reality display content of AR-HUD to the front passenger and rear passengers for viewing. Users can also review the content after the trip, enriching the use cases of AR-HUD and greatly improving the user experience.

[0006] In a first aspect, embodiments of this application provide a data processing method applied to a first vehicle, the first vehicle including a display device and a detection device. The data processing method includes: acquiring first data and second data; and generating a first video and / or any one or more frames of images from the first video based on the first data and the second data. The first data is used by the display device to display augmented reality data of the environment in which the first vehicle is located, and the second data includes data of the environment in which the first vehicle is located detected by the detection device.

[0007] This application provides a data processing method that, based on augmented reality data of the environment where a first vehicle is located displayed on a display device and data of the environment detected by a detection device, can fuse these data to obtain a video and / or any one or more frames of images from the video that provide an augmented reality visual experience. Optionally, when the display device is an AR-HUD and the detection device is a camera (such as a dashcam), the data processing method according to this application can fuse the video and / or images that provide an augmented reality visual experience, and can provide this data for viewing by, but not limited to, the driver, front passenger, rear passengers, and other users. It can also support playback after the trip, greatly expanding the application scenarios of AR-HUD and improving the user experience.

[0008] Optionally, the first data can be used not only to display three-dimensional (3D) data of the environment in which the first vehicle is located, but also two-dimensional (2D) data. This application embodiment does not limit this.

[0009] Optionally, the environment in which the first vehicle is located may include, but is not limited to, road surface, road facilities, obstacles, weather, etc., and this application embodiment does not impose any restrictions on this.

[0010] In one possible implementation, the above-mentioned generation of a first video and / or any one or more frames of images in the first video based on the first data and the second data can be achieved in ways including but not limited to: fusing the first data and the second data corresponding to the same timestamp to obtain the first video and / or any one or more frames of images in the first video.

[0011] In this embodiment, data fusion can be performed based on the time dimension. Since the first data and the second data come from different devices, if the timestamps of the first data and the second data are not aligned during fusion, the fused video and / or image may have content errors. This application embodiment, by fusing the first data and the second data corresponding to the same timestamp, can obtain videos and / or images with higher accuracy and augmented reality visual experience, thereby improving the user's viewing experience.

[0012] In one possible implementation, the first data is data in a first reference coordinate system, where the origin of the first reference coordinate system is an observation point or a first calibration point inside the first vehicle. The second data is data in a second reference coordinate system, where the origin of the second reference coordinate system is the position of the detection device or a second calibration point. The generation of the first video and / or any one or more frames from the first video based on the first and second data can be implemented, but is not limited to, the following: converting the second data into data in the first reference coordinate system, fusing the first data and the converted second data to obtain the first video and / or any one or more frames from the first video, where the first video is a video in the first reference coordinate system.

[0013] In this embodiment, data fusion can be performed based on spatial dimensions. Since the first and second data originate from different devices and reside in different reference coordinate systems, the fused video and / or image may suffer from content misalignment. This embodiment converts the first and second data into data within the same reference coordinate system before fusion, resulting in more accurate videos and / or images with augmented reality visual experiences, thus improving the user's viewing experience.

[0014] Optionally, the first calibration point and / or the second calibration point can be adjusted according to the utilization rate of the interior space of the first vehicle, and this application embodiment does not limit this.

[0015] In one possible implementation, the above data processing method further includes, but is not limited to, the following steps: receiving a first instruction, the first instruction being used to instruct the display of a first video and / or any frame or multiple frames of images in the first video through a target device; and in response to the first instruction, sending the first video and / or any frame or multiple frames of images in the first video to the target device.

[0016] In this embodiment, by receiving and responding to a first instruction, the generated video and / or images can be displayed on the target device, providing the user with an augmented reality visual experience. Optionally, during driving, the generated video and / or images can be displayed on, but not limited to, the central control display screen, the passenger-side display screen, and the rear-seat display screen, through the embodiments of this application, so that passengers, including but not limited to the passenger-side and rear-seat passengers, can view content with an augmented reality visual experience from the driver's perspective in real time, improving the user's viewing experience.

[0017] Optionally, the first instruction may be, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0018] In one possible implementation, the target device includes at least one of the following: a central control display screen of a first vehicle, a passenger-side display screen of a first vehicle, a rear-seat display screen of a first vehicle, a mobile terminal, and a cloud.

[0019] In one possible implementation, the above data processing method further includes, but is not limited to, the following steps: in response to a first instruction, instructing a display device to display a first identifier in front of the observation point, the first identifier indicating that a first video and / or any frame or multiple frames of images in the first video are being synthesized.

[0020] In this embodiment, the first video can be generated in real time through human-computer interaction. By responding to the first command, the display device can be instructed to display a first identifier in front of the observation point to indicate that the first video is being generated, thus providing users with an immersive experience of human-computer interaction and improving user experience.

[0021] Optionally, the first identifier may include, but is not limited to, "start recording", "recording", "recording in progress", "recording completed", etc., and the embodiments of this application do not limit this.

[0022] In one possible implementation, the above data processing method further includes, but is not limited to, the following steps: receiving a second instruction, the second instruction being used to configure the display content and / or display style in the first data.

[0023] In this embodiment, by receiving and responding to the second instruction, the display content and / or display style in the first data can be configured to generate videos and / or images that meet the user's expectations, thereby improving the user's augmented reality visual experience. Optionally, after the trip ends and the vehicle stops, the augmented reality display content and / or display style can be arbitrarily configured through the embodiments of this application to generate videos and / or images with an augmented reality visual experience that meet the user's expectations, thereby improving the user experience.

[0024] Optionally, the second instruction may be, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0025] In one possible implementation, the display content in the first data includes at least one of the following: driving data, location, time, weather, navigation information, intelligent driving information, and status information of the first vehicle; and / or, the display style in the first data includes at least one of the following: font, size, color, transparency, brightness, and display style of the display content.

[0026] In one possible implementation, the above data processing method further includes, but is not limited to, the following steps: receiving a third instruction, the third instruction being used to instruct the export of a first video and / or any frame or multiple frames of images in the first video, and exporting the first video and / or any frame or multiple frames of images in response to the third instruction.

[0027] In this embodiment, by receiving and responding to a third instruction, it is possible to export generated videos and / or images with augmented reality visual experience, which can be provided to users including but not limited to the driver, front passenger, rear passengers and other users for viewing. It can also be provided to users for review after the trip, which greatly expands the use cases of AR-HUD and improves the user experience.

[0028] Optionally, the third instruction may include, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0029] In one possible implementation, the above data processing method further includes, but is not limited to, the following steps: receiving a fourth instruction, the fourth instruction being used to instruct the first video and / or any frame or multiple frames of the first video to be sent to a target address; in response to the fourth instruction, sending the first video or an identifier corresponding to the first video to the target address, or sending any frame or multiple frames of the first video and the corresponding identifier to the target address.

[0030] In this embodiment, by receiving and responding to the fourth instruction, it is possible to share the generated videos and / or images with augmented reality visual experience to other platforms, providing them to a wider range of users for viewing. It is also possible to provide users with a review after the trip, which greatly expands the use cases of AR-HUD and improves the user experience.

[0031] Optionally, the identifier corresponding to the first video may specifically be a Uniform Resource Locator (URL), and this embodiment of the application does not limit this.

[0032] Optionally, the fourth instruction may include, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0033] Secondly, embodiments of this application provide a data processing apparatus, which includes units for performing the method as described in any of the first aspects.

[0034] In one possible design, the device includes:

[0035] The processing unit is used to acquire first data and second data. The first data is used by the display device to display augmented reality data of the environment in which the first vehicle is located. The second data includes data of the environment in which the first vehicle is located, detected by the detection device.

[0036] The processing unit is also configured to generate a first video and / or any frame or multiple frames of images in the first video based on the first data and the second data.

[0037] In one possible implementation, the device further includes a communication unit.

[0038] The processing unit is specifically used to acquire first data and second data through the communication unit.

[0039] Regarding the processing unit and communication unit described in the second aspect and any possible implementation, the steps performed thereon can be referred to the corresponding implementations in the first aspect.

[0040] For the technical effects of the second aspect and any possible implementation, please refer to the description of the technical effects corresponding to the first aspect and the corresponding implementation.

[0041] Optionally, in the data processing apparatus described in the second aspect above and any possible implementation:

[0042] In one implementation, the data processing apparatus is a data processing device. When the data processing apparatus is a data processing device, the communication unit can be a transceiver or an input / output interface; the processing unit can be at least one processor. Optionally, the transceiver can be a transceiver circuit. Optionally, the input / output interface can be an input / output circuit.

[0043] In another implementation, the data processing device is a chip (system) or circuit used in a data processing device. When the data processing device is a chip (system) or circuit used in a data processing device, the communication unit can be a communication interface (input / output interface), interface circuit, output circuit, input circuit, pin, or related circuit on the chip (system) or circuit; the processing unit can be at least one processor, processing circuit, or logic circuit.

[0044] Thirdly, embodiments of this application provide a data processing apparatus including a processor. The processor is coupled to a memory and can be used to execute instructions in the memory to implement the methods described in the first aspect and any of the possible implementations. Optionally, the data processing apparatus further includes a memory. Optionally, the data processing apparatus further includes a communication interface, and the processor is coupled to the communication interface.

[0045] Fourthly, embodiments of this application provide a chip, including: logic circuitry and a communication interface. The communication interface is used to receive or send information; the logic circuitry is used to receive or send information through the communication interface, causing the chip to execute the methods described in the first aspect and any of the possible implementations.

[0046] Fifthly, embodiments of this application provide a computer-readable storage medium for storing a computer program (also referred to as code or instructions); when the computer program is run on a computer, the methods described in the first aspect and any possible implementation are implemented.

[0047] Sixthly, embodiments of this application provide a computer program product, the computer program product comprising: a computer program (also referred to as code or instructions); and, when the computer program is run, causing a computer to perform the methods described in the first aspect and any possible implementation thereof.

[0048] In a seventh aspect, embodiments of this application provide a cockpit including a display device and at least one data processing device as described in the second aspect, or the third aspect, or the fourth aspect.

[0049] Eighthly, embodiments of this application provide a terminal that includes at least one data processing device as described in the second aspect, or the data processing device as described in the third aspect, or the chip as described in the fourth aspect, or the cockpit as described in the seventh aspect.

[0050] Optionally, the terminal can be a means of transportation, such as a car, truck, aircraft, drone, slow transport vehicle, spacecraft, or ship, or any other possible means of transportation used in any possible scenario. This application embodiment does not limit this.

[0051] Optionally, the terminal is used to implement the method described in the first aspect or any possible implementation of the first aspect.

[0052] Furthermore, in the process of performing the method described in the first aspect and any possible implementation above, the processes related to sending and / or receiving information in the above methods can be understood as the process of the processor outputting information, and / or the process of the processor receiving input information. When outputting information, the processor can output the information to a transceiver (or communication interface, or transmitting module) so that the transceiver can transmit it. After the information is output by the processor, it may need to undergo other processing before reaching the transceiver. Similarly, when the processor receives input information, the transceiver (or communication interface, or transmitting module) receives the information and inputs it to the processor. Furthermore, after the transceiver receives the information, the information may need to undergo other processing before being input to the processor.

[0053] Based on the above principles, for example, the information sent mentioned in the aforementioned method can be understood as information output by the processor. Similarly, the information received can be understood as information received by the processor from input.

[0054] Optionally, unless otherwise specified, or unless they contradict their actual function or internal logic in the relevant description, the operations of the processor, such as transmitting, sending, and receiving, can be more generally understood as processor output and receiving, input, and other operations.

[0055] Optionally, in performing the methods described in the first aspect and any possible implementation above, the processor may be a processor specifically designed to perform these methods, or it may be a processor that performs these methods by executing computer instructions stored in memory, such as a general-purpose processor. The memory may be a non-transitory memory, such as read-only memory (ROM), which may be integrated with the processor on the same chip or disposed on different chips. This application does not limit the type of memory or the arrangement of the memory and processor.

[0056] In one possible implementation, at least one of the aforementioned memories is located outside the device.

[0057] In yet another possible implementation, at least one of the aforementioned memories is located within the device.

[0058] In another possible implementation, a portion of the memory of the at least one memory is located inside the device, while another portion is located outside the device.

[0059] In this application, the processor and memory may also be integrated into a single device, that is, the processor and memory can be integrated together. Attached Figure Description

[0060] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 is a schematic diagram of an application scenario of a display device in the automotive field provided by an embodiment of this application;

[0062] Figure 2 is a schematic diagram from another perspective of an application scenario of a display device in the automotive field provided by an embodiment of this application;

[0063] Figure 3 is a schematic diagram of an image projection onto a driver's eye in the automotive field using a display device provided in an embodiment of this application;

[0064] Figure 4 is a schematic diagram showing a driving scenario provided in an embodiment of this application;

[0065] Figure 5 is a schematic diagram of the architecture of a data processing system provided in an embodiment of this application;

[0066] Figure 6 is a schematic diagram of generating first data according to an embodiment of this application;

[0067] Figure 7 is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0068] Figure 8 is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0069] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0070] Figure 10 is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described below with reference to the accompanying drawings.

[0072] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0073] The term "embodiment" as used herein means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that, unless otherwise specified or logically conflicting, the terminology and / or descriptions between the various embodiments of this application are consistent and can be mutually referenced, and technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0074] It should be understood that in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0075] It should be noted that, in this application, "instruction" can include direct instruction, indirect instruction, explicit instruction, and implicit instruction. When describing a certain instruction information for the purpose of instructing A, it can be understood that the instruction information carries A, directly instructs A, or indirectly instructs A.

[0076] In this application, the information indicated by the instruction information is called the information to be instructed. In specific implementations, there are many ways to indicate the information to be instructed, such as, but not limited to, directly indicating the information to be instructed, such as the information to be instructed itself or its index. It can also indirectly indicate the information to be instructed by indicating other information, where there is a correlation between the other information and the information to be instructed. It can also indicate only a part of the information to be instructed, while the other parts are known or pre-agreed upon. For example, the instruction of specific information can be achieved by using a pre-agreed (e.g., protocol-defined) arrangement of various information, thereby reducing instruction overhead to some extent. The information to be instructed can be sent as a whole or divided into multiple sub-information units, and the sending period and / or timing of these sub-information units can be the same or different. This application does not limit the specific sending method. The sending period and / or timing of these sub-information units can be predefined, for example, according to a protocol, or configured by the transmitting device by sending configuration information to the receiving device.

[0077] It should be noted that in this application, "send" can be understood as "output" and "receive" can be understood as "input". "Send information to A", where "to A" simply indicates the direction of information transmission, and A is the destination, does not limit "send information to A" to a direct transmission over the air interface. "Send information to A" includes sending information directly to A, as well as sending information indirectly to A through a transmitter. Therefore, "send information to A" can also be understood as "outputting information destined for A". Similarly, "receive information from A" indicates that the source of the information is A, including receiving information directly from A, as well as receiving information indirectly from A through a receiver. Therefore, "receive information from A" can also be understood as "inputting information from A".

[0078] To more clearly describe the solution of this application, some terms used in the embodiments of this application will be explained below.

[0079] A head-up display (HUD), also known as a head-up display system, is a multi-functional instrument panel centered on the driver and operated blindly. The HUD projects vehicle instrument information (such as speed, RPM, temperature, fuel level, etc.) and navigation information into the driver's field of vision. The driver can see this information directly in front of them without needing to look down at the instrument panel or central control display below the steering wheel, thus improving braking reaction time in emergency situations and enhancing driving safety and convenience.

[0080] Augmented reality head-up display (AR-HUD) is a combination of holographic augmented reality (AR) technology and head-up display (HUD) functionality. Through computational processing, it overlays acquired image information onto the real three-dimensional environment in front of the driving route. It can integrate information such as distance, speed, and navigation into the real-world view obtained by the driver. The combination of computational data and display scene makes digital information richer and more intuitive, improving the driving experience and driving safety.

[0081] Please refer to Figures 1 to 3. Figure 1 shows a schematic diagram of an application scenario of a display device 100 provided in this application in the automotive field. Figure 2 shows another perspective schematic diagram of an application scenario of a display device 100 provided in this application in the automotive field. Figure 3 shows an imaging schematic diagram of a display device 100 provided in this application projecting an image into the driver's eyes in the automotive field, illustrating the virtual image information that the display device 100 in Figures 1 and 2 can see when projecting images into the driver's eyes.

[0082] As shown in Figures 1 to 3, when the driver looks out of the windshield 200, they can see a virtual image with a certain depth of field (i.e., the distance of the clear image within the range before and after the focal point of the image). Information acquisition modules, such as cameras and infrared sensors, acquire information about the road ahead, including distance to the vehicle in front, the speed of the vehicle in front, and other road information. This digital information is projected onto an image closely integrated with the surrounding environment. This digital information includes: vehicle speed, navigation information (road directions), vehicle gear, cruise control, distance to the vehicle in front, speed of the vehicle in front, engine speed, battery level, remaining driving range, and infotainment system information. This allows the driver to access necessary information without shifting their gaze while driving, avoiding the driving risks that might arise from neglecting road conditions while looking down at the instrument panel or central control screen.

[0083] To achieve the aforementioned display, the vehicle needs to be equipped with a head-up display (HUD) system. This HUD system may include a display device 100 and optical elements. The display device 100 can project images, and the optical elements may include the vehicle's windshield 200, a reflective film attached to the windshield 200, or a separate screen within the vehicle's cabin. The optical elements can reflect and / or refract the image projected by the display device 100 before projecting it into the driver's eyes, allowing the driver to obtain a virtual information image in front of the windshield 200. Optionally, the display device 100 has one or more interfaces through which current and / or image signals are obtained from the vehicle's central control system or in-vehicle infotainment system. The current signals are used to power the electrical components in the display device 100, and the image signals are used to provide the displayed image.

[0084] In the automotive industry, the display device 100 that enables enhanced head-up display functionality is typically located on the center console of the vehicle. Referring to Figures 1, 2, and 3, the display device 100 can be installed within the center console and positioned in front of the steering wheel (aligned with the direction of travel of the vehicle).

[0085] It should be noted that the vehicles described in this application are all exemplified as automobiles and should not be considered as limitations on the embodiments of this application. The vehicle can be a traditional gasoline-powered vehicle, or a new energy vehicle such as a pure electric vehicle or a hybrid vehicle. The vehicle can be any of different types of automobiles, such as sedans, trucks, buses, and sport utility vehicles (SUVs), or it can be a land transportation device for carrying passengers or goods, such as a tricycle, motorcycle, or train. Alternatively, the display device of this application can also be applied to other types of transportation vehicles such as airplanes and ships.

[0086] Based on the display device 100 shown in Figures 1 to 3, the displayed content can be projected onto the AR-HUD display area, allowing the driver to see digital information with an augmented reality visual experience from a specific perspective. See Figure 4 for details; Figure 4 is a schematic diagram of a driving scenario provided by an embodiment of this application.

[0087] As shown in Figure 4, when the vehicle is in a lane-changing scenario, from the driver's perspective, the AR-HUD can be seen overlaying some AR data (3D data) and non-AR data (2D data) onto the real-world 3D environment in which the vehicle is located. The AR data includes, but is not limited to, AR lane change prompts (such as virtual lane indicator arrows and virtual lane change guide lines displayed directly on the actual road surface), while the non-AR data includes, but is not limited to, vehicle speed (e.g., 85 km / h), speed limit (e.g., 90 km / h), and navigation information (e.g., a right turn 60m ahead). The real-world 3D environment in which the vehicle is located includes, but is not limited to, the road where the vehicle is located, road facilities, vehicles or obstacles on the road, and weather conditions.

[0088] As shown in Figure 4, the AR and non-AR data displayed by the AR-HUD are integrated into the real three-dimensional environment in which the vehicle is located, making the digital information with augmented reality visual experience seen from the driver's perspective richer and more intuitive, thus improving the driving experience and driving safety.

[0089] However, the current augmented reality display content of AR-HUD (i.e., the display content similar to that shown in Figure 4 above) is only for real-time viewing within a limited field of view (FOV) of the driver's position. Passengers in the front passenger seat and rear seats cannot view it, and users cannot review it after the trip ends. This has a significant limitation on the application scenarios of AR-HUD and affects the user experience.

[0090] Therefore, how to enrich the use cases of AR-HUD and improve the user experience is a technical problem that needs to be solved in this field.

[0091] In view of this, the present application provides a data processing system, and based on the data processing system, provides a new data processing method and related apparatus, relating to the field of data processing technology. It can provide the augmented reality display content of AR-HUD to the front passenger and rear passengers for viewing, and users can also review it after the trip ends, enriching the use scenarios of AR-HUD and greatly improving the user experience.

[0092] Please refer to Figure 5, which is a schematic diagram of the architecture of a data processing system provided in an embodiment of this application.

[0093] As shown in Figure 5, the data processing system mainly includes, but is not limited to, AR-HUD, camera (driving recorder), and data processing module.

[0094] The AR-HUD is used to transmit the first data to the data processing module, and the camera is used to transmit the second data to the data processing module.

[0095] Optionally, the second data may specifically include video data of the external environment of the vehicle, which can be obtained through a camera.

[0096] Optionally, the first data can be obtained through the generation method shown in Figure 6. As shown in Figure 6, optionally, the first data obtained through generation method one can include data generated after rendering the original data using an AR engine, such as: data generated after rendering the video stream of a multimedia application in the vehicle system using an AR engine, data generated after rendering the data detection results of an advanced driving assistance system (ADAS) using an AR engine, and so on. Optionally, the first data obtained through generation method two can also include data generated after rendering the original data with environmental data collected by a camera using an AR engine, such as: data generated after rendering the navigation data of a navigation application in the vehicle system with environmental data collected by a digital video recorder (DVR).

[0097] Optionally, the original data (e.g., video streams from multimedia applications in the vehicle infotainment system, data detection results from ADAS, etc.) and the first data generated after rendering by the AR engine are located in different reference coordinate systems. Optionally, the original data (e.g., navigation data from navigation applications in the vehicle infotainment system) and the first data generated after rendering by the AR engine are fused with environmental data collected by the camera.

[0098] Optionally, the aforementioned first data can also be displayed in real time in the AR-HUD display area via AR-HUD, providing it to the user in the driver's view.

[0099] The data processing module fuses the first and second data sets to generate an AR video. Because this AR video integrates data about the vehicle's environment and rendered 3D data, it offers an augmented reality visual experience, enhancing the user experience.

[0100] Optionally, the generated AR video can be displayed in real time through various display devices, such as the vehicle's central control display, the vehicle's passenger-side display, the vehicle's rear-seat display, and mobile terminals, so that passengers, including but not limited to the passenger in the front seat and rear seat, can watch content with an augmented reality visual experience from the driver's perspective in real time, thereby improving the user's viewing experience.

[0101] Optionally, the generated AR videos can also be exported and shared, supporting the sharing of generated videos and / or images with augmented reality visual experience to other platforms, providing them to a wider range of users. They can also be provided to users for review after the trip, greatly expanding the use cases of AR-HUD and improving the user experience.

[0102] Based on the architecture of this data processing system, the data processing method provided in this application can be executed to provide the augmented reality display content of AR-HUD to the front passenger and rear passengers for viewing. Users can also review the content after the trip, which enriches the use cases of AR-HUD and greatly improves the user experience.

[0103] Please refer to Figure 7, which is a flowchart illustrating a data processing method provided in an embodiment of this application. This data processing method is applied in the field of data processing technology, such as data fusion processing of AR-HUD rendering data and camera video stream data.

[0104] Specifically, the data processing method is applied to a first vehicle, which includes a display device and a detection device. Optionally, the display device may refer to the AR-HUD in Figure 6 above, and the detection device may refer to the camera (driving recorder) in Figure 6 above.

[0105] Specifically, the data processing method includes, but is not limited to, the following steps:

[0106] S701: The data processing device acquires the first data and the second data.

[0107] S702: The data processing device generates a first video and / or any frame or multiple frames of images in the first video based on the first data and the second data.

[0108] It is understood that the data processing device in this application embodiment may be a device equipped with a processor / chip that can execute computer execution instructions, or it may be a processor / chip that can execute computer execution instructions. Optionally, the data processing device may be an electronic device, or it may be a processor / chip within an electronic device. Optionally, the data processing device may specifically be the data processing module in Figure 5 above, used to execute the data processing method in this application embodiment to provide the augmented reality display content of AR-HUD to the front passenger and rear passengers for viewing. Users can also review the content after the trip, enriching the use cases of AR-HUD and greatly improving the user experience.

[0109] Optionally, the data processing device and data processing method in the embodiments of this application can be applied to, but are not limited to, vehicle systems. The vehicle equipped with the vehicle system is an intelligent driving vehicle and can be replaced by a terminal device. The terminal device can be, but is not limited to, vehicles such as commercial vehicles, passenger cars, trains, industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), robots, etc. The embodiments of this application do not specifically limit this.

[0110] The first data is used to display augmented reality data of the environment in which the first vehicle is located, and the second data includes data of the environment in which the first vehicle is located, which is detected by the detection device.

[0111] Optionally, the environment in which the first vehicle is located may include, but is not limited to, the road surface, road facilities, obstacles, weather, etc., and may vary depending on the driving scenario. This application embodiment does not impose any restrictions on this.

[0112] Optionally, the first data can be used not only to display three-dimensional (3D) data of the environment in which the first vehicle is located, but also two-dimensional (2D) data. For details, please refer to the relevant descriptions of the first data in Figures 5 and 6 above, which will not be repeated here.

[0113] For example, the possible display content of the first data is given in different driving scenarios, as shown in Table 1 below:

[0114] Table 1

[0115] As shown in Table 1, the data generated by the AR engine after rendering the three-dimensional and / or two-dimensional data that may be included in the first data in navigation, warning, autonomous driving, and infotainment scenarios are respectively presented.

[0116] For example, in a lane-changing scenario, the first data may include data generated by rendering lane data collected by a camera and then processed by an AR engine, which is a 3D data fusion of AR lane change prompts. It may also include data generated by rendering 2D data such as vehicle speed, speed limit, and navigation information using an AR engine. Optionally, the display rules corresponding to this first data in a lane-changing scenario are also provided, i.e., a lane-changing prompt is displayed in a different color when a lane-changing action occurs.

[0117] For example, in a turning scenario, the first data may include data generated by rendering lane data collected by a camera using 3D data fusion for AR lane prompts and 3D turn alerts, as well as data generated by rendering 2D data such as vehicle speed, speed limit, and navigation information using an AR engine. Optionally, the display rules corresponding to this first data in a turning scenario are also given, that is, a turn signal color change is displayed when a turning action occurs.

[0118] Similarly, for the specific details of the first data in other scenarios, please refer to the relevant descriptions in Table 1 above, which will not be repeated here.

[0119] It should be understood that Table 1 above is only an illustrative example of several common possible driving scenarios and should not be construed as limiting the embodiments of this application. It should be understood that any new embodiments obtained by reasonable modifications, supplements, or combinations based on Table 1 above are all within the protection scope of this application.

[0120] It is understood that, based on the augmented reality data of the environment in which the first vehicle is located displayed by the aforementioned display device and the data of the environment in which the first vehicle is located detected by the detection device, it is possible to fuse them to obtain a first video with augmented reality visual experience and / or any frame or multiple frames of images in the first video.

[0121] Optionally, when the display device is an AR-HUD and the detection device is a camera (such as a dashcam), the data processing method according to the embodiments of this application can be used to fuse videos and / or images with augmented reality visual experience, and can be provided to users including but not limited to the driver, front passenger, rear passengers and other users for viewing. It can also support users to review after the trip, which greatly expands the use cases of AR-HUD and improves the user experience.

[0122] In one possible embodiment, step S702 described above can be implemented in ways including but not limited to the following:

[0123] The data processing device fuses the first data and the second data corresponding to the same timestamp to obtain the first video and / or any frame or multiple frames of the first video.

[0124] It is understood that the embodiments of this application can fuse the first data and the second data based on the time dimension.

[0125] Since the first and second data come from different devices, if the timestamps of the first and second data are not aligned during fusion, the fused video and / or image may have content errors.

[0126] This application embodiment can obtain more accurate videos and / or images with augmented reality visual experience by fusing first data and second data corresponding to the same timestamp, thereby improving the user's viewing experience.

[0127] In one possible embodiment, the first data is data in a first reference coordinate system, where the origin of the first reference coordinate system is an observation point or a first calibration point inside the first vehicle. The second data is data in a second reference coordinate system, where the origin of the second reference coordinate system is the position of the detection device or a second calibration point. Step S702 can be implemented in ways including, but not limited to, the following:

[0128] The data processing device converts the second data into data in the first reference coordinate system, and fuses the first data and the converted second data to obtain the first video and / or any frame or multiple frames of the first video.

[0129] The first video is a video in the first reference coordinate system.

[0130] Optionally, the first calibration point and / or the second calibration point can be adjusted according to the utilization rate of the interior space of the first vehicle, and this application embodiment does not limit this.

[0131] It is understood that the embodiments of this application can fuse the first data and the second data based on the spatial dimension.

[0132] Because the first and second data come from different devices and have different reference coordinate systems, the fused video and / or image may have content misalignment issues.

[0133] This application embodiment achieves higher accuracy in videos and / or images with augmented reality visual experience by converting the first data and the second data into data under the same reference coordinate system before fusing them, thereby improving the user's viewing experience.

[0134] In one possible embodiment, the above data processing method further includes, but is not limited to, the following steps:

[0135] The data processing device receives a first instruction, which instructs the target device to display a first video and / or any frame or multiple frames of images in the first video.

[0136] In response to the first instruction, the data processing device sends the first video and / or any frame or multiple frames of the first video to the target device.

[0137] Optionally, the first instruction may be, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0138] Optionally, the target device may include, but is not limited to, the central control display screen of the first vehicle, the passenger-side display screen of the first vehicle, the rear-seat display screen of the first vehicle, a mobile terminal, the cloud, etc., and the embodiments of this application do not limit this.

[0139] Understandably, by receiving and responding to the first instruction, the generated video and / or images can be displayed on the target device, providing users with an augmented reality visual experience.

[0140] Optionally, during the driving of the first vehicle, the generated video and / or images can be displayed on, but not limited to, the central control display screen, the passenger-side display screen, and the rear-seat display screen through the embodiments of this application, so that passengers, including but not limited to the passenger-side and rear-seat passengers, can watch content with augmented reality visual experience from the driver's perspective in real time, thereby improving the user's viewing experience.

[0141] For example, while the first vehicle is in motion, if a user wants to view the HUD AR display effect from the driver's perspective in real time via the central control screen / passenger screen, they can perform operations including but not limited to the following:

[0142] Step 1: The user enters the HUD interface on the central control screen / passenger screen.

[0143] Step 2: The user selects "Project AR Video".

[0144] Step 3: The central control screen / passenger screen displays AR content from the driver's perspective (including the real-world environment outside the vehicle and AR rendering data displayed by the AR-HUD).

[0145] Optionally, the above data processing method may also include, but is not limited to, the following steps:

[0146] The data processing device responds to the first instruction and instructs the display device to display the first mark in front of the observation point.

[0147] The first identifier indicates that the first video and / or any frame or multiple frames of images in the first video are being synthesized.

[0148] Optionally, the first identifier may include, but is not limited to, "start recording", "recording", "recording in progress", "recording completed", etc., and the embodiments of this application do not limit this.

[0149] It is understandable that the first video can be generated in real time through human-computer interaction, and the first video can be generated by displaying a first identifier in front of the observation point in response to the first command, thus bringing an immersive human-computer interaction experience to the user and improving the user experience.

[0150] For example, during the driving of the first vehicle, if a user wishes to record a video that blends the distinctive scenery seen by the human eye with the AR rendered data displayed by the AR-HUD, the following operations can be performed, including but not limited to:

[0151] Step 1: The driver initiates real-time video recording via voice and requests the addition of HUD information.

[0152] Step 2: The system responds to the customer with a voice message, and the HUD displays a "Recording" icon.

[0153] Step 3: The driver stops recording via voice command.

[0154] Step 4: The system responds to the customer with a voice message, and the HUD displays a "Recording Complete" icon, which disappears after 3 seconds. At this time, the recorded AR video is automatically stored in the designated location for the user to view.

[0155] In one possible embodiment, the above data processing method further includes, but is not limited to, the following steps:

[0156] The data processing device receives a second instruction for configuring the display content and / or display style in the first data.

[0157] Optionally, the second instruction may include, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0158] Optionally, the content displayed in the first data includes at least one of the following: driving data, location, time, weather, navigation information, intelligent driving information, and status information of the first vehicle.

[0159] Optionally, the status information of the first vehicle includes, but is not limited to, vehicle speed, gear, steering, lights, acceleration / deceleration, and overtaking. Optionally, the status information of the first vehicle may also include the vehicle's driving mode information, such as the aforementioned status information when the vehicle is in track drift mode; this embodiment of the application does not impose any limitations on this.

[0160] Optionally, the display style in the first data includes at least one of the following: the font, size, color, transparency, brightness, and display style of the displayed content.

[0161] Optionally, the style of the background of the first video and / or any frame or multiple frames of the first video can be adjusted by configuring the display style of the first data.

[0162] It is understandable that by receiving and responding to the second instruction, the display content and / or display style in the first data can be configured, thereby generating videos and / or images that meet the user's expectations and improving the user's visual experience of augmented reality.

[0163] Optionally, after the trip ends and the vehicle stops, the augmented reality display content and / or display style can be arbitrarily configured through the embodiments of this application to generate videos and / or images with an augmented reality visual experience that meet the user's expectations, thereby improving the user experience.

[0164] For example, after parking at the end of the trip, if a user wants to configure the AR rendering content and effects displayed on the AR-HUD to obtain the desired AR video, they can perform operations including but not limited to the following:

[0165] Step 1: The user turns on the AR-HUD. The DVR records the forward video stream of the vehicle in the background in real time, and at the same time records all the data information required by the AR-HUD in real time when the AR-HUD is turned on.

[0166] Step 2: When the user plays back the DVR video, they can choose to configure the AR rendering content and effects of the AR-HUD display. After the configuration is completed, the data processing device will redraw the new AR-HUD display effect and overlay it on the DVR video to obtain the desired AR video, which will be saved to the specified location for the user to use.

[0167] In one possible embodiment, the above data processing method further includes, but is not limited to, the following steps:

[0168] The data processing device receives a third instruction, which instructs the export of the first video and / or any frame or multiple frames of the first video.

[0169] In response to a third instruction, the data processing device exports the first video and / or any frame or multiple frames of images from the first video.

[0170] Optionally, the third instruction may include, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0171] Understandably, by receiving and responding to third-party instructions, it can support the export of generated videos and / or images with augmented reality visual experiences, which can be provided to users including but not limited to the driver, front passenger, rear passengers, and other users for viewing. It can also support providing users with playback after the trip, greatly expanding the use cases of AR-HUD and improving the user experience.

[0172] In one possible embodiment, the above data processing method further includes, but is not limited to, the following steps:

[0173] The data processing device receives a fourth instruction, which instructs the first video and / or any frame or multiple frames of the first video to be sent to the target address.

[0174] In response to the fourth instruction, the data processing device sends the first video or the identifier corresponding to the first video to the target address, or sends any one or more frames of the first video and the corresponding identifier to the target address.

[0175] Optionally, the identifier corresponding to the first video may specifically be a Uniform Resource Locator (URL), and this application embodiment does not limit this.

[0176] Optionally, the fourth instruction may include, but is not limited to, instructions in the form of voice, buttons, gestures, or central control interface operations; this application embodiment does not impose any restrictions on this.

[0177] Understandably, by receiving and responding to the fourth instruction, it is possible to support sharing the generated videos and / or images with augmented reality visual experience to other platforms, providing them to a wider range of users for viewing. It can also support providing users with a review after the trip, greatly expanding the use cases of AR-HUD and improving the user experience.

[0178] Optionally, before sharing the generated AR videos and / or images to other platforms, the content to be shared can be de-identified, and the user's privacy information can be deleted before sharing. This application embodiment does not limit this.

[0179] The methods of the embodiments of this application have been described in detail above. The following provides an apparatus for implementing any one of the methods in the embodiments of this application. For example, an apparatus is provided that includes a unit (or means) for implementing the steps performed by the device in any of the above methods.

[0180] Please refer to Figure 8, which is a schematic diagram of the structure of a data processing device provided in an embodiment of this application.

[0181] As shown in Figure 8, the data processing device 80 may include a communication unit 801 and a processing unit 802. The communication unit 801 and the processing unit 802 may be software, hardware, or a combination of software and hardware.

[0182] The communication unit 801 can implement sending and / or receiving functions, and can also be described as a transceiver unit. The communication unit 801 can also be a unit integrating an acquisition unit and a sending unit, wherein the acquisition unit is used to implement the receiving function, and the sending unit is used to implement the sending function. Optionally, the communication unit 801 can be used to receive information sent by other devices, and can also be used to send information to other devices.

[0183] In one possible design, the data processing device 80 may correspond to the data processing device in the method embodiment shown in FIG. 7 above. For example, the data processing device 80 may be an electronic device or a chip within an electronic device. The data processing device 80 may include units for performing the operations performed by the data processing device in the method embodiment shown in FIG. 7 above, and each unit in the data processing device 80 is for implementing the operations performed by the data processing device in the method embodiment shown in FIG. 7 above. The descriptions of each unit are as follows:

[0184] The processing unit 802 is used to acquire first data and second data. The first data is used by the display device to display augmented reality data of the environment in which the first vehicle is located. The second data includes data of the environment in which the first vehicle is located, detected by the detection device.

[0185] The processing unit 802 is also configured to generate a first video and / or any frame or multiple frames of images in the first video based on the first data and the second data.

[0186] In one possible implementation, the device further includes a communication unit 801.

[0187] The processing unit 802 is specifically used to acquire first data and second data through the communication unit 801.

[0188] Regarding the communication unit 801 and processing unit 802 described in this design, the steps they perform can be referred to the implementation method corresponding to the data processing device in the method embodiment shown in FIG7 above.

[0189] Regarding the technical effects of the implementation methods performed by the communication unit 801 and the processing unit 802 described in this design, please refer to the description of the technical effects corresponding to the method embodiment shown in FIG7 above.

[0190] According to embodiments of this application, the various units in the device shown in FIG8 can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the electronic device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0191] It should be noted that the implementation of each unit can also refer to the corresponding description of the method embodiment shown in Figure 7 above.

[0192] In the data processing device 80 described in Figure 8, the augmented reality display content of AR-HUD can be provided to the front passenger and rear passengers for viewing. Users can also review it after the trip, which enriches the use scenarios of AR-HUD and greatly improves the user experience.

[0193] If the data processing device 80 described above can be an electronic device, please refer to the structural schematic diagram of the electronic device shown in Figure 9.

[0194] It should be understood that the electronic device 90 shown in FIG9 is only an example. The electronic device in the embodiments of this application may also include other components, or include components that have similar functions to the various components in FIG9, or may not include all the components in FIG9.

[0195] The electronic device 90 includes a transceiver interface 901 and at least one processor 902.

[0196] The electronic device 90 can correspond to a data processing device. The transceiver interface 901 is used for transmitting and receiving signals, and at least one processor 902 executes program instructions, causing the electronic device 90 to implement the corresponding flow of the method executed by the corresponding device in the above method embodiments.

[0197] In one possible design, the electronic device 90 may correspond to the data processing device in the method embodiment shown in FIG7 above. For example, the electronic device 90 may be a data processing device or a chip within the data processing device. The electronic device 90 may include components for performing the operations performed by the data processing device in the method embodiment above, and each component in the electronic device 90 is respectively for implementing the operations performed by the data processing device in the method embodiment above. Specifically, it may be as follows:

[0198] The processor 902 is used to acquire first data and second data. The first data is used by the display device to display augmented reality data of the environment in which the first vehicle is located. The second data includes data of the environment in which the first vehicle is located, detected by the detection device.

[0199] The processor 902 is also configured to generate a first video and / or any frame or multiple frames of images in the first video based on the first data and the second data.

[0200] In one possible implementation, the device further includes a transceiver interface 901.

[0201] The processor 902 is specifically used to acquire first data and second data through the transceiver interface 901.

[0202] Regarding the transceiver interface 901 and at least one processor 902 described in this design, the steps performed can be referred to the implementation corresponding to the data processing device in the method embodiment shown in FIG7 above.

[0203] For the technical effects of the implementation methods performed by the transceiver interface 901 and at least one processor 902 described in this design, please refer to the description of the technical effects corresponding to the method embodiment shown in FIG7 above.

[0204] In the electronic device 90 described in Figure 9, the augmented reality display content of AR-HUD can be provided to the front passenger and rear passengers for viewing. Users can also review the content after the trip, which enriches the use scenarios of AR-HUD and greatly improves the user experience.

[0205] If the data processing device 80 described above can be a chip or a chip system, please refer to the schematic diagram of the chip structure shown in Figure 10.

[0206] As shown in Figure 10, chip 100 includes processor 1001 and interface 1002. The number of processors 1001 can be one or more, and the number of interfaces 1002 can be multiple. It should be noted that the functions of processor 1001 and interface 1002 can be implemented through hardware design, software design, or a combination of both; no restrictions are placed here.

[0207] Optionally, the chip 100 may also include a memory 1003 for storing necessary program instructions and data.

[0208] In this application, processor 1001 can be used to call the implementation program of the data processing method provided in one or more embodiments of this application in a data processing device from memory 1003, and execute the instructions included in the program. Interface 1002 can be used to output the execution result of processor 1001. In this application, interface 1002 can be specifically used to output various messages or information of processor 1001.

[0209] For the data processing methods provided by one or more embodiments of this application, please refer to the various embodiments shown in FIG7 above, which will not be repeated here.

[0210] The processor in this application embodiment can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0211] The memory in this application embodiment is used to provide storage space, in which data such as operating system and computer programs can be stored. The memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0212] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer-readable storage medium storing a computer program. When the computer program is run on one or more processors, it can implement the method shown in FIG7.

[0213] According to the method provided in the embodiments of this application, the embodiments of this application also provide a computer program product, which includes a computer program. When the computer program runs on a processor, it can implement the method shown in FIG7.

[0214] This application also provides a cockpit, which includes a display device and at least one data processing device 80, or electronic device 90, or chip 100.

[0215] This application embodiment also provides a terminal, which includes at least one data processing device 80, or electronic device 90, or chip 100, or the aforementioned cockpit.

[0216] Optionally, the terminal can be a means of transportation, such as a car, truck, aircraft, drone, slow transport vehicle, spacecraft, or ship, or any other possible means of transportation used in any possible scenario. This application embodiment does not limit this.

[0217] Optionally, the terminal is used to implement the method shown in Figure 7 above.

[0218] This application also provides a processing apparatus, including a processor and an interface; the processor is used to execute the method in any of the above method embodiments.

[0219] It should be understood that the above-described processing device can be a chip. The units in the various device embodiments and the electronic devices in the method embodiments correspond completely, with corresponding modules or units executing corresponding steps. For example, the communication unit (transceiver) executes the receiving or sending steps in the method embodiments, while other steps besides sending and receiving can be executed by the processing unit (processor). The specific functions of each unit can be found in the corresponding method embodiments. There can be one or more processors.

[0220] It is understood that in the embodiments of this application, the electronic device may perform some or all of the steps in the embodiments of this application. These steps or operations are merely examples, and the embodiments of this application may also perform other operations or variations thereof. Furthermore, the steps may be performed in different orders as presented in the embodiments of this application, and it is not necessarily necessary to perform all the operations in the embodiments of this application.

[0221] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0222] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0223] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0224] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the contributing part, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0225] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, Applied to a first vehicle, the first vehicle including a display device and a detection device; the data processing method includes: Acquire first data and second data, wherein the first data is used by the display device to display augmented reality data of the environment in which the first vehicle is located, and the second data includes data of the environment in which the first vehicle is located detected by the detection device; Based on the first data and the second data, generate a first video and / or any frame or multiple frames of images from the first video.

2. The data processing method according to claim 1, characterized in that, The step of generating a first video and / or any one or more frames of images from the first video based on the first data and the second data includes: The first data and the second data corresponding to the same timestamp are fused to obtain the first video and / or any frame or multiple frames of the first video.

3. The data processing method according to claim 1 or 2, characterized in that, The first data is data in a first reference coordinate system, the origin of which is the observation point or the first calibration point inside the first vehicle; the second data is data in a second reference coordinate system, the origin of which is the position of the detection device or the second calibration point. The step of generating a first video and / or any one or more frames of images from the first video based on the first data and the second data includes: Convert the second data into data in the first reference coordinate system; The first data and the converted second data are fused to obtain the first video and / or any frame or multiple frames of the first video, wherein the first video is a video in the first reference coordinate system.

4. The data processing method according to any one of claims 1 to 3, characterized in that, The data processing method further includes: Receive a first instruction, the first instruction being used to instruct the target device to display the first video and / or any frame or multiple frames of the first video; In response to the first instruction, the first video and / or any frame or multiple frames of the first video are sent to the target device.

5. The data processing method according to claim 4, characterized in that, The target device includes at least one of the following: the central control display screen of the first vehicle, the passenger-side display screen of the first vehicle, the rear-seat display screen of the first vehicle, a mobile terminal, and the cloud.

6. The data processing method according to claim 4 or 5, characterized in that, The data processing method further includes: In response to the first instruction, the display device is instructed to display a first identifier in front of the observation point, the first identifier indicating that the first video and / or any frame or multiple frames of the first video are being synthesized.

7. The data processing method according to any one of claims 1 to 6, characterized in that, The data processing method further includes: Receive a second instruction, which is used to configure the display content and / or display style in the first data.

8. The data processing method according to claim 7, characterized in that, The displayed content in the first data includes at least one of the following: driving data, location, time, weather, navigation information, intelligent driving information, and the status information of the first vehicle; and / or, The display style in the first data includes at least one of the following: the font, size, color, transparency, brightness, and display style of the displayed content.

9. The data processing method according to any one of claims 1 to 8, characterized in that, The data processing method further includes: Receive a third instruction, the third instruction being used to instruct the export of the first video and / or any frame or multiple frames of the first video; In response to the third instruction, export the first video and / or any frame or multiple frames of the first video.

10. The data processing method according to any one of claims 1 to 9, characterized in that, The data processing method further includes: Receive a fourth instruction, the fourth instruction being used to instruct the first video and / or any frame or multiple frames of the first video to be sent to a target address; In response to the fourth instruction, the first video or its corresponding identifier is sent to the target address, or any frame or multiple frames of the first video and their corresponding identifier are sent to the target address.

11. A data processing apparatus, characterized in that, Includes units for performing the method as described in any one of claims 1 to 10.

12. A data processing apparatus, characterized in that, Includes a processor for performing the method as described in any one of claims 1 to 10.

13. A chip, characterized in that, It includes logic circuits and interfaces, wherein the logic circuits and the interfaces are coupled; The interface is used for inputting and / or outputting information, and the logic circuit is used for performing the method as described in any one of claims 1 to 10.

14. A cockpit, characterized in that, It includes a display device and at least one data processing device as claimed in claim 11, or the data processing device as claimed in claim 12, or the chip as claimed in claim 13.

15. A terminal, characterized in that, This includes the data processing apparatus as described in claim 11, or the data processing apparatus as described in claim 12, or the chip as described in claim 13, or the cockpit as described in claim 14.

16. The terminal according to claim 15, characterized in that, The terminal also includes a display device and a windshield; The light emitted from the display device passes through the windshield and forms a virtual image in front of the terminal.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed, performs the method as described in any one of claims 1 to 10.

18. A computer program product, characterized in that, The computer program product includes a computer program, which, when executed, performs the method as described in any one of claims 1 to 10.

Images