Information processing method, vehicle-mounted terminal and storage medium
By loading road scene images based on vehicle status and environmental information on the vehicle terminal, the problem of lack of interactiveness and personalized experience in the prior art is solved, and a more realistic and immersive gaming experience is achieved, and the user experience is improved.
Patent Information
- Application Number
- CN202410866730.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2024-11-12
AI Technical Summary
The existing in-vehicle terminals lack interactivity and personalized experience, and cannot meet the diverse needs of users.
By starting the game application on the vehicle terminal, the road scene image generated based on the vehicle status information and the environmental information sent by the vehicle peripheral devices is loaded, and the environmental information sensed by the sensor and the road condition information collected by the vehicle peripheral devices are used to match and feature perspective angles and features Identify, generate dynamic road scene images, and provide an immersive gaming experience through adjustments to image frame sequences and switching speeds.
It improves the user's riding experience, enhances the interactivity and entertainment of the on-board terminal, provides a more realistic and immersive visual gaming experience, and meets the diverse needs of users.
Smart Images

Figure CN118925221A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to an information processing method, a vehicle-mounted terminal and a storage medium. Background Art
[0002] In-vehicle terminals have become a standard feature of vehicles, and they can provide multiple functions such as navigation, entertainment, and communication. Existing in-vehicle terminals mostly focus on practicality, mainly used to provide navigation and entertainment playback functions, lacking interactivity and personalized experience, and unable to meet the diverse needs of users. Summary of the invention
[0003] Embodiments of the present application provide an information processing method, a vehicle-mounted terminal, and a storage medium to solve one or more problems existing in the prior art.
[0004] According to the first aspect of the present application, there is provided an information processing method, which is applied to an on-board terminal of a vehicle, comprising: starting a game application; loading a road scene image in the game application, wherein the road scene image is generated based on current vehicle status information and first environmental information received from a peripheral device of the vehicle, wherein the first environmental information represents current first road condition information of the peripheral device of the vehicle.
[0005] According to one embodiment of the present application, the method includes: receiving second environmental information sensed by a sensor of the vehicle, the second environmental information representing current second road condition information around the vehicle; and generating the road scene image based on the first environmental information and the second environmental information.
[0006] According to one embodiment of the present application, the first road condition information includes a real-time image from a first perspective in the direction of vehicle travel, and the second road condition information includes a real-time image from a second perspective in front of the direction of vehicle travel collected by the vehicle peripheral equipment; the method also includes: performing perspective matching on the first perspective real-time image and the second perspective real-time image to generate the road scene image.
[0007] According to one embodiment of the present application, the vehicle peripheral equipment is within a preset distance range in front of the vehicle's traveling direction, the second environmental information includes a second scene range in front of the vehicle, the first environmental information includes a first scene range in front of the vehicle, and the generated road scene image includes a third scene range in front of the vehicle, and the third scene range is larger than the second scene range.
[0008] According to an implementation manner of the present application, the perspective matching of the first perspective real-time image and the second perspective real-time image to generate the road scene image includes: performing feature recognition on the first perspective real-time image and the second perspective real-time image respectively to obtain first feature points corresponding to the first perspective real-time image and second feature points corresponding to the second perspective real-time image; determining a geometric transformation relationship between the first perspective and the second perspective based on the first feature points and the second feature points; adjusting the first perspective real-time image based on the geometric transformation relationship so that the first perspective real-time image matches the second perspective real-time image in perspective; and fusing the adjusted first perspective real-time image with the second perspective real-time image to obtain the road scene image.
[0009] According to an embodiment of the present application, the vehicle status information includes: vehicle speed, vehicle travel direction and vehicle position; the method also includes: serializing a plurality of first-perspective real-time images of the vehicle travel direction included in the first road condition information to obtain an image frame sequence corresponding to the plurality of first-perspective real-time images; based on the vehicle speed and the vehicle travel direction, determining a mapping relationship between a switching speed between frames in the image frame sequence and the vehicle speed; based on the current vehicle status information and the mapping relationship, determining a switching speed between frames in the image frame sequence to generate the road scene image.
[0010] According to one embodiment of the present application, the method also includes: displaying a game application control window; the game application control window is used to receive control instructions for the image frame sequence in the road scene image; obtaining the switching speed between frames in the image frame sequence in the road scene image; determining an adjustment ratio of the switching speed based on a speed control parameter included in the received control instruction; and adjusting the switching speed between frames in the image frame sequence in the road scene image based on the determined adjustment ratio.
[0011] According to one embodiment of the present application, the method also includes: performing image classification and recognition on a first-perspective real-time image of the vehicle's traveling direction included in the first road condition information, and identifying a target object and real-time road information included in the first road condition information; based on the target object, generating a virtual object corresponding to the target object; and performing image fusion on the image of the virtual object with a road image corresponding to the real-time road information to generate the road scene image.
[0012] According to the second aspect of the present application, a vehicle-mounted terminal is provided, including: a startup module for starting a game application; a loading module for loading a road scene image in the game application, wherein the road scene image is generated based on current vehicle status information and first environmental information received from a vehicle peripheral device, and the first environmental information represents current first road condition information of the vehicle peripheral device.
[0013] According to a third aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:
[0015] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.
[0016] Figure 1 A schematic diagram showing a processing flow of an information processing method provided in an embodiment of the present application is shown;
[0017] Figure 2 A schematic diagram showing a flow chart of an information processing method provided in the second embodiment of the present application is shown;
[0018] Figure 3 A schematic diagram showing a flow chart of an information processing method provided in the third embodiment of the present application is shown;
[0019] Figure 4 A schematic diagram showing a flow chart of an information processing method provided in a fourth embodiment of the present application is shown;
[0020] Figure 5 A schematic diagram showing a flow chart of an information processing method provided in the fifth embodiment of the present application is shown;
[0021] Figure 6 A schematic diagram showing a flow chart of an information processing method provided in a sixth embodiment of the present application is shown;
[0022] Figure 7 A flowchart of an information processing method provided in the seventh embodiment of the present application is shown;
[0023] Figure 8 An application scenario diagram of the information processing method provided by an embodiment of the present application is shown;
[0024] Fig. 9 Another application scenario diagram of the information processing method provided by the embodiment of the present application is shown;
[0025] Fig.10 An optional schematic diagram of a vehicle-mounted terminal provided in an embodiment of the present application is shown;
[0026] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0027] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0028] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0029] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0031] The processing flow of the information processing method provided in the embodiment of the present application is described. Figure 1 , Figure 1 is a processing flow diagram of the information processing method provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S102 are shown for explanation.
[0032] Step S101, start a game application.
[0033] In some embodiments, the game application may include an application installed in a vehicle-mounted terminal of the vehicle. The vehicle-mounted terminal may be installed inside the vehicle, and the vehicle-mounted terminal may include: an electronic device for providing navigation, multimedia playback, communication, and running applications inside the vehicle. The game application can be customized for the vehicle environment, and the game application can increase interactivity through the vehicle's motion sensors. The vehicle-mounted terminal of the vehicle can be installed inside the vehicle and operated by the rear passengers. The vehicle-mounted terminal of the vehicle can be installed inside the vehicle and operated by the front passenger.
[0034] Step S102, loading a road scene image in the game application, wherein the road scene image is generated based on current vehicle state information and first environmental information received from a peripheral device of the vehicle, and the first environmental information represents current first road condition information of the peripheral device of the vehicle.
[0035] In some embodiments, a road scene image can be generated based on the current vehicle state information and the first environmental information sent by the received vehicle peripheral device. The vehicle state information may include: vehicle speed, vehicle direction of travel, and vehicle position, etc. The first environmental information may characterize the current first road condition information of the vehicle peripheral device. The vehicle peripheral device may include other vehicles on the road, roadside units, and electronic devices carried by pedestrians, etc. The vehicle peripheral device may also include other devices, which are not limited in the embodiments of the present application. The vehicle may include a communication circuit for communication, and may receive the first environmental information sent by the vehicle peripheral device through the communication circuit. For example, the vehicle may communicate with other vehicles through V2V (Vehicle-to-Vehicle Communication) to receive the first environmental information sent by other vehicles. The vehicle may also communicate with the roadside unit through V2I (Vehicle-to-Infrastructure Communication) to receive the first environmental information sent by the roadside unit. The vehicle may also communicate with the electronic device carried by pedestrians through V2P (Vehicle-to-Pedestrian Communication) to receive the first environmental information sent by the electronic device carried by pedestrians. Loading the road scene image may include: displaying the road scene image in the game application. Loading the road scene image may also include: displaying a sequence of image frames in the road scene image in the game application.
[0036] The method of the embodiment of the present application, through steps S101-S102, starts the game application on the vehicle terminal and loads the road scene image based on the current vehicle status information and the first environmental information, so as to realize the real-time interaction between the vehicle driving situation and the game content. It not only improves the user's vehicle riding experience, but also makes full use of the real-time road condition information provided by the vehicle peripheral equipment, and enhances the interactivity and entertainment of the vehicle terminal. The user can enjoy a more immersive and interactive game experience while riding in the vehicle.
[0037] The specific processing flow of the information processing method provided in the embodiment of the present application is described. Figure 2 , Figure 2 It is a flowchart of the information processing method provided in the second embodiment of the present application.
[0038] like Figure 2 As shown, the information processing method applied to the vehicle-mounted terminal of the vehicle may also include:
[0039] Step S201, receiving second environment information sensed by a sensor of the vehicle.
[0040] In this embodiment, the second environmental information may characterize the current second road condition information around the vehicle. The second road condition information may include traffic conditions in front of, on the side of, and in other areas of the vehicle's direction of travel. The first environmental information characterizes the current first road condition information of the vehicle's peripheral equipment. The first road condition information may include a first-perspective real-time image of the vehicle's direction of travel. The first road condition information may be acquired through the vehicle's peripheral equipment. The vehicle's sensors may include: radars, laser radars, cameras, ultrasonic sensors, temperature sensors, humidity sensors, etc. installed on the vehicle.
[0041] Step S202: Generate a road scene image based on the first environment information and the second environment information.
[0042] In this embodiment, the vehicle can sense real-time data of the surrounding environment through an integrated sensor suite, including radar, laser radar, and camera, etc., to form second environmental information. The second environmental information is combined with the first environmental information from the vehicle's peripheral devices, and a road scene image is generated after data processing and image rendering. The road scene image can represent the real-time road conditions within the vehicle's vehicle travel perspective.
[0043] The method of the embodiment of the present application, through steps S201-S202, integrates the first environmental information of the vehicle's peripheral equipment and the second environmental information collected by the vehicle sensor through the vehicle terminal, realizes accurate mapping of real-time road conditions, and generates dynamic road scene images. The road scene image in the direction of vehicle travel is preloaded, and the road loading range is wider, making the game experience smoother. Real-time interaction between the vehicle's driving situation and the game content is realized, enhancing the interactivity and entertainment of the vehicle terminal.
[0044] The specific processing flow of the information processing method provided in the embodiment of the present application is described. Figure 3 , Figure 3 It is a flowchart of the information processing method provided in the third embodiment of the present application.
[0045] like Figure 3 As shown, the information processing method applied to the vehicle-mounted terminal of the vehicle may also include:
[0046] Step S301, start a game application.
[0047] The specific description of step S301 is the same as that of the above step S101 and will not be repeated here.
[0048] Step S302: Perform perspective matching on the first perspective real-time image and the second perspective real-time image to generate a road scene image.
[0049] In this embodiment, the first environmental information may characterize the current first road condition information of the vehicle peripheral device. The second environmental information may characterize the current second road condition information of the vehicle periphery. The first road condition information may include a first-perspective real-time image of the vehicle's traveling direction. The first-perspective real-time image may be the same as the vehicle's traveling perspective. The first-perspective real-time image may also be different from the vehicle's traveling perspective. The second road condition information may include a second-perspective real-time image of the vehicle's traveling direction collected by the vehicle peripheral device. The second-perspective real-time image may be the same as the vehicle's traveling perspective. The second-perspective real-time image may also be different from the vehicle's traveling perspective. The vehicle's traveling perspective may include: a first-person perspective of the driver sitting in the vehicle's driving seat. The first-perspective real-time image after perspective matching has the same perspective as the second-perspective real-time image. The perspectives of the first-perspective real-time image and the second-perspective real-time image after perspective matching may include the vehicle's traveling perspective.
[0050] In some embodiments, the vehicle peripheral device is within a preset distance range in front of the vehicle's traveling direction, the second environment information includes a second scene range in front of the vehicle, the first environment information includes a first scene range in front of the vehicle, and the generated road scene image includes a third scene range in front of the vehicle, and the third scene range is larger than the second scene range. The third scene range may include a range covered by the generated road scene image.
[0051] As an example, the vehicle peripheral device L1 is included within a preset distance range in front of the vehicle's direction of travel. The vehicle peripheral device L1 can collect first environmental information of a first scene range in front of the vehicle. The vehicle can collect second environmental information of a second scene range in front of the vehicle through an integrated sensor suite. The first perspective real-time image of the vehicle's direction of travel is matched with the second perspective real-time image in front of the vehicle's direction of travel to generate a road scene image. The generated road scene image includes a third scene range in front of the vehicle, and the third scene range can be larger than the second scene range.
[0052] Step S303: loading a road scene image in the game application.
[0053] The specific description of step S303 is the same as that of the above step S102 and will not be repeated here.
[0054] The method of the embodiment of the present application, through steps S301-S303, fuses the first-perspective real-time image of the vehicle's travel direction with the second-perspective real-time image collected by the vehicle's peripheral equipment to generate a seamless road scene image. A richer and more three-dimensional road condition display is achieved, enhancing the user's understanding of the vehicle's environment. Through this perspective fusion, passengers can obtain a more realistic and immersive gaming visual experience, enhancing the interactivity and entertainment of the vehicle terminal. Users can enjoy a more immersive and interactive gaming experience while riding in a vehicle.
[0055] The specific processing flow of the information processing method provided in the embodiment of the present application is described. Figure 4 , Figure 4 It is a flowchart of the information processing method provided in the fourth embodiment of the present application.
[0056] like Figure 4 As shown, the step S302 of performing perspective matching on the first perspective real-time image and the second perspective real-time image to generate a road scene image may include:
[0057] Step S401 : performing feature recognition on the first-viewing angle real-time image and the second-viewing angle real-time image respectively to obtain first feature points corresponding to the first-viewing angle real-time image and second feature points corresponding to the second-viewing angle real-time image.
[0058] In this embodiment, feature recognition may include: using an image processing algorithm to identify key visual feature points in the first-view real-time image and the second-view real-time image. The visual feature points may include: road signs, vehicle outlines, obstacles, etc. The first feature point may include: a feature point identified in the corresponding first-view real-time image. The second feature point may include: a feature point identified in the corresponding second-view real-time image.
[0059] Step S402: determining a geometric transformation relationship between the first perspective and the second perspective based on the first feature point and the second feature point.
[0060] In this embodiment, the geometric transformation relationship may include: calculating the geometric transformation required to transform the first perspective real-time image to the same perspective as the second perspective real-time image according to the corresponding relationship between the first feature point and the second feature point. The geometric transformation may include: rotation, scaling, translation and projection. The first perspective may be different from the second perspective.
[0061] Step S403: Based on the geometric transformation relationship, the first-perspective real-time image is adjusted so that the first-perspective real-time image matches the second-perspective real-time image in terms of perspective.
[0062] Step S404: Fusing the adjusted first-view real-time image with the second-view real-time image to obtain a road scene image.
[0063] In this embodiment, adjusting the first-perspective real-time image may include: applying the calculated geometric transformation relationship to the first-perspective real-time image, adjusting the first perspective of the first-perspective real-time image to match the second perspective of the second-perspective real-time image. Fusing the adjusted first-perspective real-time image with the second-perspective real-time image to obtain a road scene image. The fusion may include gradient, feature fusion, and image blending, etc. The adjusted first-perspective real-time image and the second-perspective real-time image may also be fused in other ways, which are not limited in the embodiments of the present application.
[0064] The method of the embodiment of the present application, through steps S401-S404, fuses the first-perspective real-time image of the vehicle's travel direction with the second-perspective real-time image collected by the vehicle's peripheral equipment to generate a seamless road scene image. A richer and more three-dimensional road condition display is achieved, which enhances the user's understanding of the vehicle's environment. Through this perspective fusion, users can obtain a more realistic and immersive gaming visual experience, which enhances the interactivity and entertainment of the vehicle terminal. Users can enjoy a more immersive and interactive gaming experience while riding in a vehicle.
[0065] The specific processing flow of the information processing method provided in the embodiment of the present application is described. Figure 5 , Figure 5 It is a flowchart of the information processing method provided in the fifth embodiment of the present application.
[0066] like Figure 5 As shown, the information processing method applied to the vehicle-mounted terminal of the vehicle may also include:
[0067] Step S501 : performing serialization processing on a plurality of first-viewpoint real-time images of a vehicle's traveling direction included in the first road condition information to obtain an image frame sequence corresponding to the plurality of first-viewpoint real-time images.
[0068] In this embodiment, a plurality of first-viewpoint real-time images received continuously are arranged in time order to form an image frame sequence. The image frame sequence may include: a set of image frames connected in series in time order. The image frame sequence may be used to represent road conditions that change over time.
[0069] Step S502 : determining a mapping relationship between a switching speed between frames in an image frame sequence and a vehicle speed based on the vehicle speed and the vehicle traveling direction.
[0070] Step S503 : Based on the current vehicle state information and the mapping relationship, a switching speed between frames in the image frame sequence is determined to generate a road scene image.
[0071] In this embodiment, the switching speed may include: the rate of switching from one frame to the next frame in the image frame sequence. The mapping relationship may include: the corresponding relationship between the vehicle speed and the switching speed of each frame in the image frame sequence. The mapping relationship can be used to: adjust the switching speed between each frame according to the vehicle speed. Generating the road scene image may include: as the vehicle moves, according to the vehicle speed and the vehicle travel direction of the current vehicle, determining the switching speed of the image frame sequence, updating the image frame sequence in real time, and displaying each image frame in the image frame sequence at the determined switching speed to generate a continuous road scene image.
[0072] As an example, the road test unit included in the vehicle peripheral equipment continuously collects images of the vehicle's direction of travel at fixed time intervals and sends them to the vehicle's on-board terminal in chronological order. The on-board terminal can form an image frame sequence with the continuously collected images of the vehicle's direction of travel, and each image frame in the image frame sequence corresponds to the road condition at a time point. According to the vehicle speed and the vehicle's direction of travel, the on-board terminal can determine the mapping relationship between the vehicle speed and the switching speed of each frame in the image frame sequence. For example, the faster the vehicle speed, the faster the switching speed between each frame in the image frame sequence. The slower the vehicle speed, the slower the switching speed between each frame in the image frame sequence. The on-board terminal determines the switching speed of the image frame sequence based on the vehicle's current vehicle speed, the vehicle's direction of travel, and the mapping relationship. The image frame sequence is updated in real time, and each image frame in the image frame sequence is displayed at the determined switching speed to generate a continuous road scene image.
[0073] The method of the embodiment of the present application, through steps S501-S503, can create a continuous image frame sequence by serializing and processing multiple first-person perspective real-time images of the vehicle's travel direction, and dynamically adjust the switching speed between frames according to the vehicle's speed and travel direction, thereby realizing the generation of real-time road scene images synchronized with the vehicle state. It can provide a smooth and responsive perspective switching gaming experience, provide users with a continuously updated, realistic road scene image, and enable users to obtain a more realistic and immersive gaming visual experience, thereby enhancing the interactivity and entertainment of the vehicle-mounted terminal. Users can enjoy a more immersive and interactive gaming experience while riding in a vehicle.
[0074] The specific processing flow of the information processing method provided in the embodiment of the present application is described. Figure 6 , Figure 6 It is a flowchart of the information processing method provided in the sixth embodiment of the present application.
[0075] like Figure 6 As shown, the information processing method applied to the vehicle-mounted terminal of the vehicle may also include:
[0076] Step S601, displaying the game application control window.
[0077] Step S602: obtaining a switching speed between frames in an image frame sequence in a road scene image.
[0078] Step S603: determining an adjustment ratio of the switching speed based on the speed control parameter included in the received control instruction.
[0079] Step S604: adjusting the switching speed between frames in the image frame sequence in the road scene image based on the determined adjustment ratio.
[0080] In this embodiment, the game application control window may include: virtual operation buttons. The virtual operation buttons may include: direction key buttons, acceleration buttons, deceleration buttons, and constant speed buttons, etc. The virtual operation buttons may also include other buttons, which are not limited in the embodiments of the present application. The game application control window may be used to receive control instructions for the image frame sequence in the road scene image. The control instructions may include: instructions input by the user through the game application control window. The control instructions may be used to change parameters or behaviors in the game application. For example, the control instructions may be used to adjust the display parameters of the road scene image loaded in the game application. The speed control parameters may include: parameters contained in the control instructions. The speed control parameters may be used to determine the target speed to which the switching speed is adjusted. The adjustment ratio may include: a ratio calculated based on the speed control parameters. The adjustment ratio may be used to indicate the increase ratio of the switching speed, or to indicate the decrease ratio of the switching speed.
[0081] As an example, a game application may include a game application for simulating driving, and a player may control the switching speed of an image frame sequence in a road scene image loaded by the game application through a game application control window to control a virtual vehicle in the game application. In the game application, a game application control window is displayed, and the game application control window may include a direction key button, an acceleration button, a deceleration button, and a constant speed button. The user may select at least one of the multiple buttons through the game application control window according to his own game strategy. For example, by selecting the acceleration button, the game application may generate a corresponding control instruction according to the duration for which the acceleration button is pressed. The speed control parameter included in the control instruction may match the duration for which the acceleration button is pressed. The game application may obtain the switching speed of the image frame sequence of the road scene image. Based on the speed control parameter included in the received control instruction, it is calculated that the adjustment ratio of the switching speed may be an increase of 10%. The game application adjusts the switching speed of the image frame sequence according to the adjustment ratio. For example, the current switching speed is 30 frames / second, and the switching speed is adjusted to 33 frames / second after an increase of 10%. With the adjustment of the switching speed, the image frame sequence of the road scene image switches at a speed of 33 frames / second. The adjusted speed can be fed back to the user in the form of the speed of the virtual vehicle through the game application control window. The user can see the real-time changes in the speed of the virtual vehicle and can further adjust the speed of the virtual vehicle.
[0082] As an example, in a game application, a game application control window is displayed, and the game application control window may include a direction key button, an acceleration button, a deceleration button, and a constant speed button. The user can select at least one of the multiple buttons through the game application control window according to his own game strategy. For example, the constant speed button and the direction key button are selected, and the game application can generate a corresponding steering control instruction according to the duration of the direction key button being pressed. The steering control parameter included in the steering control instruction can match the duration of the direction key button being pressed. The game application can obtain the switching speed of the image frame sequence of the road scene image. Based on the speed control parameter included in the received constant speed control instruction, the current switching speed is kept unchanged. The game application can obtain the switching speed of the image frame sequence of the road scene image. The game application determines the steering angle based on the steering control parameter included in the steering control instruction. According to the steering angle and the vehicle position, the image frame sequence is adjusted. Taking the steering angle as an example of deflection to the right, according to the steering angle and the vehicle position, the image frame of the right perspective is increased in the image frame sequence, and the image frame of the left perspective is reduced to simulate the effect of the virtual vehicle turning right. The adjusted image frame sequence in the road scene image is displayed, and the user can see a continuous view of the virtual vehicle turning right.
[0083] The method of the embodiment of the present application, through steps S601-S604, can realize the user's intuitive control of the image frame sequence in the road scene image by displaying the game application control window. By acquiring the switching speed of the image frame sequence in real time and dynamically adjusting the frame switching speed according to the speed control parameters input by the user through the control window, accurate adjustment of the smoothness and responsiveness of the road scene image is achieved. This not only improves the user's interactive experience, allowing users to customize road scene images according to personal preferences and needs, but also enhances the immersion of the game application and the authenticity of the driving simulation. Users can obtain a more realistic and immersive game visual experience, and enhance the interactivity and entertainment of the vehicle terminal. Users can enjoy a more immersive and interactive gaming experience while riding in a vehicle.
[0084] The specific processing flow of the information processing method provided in the embodiment of the present application is described. Figure 7 , Figure 7 It is a flowchart of the information processing method provided in the seventh embodiment of the present application.
[0085] like Figure 7 As shown, the information processing method applied to the vehicle-mounted terminal of the vehicle may also include:
[0086] Step S701: performing image classification and recognition on a first-perspective real-time image of a vehicle's traveling direction included in the first road condition information, and identifying a target object and real-time road information included in the first road condition information.
[0087] In this embodiment, image classification and recognition may include: using computer vision technology to analyze the first-person perspective real-time image to identify and classify the target object and real-time road information in the image. The target object may include: specific entities identified in the image, such as vehicles, pedestrians, and obstacles. The target object may also include other physical objects, which are not limited in the embodiments of the present application. Real-time road information may include: lane lines, road surface markings, traffic signals, and speed limits. Real-time road information may also include other real-time road information, which is not limited in the embodiments of the present application.
[0088] Step S702: Based on the target object, a virtual object corresponding to the target object is generated.
[0089] Step S703: Fusing the image of the virtual object with the road image corresponding to the real-time road information to generate a road scene image.
[0090] In this embodiment, the virtual object may include: a digital representation generated based on the identified target object. The image corresponding to the virtual object may be displayed in the road scene image. The virtual object may also include: a desensitized virtualized target object. Image fusion may include: combining the image of the virtual object with the road image corresponding to the real-time road information to generate a virtualized reality image combining the real-time road and the virtual object.
[0091] As an example, the vehicle-mounted terminal performs image classification and recognition on multiple first-perspective real-time images of the vehicle's travel direction in local storage, identifies the vehicle and real-time road information included in the first road condition information, desensitizes and virtualizes the identified vehicle, and generates a virtual vehicle corresponding to the vehicle. The environmental elements included in the real-time road information are replaced with environmental elements, and multiple environmental images for replacement are preset in the scene, which may include urban street scenes, underwater worlds, grasslands, mountains, and deserts. The weather elements included in the real-time road information are replaced with weather elements, and multiple weather images for replacement are preset in the scene, which may include thunder, rain, snow, and dust. Determine the road image corresponding to the real-time road information after the environmental element replacement and / or weather element replacement. The image of the virtual vehicle is fused with the road image corresponding to the replaced real-time road information to generate a road scene image.
[0092] The method of the embodiment of the present application, through steps S701-S703, uses image classification and recognition technology to accurately extract the target object and real-time road information from the first-person perspective real-time image of the vehicle's travel direction. Real-time analysis of road conditions is achieved, and the virtual object is seamlessly integrated into the real-time road image by generating a corresponding virtual object and performing image fusion with the real-time road image, providing users with a continuously updated, realistic road scene image, allowing users to obtain a more realistic and immersive gaming visual experience, and enhancing the interactivity and entertainment of the vehicle-mounted terminal. Users can enjoy a more immersive and interactive gaming experience while riding in a vehicle.
[0093] refer to Figure 8 , an application scenario diagram of the information processing method provided in an embodiment of the present application, which is applied to information processing of a vehicle's on-board terminal.
[0094] C1 can represent the main driving vehicle including the on-board terminal, C2-C7 can represent the vehicles traveling on the same road, L1 represents the vehicle peripheral device 1, and L2 represents the vehicle peripheral device 2. The vehicle peripheral devices may include smart street lights. L1 and L2 are within a preset distance range in front of the vehicle's direction of travel. The dotted line in the figure represents the first scene range corresponding to the image captured by L1. L1 and L2 can continuously capture images of the vehicle C1's direction of travel at fixed time intervals, and can be sent to the on-board terminal of vehicle C1 in chronological order. Vehicle C1 can also sense real-time data of the surrounding environment through an integrated sensor suite, including radar, lidar, and cameras, to form second environmental information. The second environmental information can characterize the current second road condition information around the vehicle.
[0095] The second environmental information is combined with the first environmental information from the vehicle peripheral device, and multiple real-time images of the vehicle's travel direction are classified and identified to identify the vehicles C2-C7 and real-time road information included in the road. The identified vehicles C2-C7 are desensitized and virtualized to generate a virtual vehicle corresponding to the vehicle. The image of the virtual vehicle is fused with the road image corresponding to the real-time road information to obtain multiple continuous fused images of the vehicle's travel perspective, and the multiple continuous fused images are combined into an image frame sequence, and each image frame in the image frame sequence corresponds to the road condition at a time point. According to the vehicle speed and vehicle travel direction of vehicle C1, the vehicle-mounted terminal can determine the mapping relationship between the vehicle speed and the switching speed of each frame in the image frame sequence. The vehicle-mounted terminal determines the switching speed of the image frame sequence according to the current vehicle speed, vehicle travel direction and mapping relationship of vehicle C1. The image frame sequence is updated in real time, and each image frame in the image frame sequence is displayed at the determined switching speed to generate a continuous road scene image.
[0096] refer to Fig. 9 , another application scenario diagram of the information processing method provided in an embodiment of the present application is applied to information processing of an on-board terminal of a vehicle.
[0097] The vehicle terminal may include a tablet device, a vehicle console and a computer device. The vehicle terminal may also include other electronic devices, which are not limited in the embodiments of the present application. The vehicle peripheral device a may include a smart street lamp. The vehicle peripheral device a can continuously collect first-person perspective real-time images of the vehicle's travel direction at fixed time intervals, and can send them to the vehicle's vehicle network module b in chronological order. The vehicle sensor c can collect vehicle status information. The vehicle status information may include: vehicle speed, vehicle travel direction and vehicle position, etc. Taking the vehicle device 1 as a vehicle console, the vehicle device 2 as a tablet device and the vehicle device 3 as a tablet device as an example, the vehicle network module b can send multiple first-person perspective real-time images of the vehicle's travel direction to the vehicle device 1, and the vehicle sensor c can send the vehicle status information to the vehicle device 1.
[0098] In some embodiments, the vehicle-mounted device 1 can perform image classification and recognition on multiple first-perspective real-time images of the vehicle's traveling direction received, identify the vehicle and real-time road information included in the first road condition information, desensitize and virtualize the identified vehicle, and generate a virtual vehicle corresponding to the vehicle. The image of the virtual vehicle is image-fused with the road image corresponding to the real-time road information to obtain a first fused image. The first fused image is serialized to obtain a corresponding image frame sequence, and the mapping relationship between the switching speed between each frame in the image frame sequence and the vehicle speed is determined based on the vehicle speed and the vehicle's traveling direction. The switching speed between each frame in the image frame sequence 1 is determined based on the current vehicle state information and the mapping relationship. The image frame sequence is adjusted according to the determined switching speed to generate a road scene image. The vehicle-mounted device 1 can load the road scene image in the game application. The vehicle-mounted device 1 can also send the generated road scene image to the vehicle-mounted device 2 and / or the vehicle-mounted device 3 respectively to load the road scene image in the game application of the vehicle-mounted device 2 and / or the vehicle-mounted device 3.
[0099] In some embodiments, the vehicle device 1 may also forward the received multiple first-view real-time images of the vehicle's traveling direction to the vehicle device 2 and the vehicle device 3. The vehicle sensor c may also send the vehicle status information to the vehicle device 2 and the vehicle device 3 respectively.
[0100] The vehicle-mounted device 2 can perform image classification and recognition on multiple first-perspective real-time images of the vehicle's travel direction in local storage, identify the vehicle and real-time road information included in the first road condition information, desensitize and virtualize the identified vehicle, and generate a virtual vehicle corresponding to the vehicle. The image of the virtual vehicle is fused with the road image corresponding to the real-time road information to obtain a first fused image. The first fused image is serialized to obtain a corresponding image frame sequence 1. Based on the vehicle speed and the vehicle's travel direction, the mapping relationship between the switching speed between each frame in the image frame sequence 1 and the vehicle speed is determined. Based on the current vehicle state information and the mapping relationship, the switching speed between each frame in the image frame sequence 1 is determined. And the image frame sequence 1 is adjusted according to the determined switching speed to obtain an adjusted image frame sequence 1.
[0101] The vehicle-mounted device 3 can perform image classification and recognition on multiple first-perspective real-time images of the vehicle's travel direction in the local storage, identify the vehicle and real-time road information included in the first road condition information, desensitize and virtualize the identified vehicle, and generate a virtual vehicle corresponding to the vehicle. The image of the virtual vehicle is fused with the road image corresponding to the real-time road information to obtain a second fused image. The second fused image is serialized to obtain a corresponding image frame sequence 2. Based on the vehicle speed and the vehicle's travel direction, the mapping relationship between the switching speed between each frame in the image frame sequence 2 and the vehicle speed is determined. Based on the current vehicle state information and the mapping relationship, the switching speed between each frame in the image frame sequence 2 is determined. And the image frame sequence 2 is adjusted according to the determined switching speed to obtain an adjusted image frame sequence 2.
[0102] The adjusted image frame sequence 1 and the adjusted image frame sequence 2 are synchronized between the vehicle-mounted device 2 and the vehicle-mounted device 3. After data synchronization, the vehicle-mounted device 2 and the vehicle-mounted device 3 store the adjusted image frame sequence 1 and the adjusted image frame sequence 2. The vehicle-mounted device 2 and the vehicle-mounted device 3 synthesize the adjusted image frame sequence 1 and the adjusted image frame sequence 2 respectively to obtain a road scene image. The road scene image is loaded in the respective game applications of the vehicle-mounted device 2 and the vehicle-mounted device 3 respectively to display the image frame sequence in the road scene image to the passenger A and the passenger B. The passenger A and the passenger B can interact with the corresponding game application through the vehicle-mounted device 2 and the vehicle-mounted device 3. For example, the corresponding game application control window is displayed in the vehicle-mounted device 2 and the vehicle-mounted device 3, and the passenger A can input a control instruction through the corresponding game application control window displayed in the vehicle-mounted device 2 to control the road scene image loaded in the game application. Passenger B can input a control instruction through the corresponding game application control window displayed in the vehicle-mounted device 3 to control the road scene image loaded in the game application.
[0103] The following is a description of the exemplary structure of the vehicle-mounted terminal 90 provided in the embodiment of the present application. In some embodiments, Fig.10 As shown, the vehicle terminal 90 may include: a startup module 901, which can be used to start a game application; a loading module 902, which can be used to load a road scene image in the game application, wherein the road scene image is generated based on the current vehicle state information and the first environmental information sent by the vehicle peripheral device, and the first environmental information represents the current first road condition information of the vehicle peripheral device.
[0104] In some embodiments, the loading module 902 can also be used to: receive second environmental information sensed by a sensor of the vehicle, the second environmental information representing current second road condition information around the vehicle; and generate a road scene image based on the first environmental information and the second environmental information.
[0105] In some embodiments, the first road condition information includes a first-perspective real-time image of the vehicle's traveling direction, and the second road condition information includes a second-perspective real-time image in front of the vehicle's traveling direction collected by the vehicle's peripheral equipment; the loading module 902 can also be used to: match the first-perspective real-time image with the second-perspective real-time image to generate a road scene image.
[0106] In some embodiments, the vehicle peripheral device is within a preset distance range in front of the vehicle's direction of travel, the second environmental information includes a second scene range in front of the vehicle, the first environmental information includes a first scene range in front of the vehicle, and the generated road scene image includes a third scene range in front of the vehicle, and the third scene range is larger than the second scene range.
[0107] In some embodiments, the loading module 902 can be used to: perform feature recognition on the first-perspective real-time image and the second-perspective real-time image respectively to obtain a first feature point corresponding to the first-perspective real-time image and a second feature point corresponding to the second-perspective real-time image; determine the geometric transformation relationship between the first perspective and the second perspective based on the first feature point and the second feature point; adjust the first-perspective real-time image based on the geometric transformation relationship so that the first-perspective real-time image matches the second-perspective real-time image in terms of perspective; fuse the adjusted first-perspective real-time image with the second-perspective real-time image to obtain a road scene image.
[0108] In some embodiments, the vehicle status information includes: vehicle speed, vehicle travel direction and vehicle position; the loading module 902 can be used to: serialize multiple first-perspective real-time images of the vehicle travel direction included in the first road condition information to obtain an image frame sequence corresponding to the multiple first-perspective real-time images; based on the vehicle speed and the vehicle travel direction, determine the mapping relationship between the switching speed between each frame in the image frame sequence and the vehicle speed; based on the current vehicle status information and the mapping relationship, determine the switching speed between each frame in the image frame sequence to generate a road scene image.
[0109] In some embodiments, the loading module 902 can be used to: display a game application control window; the game application control window is used to receive control instructions for an image frame sequence in a road scene image; obtain a switching speed between frames in an image frame sequence in a road scene image; determine an adjustment ratio of the switching speed based on a speed control parameter included in the received control instruction; and adjust the switching speed between frames in the image frame sequence in the road scene image based on the determined adjustment ratio.
[0110] In some embodiments, the loading module 902 can be used to: perform image classification and recognition on the first-perspective real-time image of the vehicle's traveling direction included in the first road condition information, and identify the target object and real-time road information included in the first road condition information; based on the target object, generate a virtual object corresponding to the target object; and fuse the image of the virtual object with the road image corresponding to the real-time road information to generate a road scene image.
[0111] It should be noted that the description of the vehicle-mounted terminal in the embodiment of the present application is similar to the description of the above-mentioned method embodiment, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figures 1 to 9 The present invention can be understood by referring to the description of any one of the accompanying drawings.
[0112] According to an embodiment of the present application, the present application also provides an electronic device and a non-transitory computer-readable storage medium.
[0113] Fig.11 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0114] like Fig.11 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0115] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0116] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as information processing methods. For example, in some embodiments, the information processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the information processing method in any other appropriate manner (e.g., by means of firmware).
[0117] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0118] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0119] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0120] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0121] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0122] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0124] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An information processing method, applied to a vehicle-mounted terminal of a vehicle, comprising: Launch a game application; A road scene image is loaded in the game application, wherein the road scene image is generated based on current vehicle state information and first environmental information received from a vehicle peripheral device, wherein the first environmental information represents current first road condition information of the vehicle peripheral device.
2. The method according to claim 1, comprising: Receiving second environmental information sensed by a sensor of the vehicle, where the second environmental information represents current second road condition information around the vehicle; The road scene image is generated based on the first environment information and the second environment information.
3. The method according to claim 2, wherein the first road condition information comprises a real-time image of a first perspective in the direction of travel of the vehicle, and the second road condition information comprises a real-time image of a second perspective in front of the direction of travel of the vehicle collected by the peripheral equipment of the vehicle; The method further comprises: The first perspective real-time image and the second perspective real-time image are matched in perspective to generate the road scene image.
4. According to the method according to claim 1, the vehicle peripheral equipment is within a preset distance range in front of the vehicle's traveling direction, the second environmental information includes a second scene range in front of the vehicle, the first environmental information includes a first scene range in front of the vehicle, and the generated road scene image includes a third scene range in front of the vehicle, and the third scene range is larger than the second scene range.
5. The method according to claim 3, wherein the performing perspective matching on the first perspective real-time image and the second perspective real-time image to generate the road scene image comprises: Performing feature recognition on the first-viewing angle real-time image and the second-viewing angle real-time image respectively to obtain a first feature point corresponding to the first-viewing angle real-time image and a second feature point corresponding to the second-viewing angle real-time image; Determining a geometric transformation relationship between the first perspective and the second perspective based on the first feature point and the second feature point; Based on the geometric transformation relationship, adjusting the first perspective real-time image so that the first perspective real-time image matches the second perspective real-time image in perspective; The adjusted first-view real-time image is fused with the second-view real-time image to obtain the road scene image.
6. The method according to claim 1, wherein the vehicle status information comprises: vehicle speed, vehicle direction of travel, and vehicle position; The method further comprises: Performing serialization processing on a plurality of first-viewpoint real-time images of the vehicle's traveling direction included in the first road condition information to obtain an image frame sequence corresponding to the plurality of first-viewpoint real-time images; Based on the vehicle speed and the vehicle traveling direction, determining a mapping relationship between a switching speed between frames in the image frame sequence and the vehicle speed; Based on the current vehicle state information and the mapping relationship, a switching speed between frames in the image frame sequence is determined to generate the road scene image.
7. The method according to claim 1, further comprising: Display the game application control window; The game application control window is used to receive control instructions for the image frame sequence in the road scene image; Obtaining a switching speed between frames in an image frame sequence in the road scene image; Determining an adjustment ratio of the switching speed based on a speed control parameter included in the received control instruction; Based on the determined adjustment ratio, a switching speed between frames in the image frame sequence in the road scene image is adjusted.
8. The method according to claim 1, further comprising: Performing image classification and recognition on the first-perspective real-time image of the vehicle's traveling direction included in the first road condition information, and identifying the target object and real-time road information included in the first road condition information; Based on the target object, generating a virtual object corresponding to the target object; The image of the virtual object is fused with the road image corresponding to the real-time road information to generate the road scene image.
9. A vehicle-mounted terminal, comprising: A startup module, used to start a game application; A loading module is used to load a road scene image in the game application, wherein the road scene image is generated based on current vehicle status information and first environmental information received from a vehicle peripheral device, wherein the first environmental information represents current first road condition information of the vehicle peripheral device.
10. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to any one of claims 1 to 8.
