Method, device, storage medium and equipment for generating display picture

Abstract

The present disclosure provides a display picture generation method and device, a storage medium and equipment, in a sentinel mode, in response to determining that there is a candidate object to be warned around the vehicle, based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system, the azimuth angle of the candidate object relative to the display screen is determined; based on the azimuth angle, the target gaze direction of the virtual interactive image is determined; based on the target gaze direction, the virtual interactive image is rendered to generate a display picture of the target object gazed by the virtual interactive image, thereby, the target object can perceive that it has been locked when seeing the virtual interactive image gazing at it, thereby, direct psychological deterrence is generated to the target object, directional warning function is realized, and warning effect is effectively improved.

Description

[0001] This disclosure claims priority to Singapore Patent Application No. SG10202602317Y, filed on July 3, 2026, with the Intellectual Property Office of Singapore (IPOS), the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to human-computer interaction technology, and in particular to a method, apparatus, storage medium, and device for generating display screens. Background Technology

[0003] As vehicle sentry mode gradually becomes a core safety feature in mainstream models, users have increasingly higher demands for the alerting effect of sentry mode on suspicious objects outside the vehicle. Currently, related technologies typically rely on audible and visual warnings, which are relatively ineffective. Summary of the Invention

[0004] This disclosure provides a method, apparatus, storage medium, and device for generating display screens to provide targeted warnings of suspicious objects, thereby enhancing the warning effect.

[0005] A first aspect of this disclosure provides a method for generating a display screen, comprising: in sentry mode, in response to determining that there is a candidate object to be warned around a vehicle, determining an azimuth angle of the candidate object relative to the display screen based on a first position of the candidate object in a first coordinate system and a second position of the display screen in the first coordinate system; determining a target gaze direction of a virtual interactive avatar based on the azimuth angle; rendering the virtual interactive avatar based on the target gaze direction to generate a display screen in which the virtual interactive avatar gazes at the target object in the target gaze direction.

[0006] A second aspect of this disclosure provides a display screen generation apparatus, comprising: a first processing module, in sentry mode, in response to determining that a candidate object to be warned exists around a vehicle, determining an azimuth angle of the candidate object relative to the display screen based on a first position of the candidate object in a first coordinate system and a second position of the display screen in the first coordinate system; a second processing module, configured to determine a target gaze direction of a virtual interactive avatar based on the azimuth angle; and a third processing module, configured to render the virtual interactive avatar based on the target gaze direction to generate a display screen in which the virtual interactive avatar gazes at the target object in the target gaze direction.

[0007] A third aspect of this disclosure is to provide a computer-readable storage medium storing a computer program that is executed by a processor to perform the method for generating a display screen according to any embodiment of this disclosure.

[0008] A fourth aspect of this disclosure provides an electronic device, the electronic device comprising: a memory for storing processor-executable instructions; and a processor for reading the executable instructions from the memory, the processor executing the executable instructions to implement the display screen generation method described in any embodiment of this disclosure.

[0009] A fifth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs a method for generating a display screen provided in any embodiment of this disclosure.

[0010] The present disclosure discloses a method, apparatus, storage medium, and device for generating display screens in the above embodiments. When a vehicle is in sentry mode, in response to determining that there is a candidate object to be warned around the vehicle, the azimuth angle of the candidate object relative to the display screen can be determined based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system. Then, based on the azimuth angle, the target gaze direction of the virtual interactive image can be determined. Subsequently, the virtual interactive image can be rendered based on the target gaze direction to generate a display screen of the virtual interactive image gazing at the target object in the target gaze direction. This allows the target object to perceive that it has been locked when it sees the virtual interactive image gazing at it, thereby directly deterring the target object psychologically, realizing a directional warning function, and effectively improving the warning effect. Attached Figure Description

[0011] Figure 1 This is an exemplary application scenario of an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a method for generating a display screen according to an exemplary embodiment of this disclosure; Figure 3 This is a flowchart illustrating a method for generating a display screen provided in another exemplary embodiment of this disclosure; Figure 4 This is a flowchart illustrating a method for generating a display screen according to yet another exemplary embodiment of this disclosure; Figure 5 This is a schematic diagram illustrating the principle of azimuth angle calculation for candidate objects provided in an exemplary embodiment of this disclosure; Figure 6 This is a flowchart illustrating a method for generating a display screen provided in yet another exemplary embodiment of this disclosure; Figure 7 This is a schematic diagram illustrating the principle of determining a target object from candidate objects, provided by an exemplary embodiment of this disclosure; Figure 8 This is a visual schematic diagram of a virtual interactive image provided in an exemplary embodiment of this disclosure; Figure 9This is a schematic diagram illustrating the display effect of a display screen provided in an exemplary embodiment of this disclosure; Figure 10 This is a flowchart illustrating a method for generating a display screen according to yet another exemplary embodiment of this disclosure; Figure 11 This is a flowchart illustrating a method for generating a display screen provided in yet another exemplary embodiment of this disclosure; Figure 12 This is a schematic diagram of the structure of a display screen generation apparatus provided in an exemplary embodiment of the present disclosure; Figure 13 This is a schematic diagram of the structure of a display screen generation apparatus provided in another exemplary embodiment of this disclosure; Figure 14 This is a schematic diagram of the structure of a display screen generation apparatus provided in yet another exemplary embodiment of the present disclosure; Figure 15 This is a structural diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0012] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0013] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0014] Application Overview In developing this disclosure, the inventors discovered that as vehicle sentry mode gradually becomes a core safety feature in mainstream models, users have increasingly higher demands for the alerting effect of sentry mode on suspicious objects outside the vehicle. In related technologies, indiscriminate alerts are typically provided through sound and light, resulting in poor alerting effectiveness.

[0015] Exemplary Overview Figure 1 This is an exemplary application scenario of an embodiment of this disclosure. For example... Figure 1 As shown, when the vehicle (or self-driving car) 11 is parked in a certain area, such as but not limited to parking lot 12, the sentry mode can be activated, and the display screen generated by the display screen generation method disclosed herein can be used to generate a display screen and display it on the display screen 13 of the self-driving car 11 to provide an external warning.

[0016] Specifically, in sentry mode, in response to the determination that there are candidate objects to be warned around vehicle 11, the azimuth angle of the candidate object relative to the display screen 13 can be determined based on the first position of the candidate object in the first coordinate system and the second position of the display screen 13 in the first coordinate system; then, based on the azimuth angle, the target gaze direction of the virtual interactive image can be determined; then, the virtual interactive image can be rendered based on the target gaze direction to generate a display screen of the virtual interactive image staring at the target object 14 in the target gaze direction, so that when the target object 14 sees the virtual interactive image staring at it, it can perceive that it has been locked, thereby directly deterring the target object 14, realizing the directional warning function, and effectively improving the warning effect.

[0017] Exemplary methods Figure 2 This is a flowchart illustrating a method for generating a display screen according to an exemplary embodiment of this disclosure. The method for generating a display screen provided in this embodiment can be applied to electronic devices, in-vehicle computing platforms, or system-on-chip (SoC) and microprogrammed control units (MCUs) within an in-vehicle computing platform, or to hardware and / or software within electronic devices or SoCs. Electronic devices include, but are not limited to, servers, terminal devices (e.g., in-vehicle terminals), and embodied intelligent agents; SoCs include, but are not limited to, SoCs on in-vehicle terminals and SoCs on embodied intelligent agents. Figure 2 As shown, the method of this disclosure may include the following steps 210-230: Step 210: In sentry mode, in response to determining that there is a candidate object to be warned around the vehicle, the azimuth angle of the candidate object relative to the display screen is determined based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system.

[0018] Sentry mode is a car safety feature. Its main purpose is to protect the vehicle from unauthorized attacks such as broken windows, scratches, and collisions by using algorithms to monitor the vehicle's surroundings and status in real time when the vehicle is unattended.

[0019] Optionally, when the vehicle's Sentry Mode function is enabled, it can automatically enter Sentry Mode working state after the vehicle is locked, or it can enter Sentry Mode working state in response to a Sentry Mode activation request from the user terminal. Enabling Sentry Mode function means switching it from a disabled state to an enabled state, preparing for entering Sentry Mode working state, but not yet actually entering it. The specific triggering method for entering Sentry Mode working state is not limited.

[0020] In Sentry Mode, the vehicle is in the Sentry Mode operating state. In Sentry Mode, based on the perception of the vehicle's surrounding environment, it can determine whether there are potential objects requiring alerts in the vicinity. Potential objects requiring alerts are those exhibiting suspicious actions such as colliding with the vehicle, pulling on the door, banging on the vehicle, or staring at the window. Potential objects include, but are not limited to, one or more of the following: pedestrians, other vehicles, and cyclists.

[0021] The first coordinate system is the unified coordinate system of the display screen and the candidate objects. The first coordinate system is, for example, but not limited to, the coordinate system of the preset plane of the vehicle coordinate system or the Bird's Eye View (BEV) coordinate system. The preset plane is, for example, but not limited to, the XOY plane of the vehicle coordinate system or a plane parallel to the XOY plane where the center point of the display screen is located. Here, X represents the lateral direction, such as a direction perpendicular to the vehicle's centerline towards the right of the vehicle; Y represents the longitudinal direction, such as a direction along the vehicle's centerline towards the rear of the vehicle.

[0022] The first position is the position of the candidate object in the first coordinate system. Optionally, the first position of the candidate object in the first coordinate system can be determined based on the perception results of the candidate object. The second position is the position of the display screen in the first coordinate system, which can be pre-calibrated.

[0023] The azimuth angle of the candidate object relative to the display screen refers to the angle between the vector pointing from the display screen to the candidate object and the projection line of the display screen normal in the first coordinate system, or the angle between the vector pointing from the display screen to the candidate object and a first preset direction of the first coordinate system. The first preset direction is the direction along the vehicle's centerline towards the rear of the vehicle.

[0024] Step 220: Determine the target gaze direction of the virtual interactive avatar based on the azimuth angle.

[0025] Among them, the virtual interactive avatar is a virtual avatar designed for Sentinel mode, simulating an image with the ability to gaze. For example, the virtual interactive avatar can be, but is not limited to, a virtual character image, a virtual eye image, etc.

[0026] The azimuth angle reflects the orientation of the candidate object relative to the display screen. The virtual interactive avatar is an image used to interact with the target object among the candidates; that is, the virtual interactive avatar stares at the target object to indicate that the target object has been locked by the vehicle's sentry mode. Therefore, the target staring direction of the virtual interactive avatar can be determined based on the azimuth angle of the candidate object.

[0027] Optionally, the target gaze direction is the direction from the display screen to the target object among the candidate objects.

[0028] Optionally, the target object can be determined from the candidate objects based on preset rules. The preset rules include, but are not limited to, randomly selecting the target object from the candidate objects.

[0029] Step 230: Render the virtual interactive image based on the target's gaze direction to generate a display screen of the target object in which the virtual interactive image is gazing at the target.

[0030] The specific visual representation of the virtual interactive avatar can be set according to actual needs. The gaze direction of the virtual interactive avatar can be represented, for example, but not limited to, a simulated light beam emanating from the virtual interactive avatar along the target's gaze direction. The simulated light beam can be, for example, but not limited to, a light beam of a preset color. The preset color can include, for example, but not limited to, one or more colors.

[0031] Optionally, the virtual interactive avatar can be rendered using the vehicle's rendering engine based on the target's gaze direction, generating a display screen showing the target object as the virtual interactive avatar gazes in the target's gaze direction.

[0032] Optionally, after generating the display screen, the display screen can also be displayed on a display screen, so that the display screen displays a virtual interactive image of the target object, so as to achieve the purpose of interaction between the virtual interactive image and the target object. The target object can see the virtual interactive image displayed on the display screen through the vehicle window, so that the target object can perceive that it has been locked by the vehicle's sentry mode, thereby creating a direct psychological deterrent to the target object.

[0033] Optionally, there can be one or more target gaze directions. When there are multiple target gaze directions, the virtual interactive avatar can be rendered based on each target gaze direction to generate multiple display screens of virtual interactive avatars gazing at target objects in different target gaze directions, so as to simultaneously warn and deter multiple suspicious target objects.

[0034] The display screen generation method provided in this embodiment, in sentry mode, in response to determining that there is a candidate object to be warned around the vehicle, can determine the azimuth angle of the candidate object relative to the display screen based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system; then, based on the azimuth angle, determine the target gaze direction of the virtual interactive image; then, the virtual interactive image can be rendered based on the target gaze direction to generate a display screen of the virtual interactive image gazing at the target object in the target gaze direction, so that when the target object sees the virtual interactive image gazing at it, it can perceive that it has been locked, thereby generating a direct psychological deterrent to the target object, realizing the directional warning function, and effectively improving the warning effect.

[0035] Figure 3This is a flowchart illustrating a method for generating a display screen according to another exemplary embodiment of this disclosure.

[0036] In some alternative embodiments, based on any of the above embodiments, such as Figure 3 As shown, the method of this embodiment may further include steps 310-330.

[0037] Step 310: Determine the perception result based on the sensor data collected by the sensors on the vehicle.

[0038] The perception result includes state information of one or more objects. These objects include, but are not limited to, pedestrians, cyclists, other vehicles, and other objects of any type or preset type around the vehicle. The object's state information includes, but is not limited to, one or more of the following: the object's position in the image, the object's distance from the vehicle, the position of the closest point on the object to the vehicle in the image, the object's speed as it approaches the vehicle, and the object's body posture.

[0039] Sensors include, but are not limited to, cameras, ultrasonic radar, millimeter-wave radar, and lidar. Sensor data includes, but is not limited to, image data acquired by cameras and radar data acquired by various radars. Cameras may include cameras with one or more viewpoints.

[0040] Optionally, the sensor data can be processed based on a pre-configured perception module on the vehicle to obtain the perception result. This disclosure does not limit the specific operations of the perception module in determining the perception result.

[0041] Step 320: Based on the perception results, determine the candidate objects.

[0042] The perception results reflect the state of each object relative to the vehicle. Based on these results, it can be determined whether each object poses a certain risk to the vehicle. When any object performs a risky action towards the vehicle, it can be designated as a candidate object. Optionally, conditions for determining candidate objects can be pre-set. Then, based on the perception results and these pre-set conditions, candidate objects are selected from the objects in the perception results. These conditions include, but are not limited to, the possibility of an object colliding with the vehicle, the object patting the vehicle, the object lingering near the vehicle for an extended period, or the object suddenly approaching the vehicle and staring at the window. When any object performs one or more of these risky actions, it can be designated as a candidate object.

[0043] Step 330: Based on the perspective transformation matrix and the first pixel position in the state information of the candidate object, determine the first position of the candidate object in the first coordinate system.

[0044] The first pixel position is the pixel position in the image of the point on the candidate object that is closest to the vehicle.

[0045] Optionally, the first pixel position can be obtained by the perception module. For example, through target detection and tracking, the pixel positions of the points closest to each perceived object and the vehicle in the image are obtained and used as the first pixel position. When any object is a candidate object, the first pixel position of the candidate object can be obtained from the state information of the candidate object.

[0046] The perspective transformation matrix is ​​the transformation matrix from the image coordinate system of a camera on a vehicle to the first coordinate system. Cameras with different viewing angles can have their own perspective transformation matrices. For example, the front-view camera, left-view camera, right-view camera, and rear-view camera each correspond to different perspective transformation matrices. The perspective transformation matrix can be pre-calibrated.

[0047] For example, taking a fisheye camera, the intrinsic parameters and distortion parameters of the fisheye camera can be determined through checkerboard calibration. Then, for the original image acquired by the fisheye camera, distortion correction processing is performed based on the intrinsic parameters and distortion parameters of the fisheye camera to obtain a distortion-corrected image. The first coordinate points corresponding to multiple preset reference points on the checkerboard on the distortion-corrected image are determined, as well as the second coordinate points corresponding to the multiple preset reference points on the desired BEV image. The perspective transformation matrix M is calculated based on the first and second coordinate points. The desired BEV image is a BEV image including each preset reference point, defined according to the calibration process, used to calculate the perspective transformation matrix. The specific method for determining the perspective transformation matrix is ​​not limited to the example described above.

[0048] Optionally, the first pixel position can be the distorted pixel position. Then, based on the first pixel position, a perspective transformation matrix can be used to transform the first pixel position to the first coordinate system, obtaining the first position of the candidate object. Alternatively, the first pixel position can be the undistorted pixel position. Then, based on the distortion parameters of the camera capturing the image, the first pixel position can be transformed to the distorted image coordinate system to obtain the second pixel position. Then, based on the second pixel position and the perspective transformation matrix, the second pixel position can be transformed back to the first coordinate system, obtaining the first position of the candidate object.

[0049] In the embodiments of this disclosure, based on the perception results determined by sensor data, candidate objects are identified, and suspicious objects with risks are filtered out. For each candidate object, based on the first pixel position closest to the vehicle and the perspective transformation matrix, the first position of the candidate object in the first coordinate system is determined, which can provide an accurate position reference for determining the azimuth angle of the candidate object.

[0050] In some optional embodiments, determining candidate objects based on the perception results in step 320 may include: Based on the state information of one or more objects in the perception results and the preset risk triggering conditions, determine the objects whose state information meets the risk triggering conditions from the one or more objects; in response to the existence of at least one object that meets the risk triggering conditions among the one or more objects, the object that meets the risk triggering conditions is taken as a candidate object.

[0051] Among them, the risk trigger condition is the condition for determining whether an object has triggered a risk; that is, the risk trigger condition is the condition for determining whether the object has performed a risky action. Risky actions include, but are not limited to, one or more of the following: colliding with a vehicle, hitting the vehicle, pulling on a car door, suddenly approaching a vehicle and staring at the window, or loitering near a vehicle for a duration exceeding a certain threshold. Specific risk trigger conditions can be set according to actual needs.

[0052] Optionally, based on the object's state information, the object's position, distance from the vehicle, orientation, speed of approach to the vehicle, line of sight, body posture, and other states can be determined. These states can then be matched against risk triggering conditions. Based on the matching results, it can be determined whether the object's state information meets the risk triggering conditions. If at least one object's state information meets the risk triggering conditions, then the object meeting the risk triggering conditions is considered a candidate object.

[0053] For example, when any object is detected to be less than a distance threshold, the duration for which the object is less than the distance threshold can be recorded. If the duration for which the object is less than the distance threshold exceeds the duration threshold, it indicates that the object has been lingering near the vehicle for a long time and may pose a risk. Therefore, the object can be considered as a candidate object.

[0054] For example, one can determine that an object is suddenly approaching a vehicle based on a sudden increase in the object's speed. Based on the object's eye position and line of sight, one can determine that the object is staring at the car window, indicating that the object is observing the situation inside the car. This behavior is suspicious and poses a certain risk, and the object can be considered a candidate.

[0055] For example, based on the distance between the object and the vehicle being 0 or close to 0, and considering that the closest point on the object to the vehicle is near the door handle, it can be determined that the object is pulling on the door; or that the closest point is near the window, it can be determined that the object is banging on the window. Alternatively, based on the vibration of the vehicle, door, or window, it can be determined that the object is banging on the vehicle, door, or window, etc.

[0056] In practical applications, the specific operations for determining whether an object's state information meets the risk triggering conditions are not limited to the examples above.

[0057] In the embodiments of this disclosure, based on the perceived state information of the objects, suspicious objects are identified as candidate objects through preset risk triggering conditions, while objects that are obviously without risk are excluded, which can effectively improve the accuracy and effectiveness of the candidate objects.

[0058] In some optional embodiments, the method of this disclosure may further include: In response to the existence of at least one object among the aforementioned objects that meets the risk triggering condition, the control screen switches from a first preset state to a first warning state; after generating the display screen, the display screen is displayed; or, in response to the existence of no object among the aforementioned objects that meets the risk triggering condition, the control screen remains in the first preset state.

[0059] The first preset state is, for example but not limited to, a normally dark state. That is, after entering sentry mode, if there are no suspicious objects around the vehicle, the display screen can be in a normally dark state to save energy. The first warning state is the state in which the warning mode interface is displayed. If at least one object meeting the risk triggering conditions exists around the vehicle, it indicates the presence of a suspicious object. The suspicious object control display screen then switches from the first preset state to the first warning state, preparing to display the warning image. After generating the virtual interactive information display screen for the target object, the image can be displayed on the screen to warn the target object.

[0060] If no object that meets the risk triggering conditions is detected, it means that there are no suspicious objects around the vehicle. The display screen can be kept in the first preset state to reduce power consumption and save energy.

[0061] In the embodiments of this disclosure, when it is determined that at least one object meeting the risk triggering conditions exists around the vehicle, the display screen is promptly switched from a first preset state to a first warning state. This allows the display screen to be promptly displayed after generating a virtual interactive information view of the target object to warn the target object. Maintaining the display screen in the first preset state when no risky object exists around the vehicle helps reduce power consumption and save energy.

[0062] Figure 4 This is a flowchart illustrating a method for generating a display screen according to yet another exemplary embodiment of this disclosure.

[0063] In some alternative embodiments, based on any of the above embodiments, the first coordinate system is a two-dimensional coordinate system of a preset plane of the vehicle coordinate system.

[0064] like Figure 4As shown, step 210, which determines the azimuth angle of the candidate object relative to the display screen based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system, may include steps 2110-2120.

[0065] Step 2110: Based on the first position and the second position, determine the first angle between the line segment between the first position and the second position and the first preset direction in the first coordinate system.

[0066] The first preset direction is the direction of the projection line of the normal of the display screen onto the preset plane.

[0067] The first included angle is the angle between the line segment between the first position and the second position and the first preset direction in the first coordinate system. Optionally, based on the first and second positions, a vector pointing from the first position to the second position can be determined, and the first included angle can be determined based on the direction vector of this vector and the first preset direction. Alternatively, based on the first and second positions, the straight line containing the first and second positions can be determined, and the first included angle can be determined based on the angle between this straight line and the straight line in the first preset direction. The specific method for calculating the first included angle is not limited.

[0068] In some optional embodiments, when the projection direction of the normal of the display screen onto the preset plane is the Y direction of the first coordinate system, the first preset direction is the Y direction. The first included angle can be calculated based on the first and second positions using the arctangent function.

[0069] Step 2120: Take the first included angle as the azimuth angle of the candidate object relative to the display screen.

[0070] After determining the first included angle, the first included angle can be used as the azimuth angle of the candidate object relative to the display screen.

[0071] In some optional examples, Figure 5 This is a schematic diagram illustrating the principle of azimuth angle calculation for a candidate object provided in an exemplary embodiment of this disclosure. For example... Figure 5 As shown, the first coordinate system is the XOY plane of the vehicle coordinate system, where the positive X direction is perpendicular to the vehicle's centerline to the right, and the positive Y direction is along the vehicle's centerline backward. The second position of the display screen 13 is the position T2(xt,yt) of the center point of the display screen 13 in the first coordinate system, and the first position of the candidate object in the first coordinate system is P2(x2,y2). The first preset direction is the Y direction, so the azimuth angle can be expressed as follows: Formula (1) in, This represents the arctangent function, and azimuth represents the azimuth angle, also known as the horizontal angle.

[0072] In the embodiments of this disclosure, based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system, the angle between the line segment between the two and the first preset direction is determined as the azimuth angle of the candidate object, thereby realizing the accurate calculation of the azimuth angle of the candidate object and providing an accurate and effective azimuth reference for determining the target gaze direction of the virtual interactive image.

[0073] Figure 6 This is a flowchart illustrating a method for generating a display screen provided in another exemplary embodiment of this disclosure.

[0074] In some alternative embodiments, based on any of the above embodiments, such as Figure 6 As shown, step 220, which determines the target gaze direction of the virtual interactive image based on the azimuth angle, may include steps 2210-2220.

[0075] Step 2210: Determine the target object from the candidate objects based on the azimuth angle and the preset angle range.

[0076] The preset angle range is an angle range pre-set based on the visible range of the display screen. The preset angle range can be less than or equal to the visible range of the display screen. The visible range of the display screen refers to the range that objects outside the vehicle can see on the display screen, or the range that the virtual interactive avatar displayed on the screen can simulate seeing. In other words, it refers to the assumed field of view that the virtual interactive avatar can see when looking outwards from the display screen.

[0077] For example, with the first preset direction as the 0° direction, the preset angle range is, for example, but not limited to, the range of [-S1°, S2°]. S1 and S2 are both positive numbers. S1 and S2 can be the same or different. For example, [-S1°, S2°] = [-75°, 75°], [-80°, 75°], etc.

[0078] Optionally, the target object can be determined from the candidate objects based on whether their azimuth angles are within a preset angle range. For example, candidate objects whose azimuth angles are within the preset angle range can be used as target objects. This is because candidate objects outside the preset angle range cannot be seen on the display screen, and therefore cannot be deterred by the virtual interactive image. Therefore, using one or more candidate objects within the preset angle range as target objects helps improve the effectiveness of the target object selection.

[0079] Step 2220: Determine the target gaze direction of the virtual interactive avatar based on the azimuth angle of the target object.

[0080] After identifying the target object, the target gaze direction of the virtual interactive avatar can be determined based on the azimuth angle of the target object. This allows the virtual interactive avatar to gaze at the target object along the target gaze direction, thereby interacting with the target object through the virtual interactive avatar to alert the target object.

[0081] Optionally, a mapping relationship between azimuth and gaze direction can be established in advance, and then based on the azimuth of the target object, the azimuth can be mapped to the target gaze direction of the virtual interactive image through the mapping relationship.

[0082] Optionally, the virtual interactive avatar can have a corresponding avatar model, which may be, for example, but not limited to, a 3D model in a preset 3D space. The preset 3D space may be, for example, but not limited to, a virtual 3D space used for rendering. The target's gaze direction indicates the orientation of the avatar model. The avatar model is observed through a virtual camera in the preset 3D space used for rendering, and the avatar model is mapped to the image coordinate system of the virtual camera. The image coordinate system of the virtual camera has a mapping relationship with the display interface of the display screen. The generated image from the virtual camera can be displayed as a screen image on the display interface, thus realizing the rendering and display of the virtual interactive avatar.

[0083] In some optional embodiments, step 2210, which determines the target object from the candidate objects based on the azimuth angle and a preset angle range, may include: If only one candidate object (referred to as the first candidate object) has an azimuth angle within a preset angle range, the first candidate object is selected as the target object; or, if multiple candidate objects (referred to as the second candidate objects) have azimuth angles within a preset angle range, the second candidate object with the longest candidate duration is selected as the target object based on the candidate duration of the multiple second candidate objects; the candidate duration is the duration during which the second candidate object belongs to the candidate object to be warned.

[0084] If there is only one first candidate object whose azimuth angle is within the preset angle range, it means that only the first candidate object can see the display screen. Therefore, the first candidate object can be used as the target object.

[0085] If multiple candidate objects exist within a preset angle range, the most critical target object can be further selected from these candidates. Candidate duration refers to the length of time a candidate object has been identified as a candidate. For example, after an object is initially identified as a candidate, its candidate duration can be maintained in real-time. Then, when determining the target object from multiple candidate objects, the candidate object with the longest candidate duration can be selected as the target object. A longer candidate duration indicates a higher degree of suspicion for the candidate object, thus requiring more timely alerts.

[0086] In some alternative embodiments, Figure 7 This is a schematic diagram illustrating the principle of determining a target object from candidate objects, provided by an exemplary embodiment of this disclosure. For example... Figure 7 As shown, in the BEV bird's-eye view coordinate system (i.e., the first coordinate system), with the Y direction as 0°, the 360° angle centered on the center point of the display screen or its projection point on the preset plane is represented as the range [-180°, 180°]. The preset angle range is [-75°, 75°]. Candidate objects include candidate object A, candidate object B, and candidate object C. Among them, the azimuth angle of candidate object A is greater than 75°, the azimuth angle of candidate object B is less than -75°, and the azimuth angle of candidate object C is within the range [-75°, 75°]. The virtual interactive image displayed on the screen can be seen by candidate object C but not by candidate objects A and B. Therefore, candidate object C is selected as the target object.

[0087] In the embodiments of this disclosure, candidate objects that can see the display screen are selected as target objects from suspicious candidate objects according to a preset angle range, thereby further improving the effectiveness of the target objects. Furthermore, when multiple second candidate objects exist within the preset angle range, the second candidate object with the longest candidate duration is selected as the target object based on its candidate duration, further improving the effectiveness of the target objects. This allows for more targeted warnings to higher-risk target objects, further enhancing the warning effect.

[0088] In some optional embodiments, step 220, which determines the target gaze direction of the virtual interactive avatar based on the azimuth angle, may further include: In response to the failure to determine the target object based on the azimuth angle and the preset angle range, the target gaze direction of the virtual interactive image is determined as the second preset direction.

[0089] The second preset direction can be set according to actual needs. For example, the second preset direction can be the gaze direction corresponding to a preset azimuth angle within the visible range of the display screen. The preset azimuth angle is, for example, but not limited to, 0° or other azimuth angles.

[0090] Optionally, the second preset direction may be, but is not limited to, directly in front of the display screen. That is, the virtual interactive avatar's model gazes at the virtual camera used for rendering, so that the rendered display image appears to be gazing directly in front of the display screen. In this case, since there are no suspicious target objects within the visible range of the display screen, the target object gazed at by the virtual interactive avatar is equivalent to any object in that gaze direction; for example, the target object gazed at by the virtual interactive avatar can be considered as the portion of the vehicle located in that gaze direction.

[0091] For example, Figure 8 This is a visual illustration of a virtual interactive image provided in an exemplary embodiment of this disclosure. For example... Figure 8 The diagram illustrates different gaze directions of the virtual interactive avatar. The left side of the screen shows the virtual interactive avatar gazing directly in front of it, while the right side shows it gazing to the right front of the screen. The dashed arrows in the diagram represent beams of light emitted by the virtual interactive avatar along the gaze direction. It should be noted that... Figure 8 This is merely an illustrative example of the gaze direction of a virtual interactive avatar and is not a limitation on the displayed image. In practical applications, the displayed image may also include background images, gaze beams of specific colors and effects, etc. Furthermore, the specific virtual interactive avatar is not limited to the virtual eyeball shown in the image.

[0092] In the embodiments of this disclosure, when there is no candidate object within the visible range of the display screen, but a candidate object exists outside the visible range, the target gaze direction of the virtual interactive image can be determined as the second preset direction, thereby generating a display screen showing the virtual interactive image gazing in the target gaze direction, which is convenient for vehicle users to view.

[0093] In some optional embodiments, based on any of the above embodiments, step 230, which renders the virtual interactive avatar based on the target gaze direction to generate a display screen of the target object in the target gaze direction of the virtual interactive avatar, may include: Based on the target's gaze direction and the image model of the virtual interactive image, a dynamic display is generated by rendering a gaze beam emanating from the virtual interactive image and flashing according to a preset pattern.

[0094] The virtual interactive avatar's model can be, but is not limited to, a pre-defined three-dimensional model in a three-dimensional space. This three-dimensional space can be, for example, but is not limited to, a virtual three-dimensional space. The avatar model is placed in the three-dimensional space according to a preset initial gaze direction. After determining the target gaze direction, the virtual interactive avatar is rendered based on this direction, generating a display image of the virtual interactive avatar gazing at the target object in the target gaze direction.

[0095] Optionally, the image model can be rotated according to the target gaze direction and the initial gaze direction of the image model, so that the gaze direction of the image model is rotated to the target gaze direction. Then, based on the intrinsic and extrinsic parameters of the virtual camera in 3D space, the image model is mapped to the image coordinate system of the virtual camera to generate a layer of virtual interactive image. Based on the virtual interactive image layer and the background layer, layer fusion is performed to obtain the display screen. In the process of generating the virtual interactive image layer, a gaze beam emanating from the virtual interactive image and flashing according to a preset rule can be generated according to the target gaze direction of the virtual interactive image, thereby generating a dynamic virtual interactive image layer. Based on the dynamic virtual interactive image layer and the background layer, a dynamic display screen is generated. Optionally, the background layer can be a static background layer or a dynamic background layer, which is not limited. The dynamic effect of the dynamic display screen is achieved through the dynamic updating of the display screen. For example, the gaze beam blinking according to a preset rule means that the display effect of the gaze beam in the display screen is dynamically updated according to a preset rule. For example, the brightness and / or color of the gaze beam changes according to a preset frame interval, such as changing the brightness and / or color every 3 frames to achieve the blinking effect. The specific operation of the dynamic update of the display screen will not be elaborated.

[0096] For example, Figure 9 This is a schematic diagram illustrating the display effect of a display screen provided in an exemplary embodiment of this disclosure. For example... Figure 9 As shown, the display screen includes a virtual interactive avatar and a background image, with a beam of light emanating from the eyes of the virtual interactive avatar. Optionally, the beam of light can be a beam of light with a certain brightness and / or color. Optionally, the beam of light can be a beam of any single color or a combination of multiple colors. For example, the beam of light can be, but is not limited to, a red beam, a yellow beam, a green beam, etc. The blinking effect can be an effect of brightness and color changes.

[0097] In the embodiments of this disclosure, based on the target's gaze direction and the image model of the virtual interactive image, a dynamic display image is rendered that emits a gaze beam from the virtual interactive image and the gaze beam flashes according to a preset pattern. This can further enhance the effect of the virtual interactive image gazing at the target object, making it easier for the target object to perceive that it has been locked by the vehicle's sentry mode, and further enhance the warning effect.

[0098] Figure 10 This is a flowchart illustrating a method for generating a display screen according to yet another exemplary embodiment of this disclosure.

[0099] In some alternative embodiments, based on any of the above embodiments, such as Figure 10 As shown, after generating the display screen, steps 240-250 may also be included.

[0100] Step 240: In response to the target object's azimuth angle changing from the first azimuth angle to the second azimuth angle, determine the rotation trajectory of the virtual interactive image based on the first azimuth angle and the second azimuth angle.

[0101] The first azimuth angle can be the azimuth angle of the target object calculated in the above embodiments. The second azimuth angle is an azimuth angle different from the first azimuth angle. After generating the display screen, the status information of the target object can be continuously monitored according to the steps of the above embodiments, and the azimuth angle of the target object can be determined. When it is determined that the azimuth angle of the target object changes from the first azimuth angle to the second azimuth angle, it indicates that the azimuth angle of the target object relative to the display screen has changed. In order for the virtual interactive image to follow and lock onto the target object, the rotation trajectory of the virtual interactive image is determined based on the first azimuth angle and the second azimuth angle.

[0102] Optionally, the rotation trajectory of the virtual interactive avatar can be determined based on the angular change of the gaze direction corresponding to the second azimuth angle relative to the gaze direction corresponding to the first azimuth angle. The rotation trajectory may include, but is not limited to, the number of frames in which the virtual interactive avatar's model rotates, and the angle and direction of rotation in each frame. In other words, the virtual interactive avatar rotates from the gaze direction corresponding to the first azimuth angle to the gaze direction corresponding to the second azimuth angle through one or more frames of rotation.

[0103] Step 250: Render the virtual interactive image based on the rotation trajectory to generate a dynamic display screen in which the gaze direction of the virtual interactive image changes with the rotation trajectory.

[0104] The rotation trajectory includes the target number of frames the virtual interactive avatar needs to rotate, as well as the angle and direction of rotation for each frame. The target number of frames can be one or more. Each rotation corresponds to an intermediate gaze direction, generating an intermediate display screen. For example, moving from the gaze direction corresponding to the first azimuth angle to the gaze direction corresponding to the second azimuth angle can be achieved through n rotations, each with a smaller angle. Starting from the gaze direction corresponding to the first azimuth angle, rotating once yields an updated gaze direction. Based on this updated gaze direction, an intermediate display screen is rendered and displayed. Simultaneously, during the display process, the gaze direction is updated, and a new intermediate display screen is rendered and displayed. This process continues until the gaze direction corresponding to the second azimuth angle is reached. Then, rendering can continue based on the gaze direction of the target object or the next target object.

[0105] Optionally, when the azimuth angle of the target object changes from the first azimuth angle to the second azimuth angle, it can be further determined whether the second azimuth angle is within a preset angle range. If the second azimuth angle is within the preset angle range, the rotation trajectory of the virtual interactive image is determined based on the first and second azimuth angles. The virtual interactive image is then rendered based on the rotation trajectory to generate a dynamic display screen showing that the gaze direction of the virtual interactive image changes with the rotation trajectory. If the second azimuth angle is not within the preset angle range, the target object can be re-determined according to the above-described process for determining the target object. Alternatively, the target object can be monitored for a preset duration to determine if it has left the visible range of the display screen, and then the target object can be re-determined.

[0106] Optionally, when the azimuth angle of the target object changes from the first azimuth angle to the second azimuth angle, it can be further determined whether the target object is still a suspicious candidate object, that is, whether the target object still has any risky actions. If it is determined that the target object does not have any risky actions, or the target object has not had any risky actions for a preset period of time, the target object can be re-determined. Alternatively, if the target object is still a suspicious candidate object and is still within the preset angle range, the rotation trajectory of the virtual interactive image is determined based on the first and second azimuth angles, and the virtual interactive image is rendered based on the rotation trajectory to generate a dynamic display screen in which the gaze direction of the virtual interactive image changes with the rotation trajectory.

[0107] In the embodiments of this disclosure, when the azimuth angle of the target object changes, the rotation trajectory of the virtual interactive image is determined based on the change in the azimuth angle of the target object. Then, the virtual interactive image is rendered based on the rotation trajectory to generate a dynamic display screen in which the gaze direction of the virtual interactive image follows the change in the rotation trajectory. The dynamic display screen is then displayed on the display screen to present the effect of the virtual interactive image's gaze direction following and locking onto the target object, further enhancing the deterrent effect on the target object and thus further enhancing the warning effect.

[0108] In some optional embodiments, display images of different azimuth angles can be pre-rendered and cached for different azimuth angles. After the target object is determined, during the rendering process, the display image corresponding to the azimuth angle of the target object can be directly obtained from the cached display images and displayed, or the display image that changes with the azimuth angle can be displayed according to the change of the azimuth angle of the target object at a preset frame rate to obtain a dynamic display image effect, thereby reducing display latency and improving display efficiency.

[0109] Figure 11 This is a flowchart illustrating a method for generating a display screen provided in another exemplary embodiment of this disclosure.

[0110] In some alternative embodiments, based on any of the above embodiments, such as Figure 11 As shown, after generating the display screen, the method of this embodiment may further include steps 410-420.

[0111] Step 410: Using the target object as the following object, control the gaze direction of the virtual interactive image on the display screen to rotate according to the change of the target object's azimuth angle.

[0112] Here, "following object" refers to the virtual interactive avatar's gaze direction following the locked object. Optionally, the virtual interactive avatar's image model can be rotated, and the screen rendering can be performed based on the rotated image model to achieve the effect that the virtual interactive avatar's gaze direction in the displayed screen rotates according to the change of the target object's azimuth angle. The specific rotation principle can be found in the aforementioned embodiments.

[0113] Optionally, after identifying the target object, the target object can be followed for at least a first duration, which may be, for example, but is not limited to, 2 seconds or other durations. If the azimuth angle of the target object remains within a preset angle range and the target object remains a suspected candidate object, the gaze direction of the virtual interactive image on the display screen is controlled to rotate in accordance with the change in the azimuth angle of the target object.

[0114] Step 420: In response to the target object changing from a following object to a non-following object within a first duration, the gaze direction of the virtual interactive image in the display screen is maintained as the first gaze direction of the target object in the last frame before it becomes a non-following object.

[0115] Among them, non-following objects refer to objects that the virtual interactive avatar does not need to follow or lock temporarily.

[0116] Optionally, if the target object leaves the preset angle range or becomes a non-suspect object within the first time period, it can be determined that the target object has changed from a following object to a non-following object within the first time period. The following action of the virtual interactive avatar can be paused, maintaining the gaze direction of the virtual interactive avatar on the displayed screen as the first gaze direction of the target object in the last frame before it became a non-following object. For example, maintaining the gaze direction in the last frame before the target object left the preset angle range or became a non-suspecting object. The tracking of the target object's state information can continue to determine whether it is necessary to end the following of the target object, re-identify the target object, or, if the target object does not exist, end the following state of the sentry mode.

[0117] In the embodiments of this disclosure, after the target object is determined, the target object is used as the following object, and the gaze direction of the virtual interactive image in the display screen is controlled to rotate to follow the change of the azimuth angle of the target object, so as to realize the following and locking of the target object, which helps the target object to know that it has been locked, and further enhances the warning effect.

[0118] In some alternative embodiments, such as Figure 11 As shown, the method in this embodiment may further include steps 430-460.

[0119] Step 430: In response to the target object becoming a following object again within a second time period after becoming a non-following object, determine the current gaze direction of the virtual interactive image based on the current azimuth angle of the target object.

[0120] The second duration can be set according to actual needs. The second duration is, for example, but not limited to, 2 seconds or other durations.

[0121] If the target object becomes a following object again within the second time period after becoming a non-following object, for example, if the target object performs a risky action and the target object's azimuth angle re-enters the preset angle range, then the following warning for the target object will continue, and the current gaze direction of the virtual interactive image will be determined based on the target object's current azimuth angle.

[0122] Step 440: Based on the first gaze direction and the current gaze direction, generate a dynamic display screen showing the virtual interactive image rotating from the first gaze direction to the current gaze direction.

[0123] The first gaze direction is the gaze direction of the last frame before the target object becomes a non-following object. Since the target object becomes a non-following object for a period of time, its azimuth angle changes, and the current gaze direction may be different from the first gaze direction. Based on the first gaze direction and the current gaze direction, a dynamic display screen can be generated to show the virtual interactive image rotating from the first gaze direction to the current gaze direction, ensuring that the movement of the virtual interactive image is natural and non-linear.

[0124] Optionally, the virtual interactive image can be rendered based on the rotation trajectory from the first gaze direction to the current gaze direction, generating a dynamic display screen of the virtual interactive image rotating from the first gaze direction to the current gaze direction.

[0125] Step 450: Display the dynamic display screen.

[0126] The dynamic display screen includes multiple frames, which are displayed sequentially on the display screen to achieve the effect of dynamic display.

[0127] Optionally, each time a display frame is generated, the display frame can be displayed on the display screen, and the display screen can be dynamically refreshed to achieve the effect of dynamic display.

[0128] Step 460: In response to the target object not becoming a following object within a second time period after becoming a non-following object, the display screen is updated to a display screen where the gaze direction of the virtual interactive image is the target direction.

[0129] The target direction is the second preset direction or the second gaze direction corresponding to the azimuth angle of the next target object.

[0130] Optionally, if the target object does not become a following object within a second time period after becoming a non-following object, it indicates that the target object has been removed from the risk or has left the visible range of the display screen. Following the above-described process for determining the target object, the target object can be re-determined. If a next target object can be determined, the second gaze direction corresponding to the azimuth angle of the next target object is taken as the target direction. Based on the target direction, a display screen is generated showing the virtual interactive image gazing at the target object in that direction, thus updating the display screen. This update allows the virtual interactive information to gaze at the new target object, thereby alerting the new target object.

[0131] If the next target object cannot be identified, and there are still suspicious candidate objects, the second preset direction will be used as the target direction, and the display screen will be updated to show the direction of the virtual interactive image's gaze as the target direction.

[0132] In the embodiments of this disclosure, if the target object becomes a following object again within a second time period after becoming a non-following object, it can continue to follow the target object. Based on the current azimuth angle of the target object, the current gaze direction of the virtual interactive image is determined. Then, based on the first gaze direction and the current gaze direction, a dynamic display screen is generated showing the virtual interactive image rotating from the first gaze direction to the current gaze direction. This helps to improve the motion continuity of the virtual interactive image, ensures that the virtual interactive image moves naturally and with non-linear easing, thereby improving the visual effect of the display screen.

[0133] In some optional embodiments, based on any of the above embodiments, the method of this disclosure may further include: Based on the azimuth angle, generate a warning voice message for the candidate object; play the warning voice message through the vehicle's audio playback device.

[0134] The specific warning voice message for potential targets includes the target's location information. This directional warning voice message, played via a sound player, allows the target to immediately recognize that they have been locked by the vehicle's sentry mode, further enhancing the warning effect. It also compensates for the limitation of virtual interactive images displayed on the screen being invisible to suspicious objects outside the visible range of the physical screen, thus increasing the warning range. Furthermore, the directional nature of the warning voice message helps prevent or reduce the possibility of non-suspicious objects mistakenly believing they have been warned.

[0135] For example, the content of the warning voice may include, but is not limited to, "The man in front on the left, please move away from the vehicle" or "The lady on the right, please do not tap on the window." The warning can be tailored to specific individuals by combining object attributes such as gender and age with the azimuth angle. The object attributes such as gender and age can be implemented using appropriate detection or recognition models, and this embodiment does not impose limitations.

[0136] In some optional embodiments, the azimuth angle of the candidate object and the vehicle's lights can be combined to provide light warnings for the candidate object.

[0137] In some optional embodiments, the rotation angle range of the virtual interactive avatar can be limited, and the rotation angle range can be less than or equal to a preset angle range. For example, the preset angle range is [-75°, 75°], and the rotation angle range of the virtual interactive avatar is [-60°, 60°]. The specific rotation angle range can be set according to actual needs, and this disclosure does not limit it.

[0138] In some optional embodiments, the Sentinel Mode operating state may include three states: sleep state, follow state, and default state. When there are no suspicious candidate objects around the vehicle, the Sentinel Mode operating state is in sleep state. In sleep state, no display screen needs to be rendered, and the display screen is turned off, i.e., the display screen is in a normally dark state.

[0139] Optionally, in dormant mode, the rendering engine can prepare layer resources for the alert interface. Layer resources include, but are not limited to, the image model and textures of the virtual interactive avatar, alert background images, etc.

[0140] When a candidate object is detected around the vehicle, meeting the risk trigger conditions, the control display screen switches to the warning interface, illuminates, and displays the warning message. Depending on whether a target object is within the visible range of the display screen among the candidate objects, the system determines whether to enter the default state or the follow state.

[0141] If the target object is not found among the candidate objects, the system enters the default state. In the default state, the target gaze direction of the virtual interactive avatar is the second preset direction. Based on the second preset direction, a display screen is rendered showing the virtual interactive avatar gazing in that direction, or a display screen showing a beam of light emanating from the virtual interactive avatar along the second preset direction, or a dynamic display screen showing a beam of light emanating from the virtual interactive avatar along the second preset direction and flashing according to a preset pattern. The display screen is then shown.

[0142] If the target object is among the candidate objects, the system enters the follow state. Following the follow process described in the above embodiment, a dynamic display screen is generated in which the gaze direction of the virtual interactive image changes with the azimuth angle of the target object, thereby achieving the effect of tracking and locking onto the target object. The follow process includes, but is not limited to, steps 410 to 460 described above, and will not be elaborated further. In the default state or follow state, in response to the fact that all candidate objects have become non-candidate objects, i.e., there are no candidate objects around the vehicle, the system switches to the sleep state to save energy.

[0143] Here, "dormant state" can refer to the first preset state mentioned above, while "default state" and "following state" are both alert states. For example, the default state can be the first alert state mentioned above, and the following state can be the second alert state. The specific state settings are not limited.

[0144] Optionally, in the follow state, the beam of light emitted by the virtual interactive image can also flash according to a preset pattern to further enhance the mental effect.

[0145] In some optional embodiments, the azimuth angle of the candidate object or target object can be updated according to a preset period to provide continuous azimuth angle input for the generation of subsequent display screens. The preset period can be set according to actual needs. Optionally, the preset period is, for example, but not limited to, a period within 100ms to 300ms.

[0146] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in this technical solution comply with relevant laws and regulations and do not violate public order and good morals. Furthermore, the collection and use of user personal information in this technical solution does not involve the illegal collection or use of user personal information.

[0147] The methods of any embodiment of this disclosure can be implemented alone or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0148] The display screen generation method provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices, servers, hardware and software in terminal devices, hardware and software in servers, etc. Alternatively, the display screen generation method provided in this disclosure can be executed by a processor, such as by a processor calling corresponding instructions stored in memory to execute any of the display screen generation methods mentioned in this disclosure. Alternatively, the display screen generation method provided in this disclosure can also be executed by the corresponding modules or units included in the exemplary apparatus described below, which are run by a processor.

[0149] Exemplary device Figure 12This is a schematic diagram of the structure of a display screen generation apparatus provided in an exemplary embodiment of this disclosure. The display screen generation apparatus of this disclosure can be used to implement the display screen generation method provided in any of the above embodiments of this disclosure. Figure 12 The apparatus shown may include: a first processing module 51, a second processing module 52 and a third processing module 53.

[0150] The first processing module 51 is used in sentry mode to determine the azimuth angle of the candidate object relative to the display screen based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system, in response to determining that there is a candidate object to be warned around the vehicle.

[0151] The second processing module 52 is used to determine the target gaze direction of the virtual interactive image based on the azimuth angle; The third processing module 53 is used to render the virtual interactive image based on the target's gaze direction, and generate a display screen of the target object in which the virtual interactive image gazes at the target.

[0152] Figure 13 This is a schematic diagram of the structure of a display screen generation apparatus provided in another exemplary embodiment of this disclosure.

[0153] In some alternative embodiments, based on any of the above embodiments, such as Figure 13 As shown, the apparatus of this embodiment may further include: a fourth processing module 54, a fifth processing module 55, and a sixth processing module 56.

[0154] The fourth processing module 54 is used to determine the perception result based on the sensor data collected by the sensors on the vehicle.

[0155] The perception results include the state information of one or more objects.

[0156] The fifth processing module 55 is used to determine candidate objects based on the perception results.

[0157] The sixth processing module 56 is used to determine the first position of the candidate object in the first coordinate system based on the perspective transformation matrix and the first pixel position in the state information of the candidate object. The first pixel position is the pixel position corresponding to the point on the candidate object that is closest to the vehicle.

[0158] In some optional embodiments, the fifth processing module 55 is specifically used for: Based on the state information of one or more objects in the perception results and the preset risk triggering conditions, determine the objects whose state information meets the risk triggering conditions from one or more objects; in response to the existence of at least one object that meets the risk triggering conditions among one or more objects, the object that meets the risk triggering conditions is taken as a candidate object.

[0159] In some alternative embodiments, such as Figure 13 As shown, the apparatus in this embodiment may further include: The first control module 57 is configured to control the display screen to switch from a first preset state to a first warning state in response to the existence of at least one object among one or more objects that meets the risk triggering condition; after generating the display screen, the display screen will be displayed on the display screen; or, in response to the absence of an object among one or more objects that meets the risk triggering condition, the display screen will be controlled to remain in the first preset state.

[0160] In some optional embodiments, based on any of the above embodiments, the first coordinate system is a two-dimensional coordinate system on a preset plane of the vehicle coordinate system. The first processing module 51 is specifically used for: Based on the first position and the second position, a first angle is determined between the line segment between the first position and the second position and a first preset direction in the first coordinate system. The first preset direction is the direction of the projection line of the normal to the display screen onto a preset plane. This first angle is used as the azimuth angle of the candidate object relative to the display screen.

[0161] Figure 14 This is a schematic diagram of the structure of a display screen generation apparatus provided in another exemplary embodiment of the present disclosure.

[0162] In some alternative embodiments, based on any of the above embodiments, such as Figure 14 As shown, the second processing module 52 may include: a first processing unit 521 and a second processing unit 522.

[0163] The first processing unit 521 is used to determine the target object from the candidate objects based on the azimuth angle and a preset angle range.

[0164] The second processing unit 522 is used to determine the target gaze direction of the virtual interactive image based on the azimuth angle of the target object.

[0165] In some optional embodiments, the first processing unit 521 is specifically used for: If there is only one first candidate object whose azimuth angle is within the preset angle range, the first candidate object is selected as the target object; or, if there are multiple second candidate objects whose azimuth angles are within the preset angle range, the second candidate object with the longest candidate duration is selected as the target object based on the candidate duration of the multiple second candidate objects; the candidate duration is the duration during which the second candidate object belongs to the candidate object to be warned.

[0166] In some optional embodiments, the second processing unit 522 may also be used for: In response to the failure to determine the target object based on the azimuth angle and the preset angle range, the target gaze direction of the virtual interactive image is determined as the second preset direction.

[0167] In some optional embodiments, based on any of the above embodiments, the third processing module 53 is specifically used for: Based on the target's gaze direction and the image model of the virtual interactive image, a dynamic display is generated by rendering a gaze beam emanating from the virtual interactive image and flashing according to a preset pattern.

[0168] In some optional embodiments, based on any of the above embodiments, the third processing module 53 may also be used for: In response to the target object's azimuth angle changing from a first azimuth angle to a second azimuth angle, the rotation trajectory of the virtual interactive avatar is determined based on the first and second azimuth angles. The virtual interactive avatar is then rendered based on this rotation trajectory, generating a dynamic display showing the virtual interactive avatar's gaze direction changing along the rotation trajectory.

[0169] In some optional embodiments, based on any of the above embodiments, the third processing module 53 is further configured to: The target object is used as the following object, and the gaze direction of the virtual interactive avatar in the display screen is controlled to rotate according to the azimuth angle of the target object. In response to the target object changing from a following object to a non-following object within the first time period, the gaze direction of the virtual interactive avatar in the display screen is maintained as the first gaze direction of the target object in the last frame before it became a non-following object.

[0170] In some optional embodiments, the apparatus of this disclosure may further include a first control module 57. The second processing module 52 is further configured to, in response to the target object becoming a following object again within a second time period after becoming a non-following object, determine the current gaze direction of the virtual interactive avatar based on the current azimuth angle of the target object.

[0171] The third processing module 53 can also be used to generate a dynamic display screen showing the virtual interactive image rotating from the first gaze direction to the current gaze direction based on the first gaze direction and the current gaze direction.

[0172] The first control module 57 can also be used to display dynamic images on a display screen.

[0173] The third processing module 53 can also be used to update the display screen to a display screen in which the gaze direction of the virtual interactive image is the target direction if the target object does not become a following object within a second time period after becoming a non-following object.

[0174] The target direction is the second preset direction or the second gaze direction corresponding to the azimuth angle of the next target object.

[0175] In some optional embodiments, based on any of the above embodiments, the apparatus of this disclosure may further include: a seventh processing module, configured to generate a warning voice for a candidate object based on the azimuth angle; and a second control module, configured to play the warning voice through a sound playback device on the vehicle.

[0176] The embodiments of this exemplary device correspond to the embodiments of the exemplary method described above in terms of implementation. The corresponding content between the two can be referenced, combined, and cited, and will not be repeated here. The beneficial technical effects corresponding to the embodiments of this exemplary device can be found in the corresponding beneficial technical effects of the embodiments of the exemplary method described above, and will not be repeated here.

[0177] Exemplary electronic devices Figure 15 A structural diagram of an electronic device provided in this disclosure includes at least one processor 91 and a memory 92.

[0178] The processor 91 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0179] The memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute one or more computer program instructions to implement the display screen generation methods and / or other desired functions of the various embodiments of this disclosure described above.

[0180] In one example, the electronic device 90 may also include an input device 93 and an output device 94, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0181] The input device 93 may also include, for example, a touchscreen, a microphone, various sensors, etc. Sensors may include, for example, image sensors (e.g., cameras, webcams), LiDAR, millimeter-wave radar, ultrasonic radar, positioning sensors, pressure sensors, air quality sensors, temperature sensors, etc. Image sensors, LiDAR, millimeter-wave radar, ultrasonic radar, etc., can be used for environmental perception, i.e., detecting moving and static objects in the surrounding environment. Moving and static objects may include, for example, static objects such as lane lines, curbs, arrows, signs, trees, and buildings, as well as dynamic objects such as surrounding vehicles, pedestrians, and cyclists. Positioning sensors are used to locate the mobile device (e.g., a bicycle, a robot, etc.) where the electronic device is located. Positioning sensors may include, for example, an Inertial Measurement Unit (IMU), a Global Positioning System (GPS), etc. Pressure sensors can be used to detect seat pressure. Temperature sensors can be used to detect the temperature inside the vehicle cabin. Air quality sensors can be used to detect the air quality inside the vehicle cabin.

[0182] The output device 94 can output various information to the outside, including, for example, a display, a speaker, a communication network and its connected remote output devices, etc.

[0183] Of course, for the sake of simplicity, Figure 15 Only some of the components of the electronic device 90 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 90 may include any other suitable components depending on the specific application.

[0184] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the methods for generating display screens of various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0185] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0186] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods for generating display screens of various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0187] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0188] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0189] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, this disclosure is also intended to include such modifications and variations if they fall within the scope of the claims of this disclosure and their equivalents.

Claims

1. A method for generating a display screen, comprising: In sentry mode, in response to the determination that there is a candidate object to be warned around the vehicle, the azimuth angle of the candidate object relative to the display screen is determined based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system. Based on the azimuth angle, determine the target gaze direction of the virtual interactive avatar; The virtual interactive avatar is rendered based on the target gaze direction to generate a display screen showing the virtual interactive avatar gazing at the target object in the target gaze direction.

2. The method according to claim 1, wherein, The first coordinate system is a two-dimensional coordinate system based on a preset plane of the vehicle coordinate system; determining the azimuth angle of the candidate object relative to the display screen based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system includes: Based on the first position and the second position, a first angle is determined between the line segment between the first position and the second position and the first preset direction in the first coordinate system; the first preset direction is the direction of the projection line of the normal of the display screen onto the preset plane; The first included angle is taken as the azimuth angle of the candidate object relative to the display screen.

3. The method according to claim 1, wherein, Determining the target gaze direction of the virtual interactive avatar based on the azimuth angle includes: Based on the azimuth angle and the preset angle range, the target object is determined from the candidate objects; Based on the azimuth angle of the target object, the target gaze direction of the virtual interactive avatar is determined.

4. The method according to claim 3, wherein, The step of determining the target object from the candidate objects based on the azimuth angle and the preset angle range includes: In response to the fact that only one candidate object among the candidate objects has an azimuth angle within the preset angle range, the first candidate object is selected as the target object; or... In response to the existence of multiple second candidate objects with azimuth angles within the preset angle range among the candidate objects, the second candidate object with the longest candidate duration is selected as the target object based on the candidate duration of the multiple second candidate objects; the candidate duration is the duration during which the second candidate object belongs to the candidate object to be warned.

5. The method according to claim 3, wherein, Determining the target gaze direction of the virtual interactive avatar based on the azimuth angle further includes: In response to the failure to determine the target object based on the azimuth angle and the preset angle range, the target gaze direction of the virtual interactive image is determined to be the second preset direction.

6. The method according to claim 1, wherein, The step of rendering the virtual interactive avatar based on the target gaze direction to generate a display screen showing the virtual interactive avatar gazing at the target object in the target gaze direction includes: Based on the target gaze direction and the image model of the virtual interactive image, a dynamic display screen is generated by rendering, in which a gaze beam is emitted from the virtual interactive image and the gaze beam flashes according to a preset pattern.

7. The method according to claim 1, wherein, After generating the display screen, the process also includes: In response to the target object's azimuth angle changing from a first azimuth angle to a second azimuth angle, the rotation trajectory of the virtual interactive image is determined based on the first azimuth angle and the second azimuth angle; The virtual interactive avatar is rendered based on the rotation trajectory, generating a dynamic display screen in which the gaze direction of the virtual interactive avatar changes with the rotation trajectory.

8. The method according to claim 1, wherein, After generating the display screen, the method further includes: Using the target object as the following object, the gaze direction of the virtual interactive image in the display screen is controlled to rotate to follow the change of the azimuth angle of the target object; In response to the target object changing from a following object to a non-following object within a first duration, the gaze direction of the virtual interactive image in the display screen is maintained as the first gaze direction of the target object in the last frame before it becomes a non-following object.

9. The method according to claim 8, further comprising: In response to the target object becoming a following object again within a second time period after becoming a non-following object, the current gaze direction of the virtual interactive image is determined based on the current azimuth angle of the target object; Based on the first gaze direction and the current gaze direction, a dynamic display screen is generated showing the virtual interactive image rotating from the first gaze direction to the current gaze direction; The dynamic display image is displayed on the display screen; or... In response to the fact that the target object does not become a following object within a second time period after becoming a non-following object, the display screen is updated to a display screen in which the gaze direction of the virtual interactive image is the target direction; The target direction is the second preset direction or the second gaze direction corresponding to the azimuth angle of the next target object.

10. The method according to any one of claims 1-9, further comprising: Based on sensor data collected by sensors on the vehicle, a perception result is determined; the perception result includes the state information of one or more objects. Based on the perception results, the candidate objects are determined; Based on the perspective transformation matrix and the first pixel position in the state information of the candidate object, the first position of the candidate object in the first coordinate system is determined, where the first pixel position is the pixel position corresponding to the point on the candidate object that is closest to the vehicle.

11. The method according to claim 10, wherein, The step of determining the candidate object based on the perception result includes: Based on the state information of one or more objects in the perception results and the preset risk triggering conditions, determine the objects whose state information meets the risk triggering conditions from the one or more objects; In response to the existence of at least one object among the one or more objects that satisfies the risk triggering condition, the object that satisfies the risk triggering condition is selected as the candidate object.

12. The method of claim 11, further comprising: In response to the existence of at least one object among the one or more objects that satisfies the risk triggering condition, the display screen is controlled to switch from a first preset state to a first warning state; After generating a display screen showing the virtual interactive avatar gazing at the target object in the direction of the target's gaze, the display screen is then displayed on the display screen. or, In response to the absence of an object satisfying the risk triggering condition among the one or more objects, the display screen is controlled to maintain the first preset state.

13. The method according to any one of claims 1-9, further comprising: Based on the azimuth angle, generate a warning voice message for the candidate object; The warning message is played through the audio playback device on the vehicle.

14. An apparatus for generating a display screen, comprising: The first processing module is configured, in sentry mode, in response to determining that there is a candidate object to be warned around the vehicle, to determine the azimuth angle of the candidate object relative to the display screen based on the first position of the candidate object in the first coordinate system and the second position of the display screen in the first coordinate system; The second processing module is used to determine the target gaze direction of the virtual interactive image based on the azimuth angle of the candidate object relative to the display screen. The third processing module is used to render the virtual interactive image based on the target gaze direction, and generate a display screen of the virtual interactive image gazing at the target object in the target gaze direction.

15. A computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the method for generating a display screen according to any one of claims 1-13.

16. An electronic device, the electronic device comprising: Memory is used to store processor-executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for generating the display screen according to any one of claims 1-13.

Images