Vehicle environment alarm method and apparatus

Abstract

A vehicle environment alarm method and apparatus. The vehicle environment alarm method comprises: acquiring an environmental image around a vehicle as a first environmental image, the environmental image comprising an alarm line used for at least dividing the environmental image into a safety area and a risk area (S301); when it is determined that there is a target object in the safety area of the first environmental image, recording first positional information of the target object in the first environmental image, acquiring an environmental image around the vehicle again as a second environmental image, and recording second positional information of the target object in the second environmental image (S302); on the basis of the first positional information and the second positional information, determining position change information of the target object relative to the alarm line (S303); and when it is determined that the position change information satisfies an alarm condition, triggering an alarm operation of the vehicle for the target object (S304).

Description

Vehicle environment warning method and device CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the patent application number 202410075316.0 filed on January 17, 2024 with the Chinese Patent Office. The entire contents of the patent application are incorporated herein by reference. Technical Field

[0001] The embodiments of the present application relate to, but are not limited to, the automotive field, and in particular relate to, but are not limited to, a vehicle environment alarm method and device. Background Art

[0002] As the main means of transportation for humans today, cars usually have high property value and need to be protected. Summary of the Invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] According to a first aspect of the present application, a vehicle environment alarm method is provided, the method comprising: acquiring an environmental image of the vehicle's surroundings as a first environmental image, the environmental image comprising a warning line for dividing the environmental image into at least a safe area and a risk area; upon determining that a target object exists in the safe area of ​​the first environmental image, recording first position information of the target object in the first environmental image, acquiring the environmental image of the vehicle's surroundings again as a second environmental image, and recording second position information of the target object in the second environmental image; determining position change information of the target object relative to the warning line based on the first position information and the second position information; and triggering an alarm operation of the vehicle for the target object upon determining that the position change information satisfies an alarm condition.

[0005] According to a second aspect of the present application, a vehicle environment warning device is provided, the device comprising: an image acquisition unit for acquiring an environmental image around the vehicle as a first environmental image, the environmental image comprising a warning line for dividing the environmental image into at least a safe area and a risk area; a position information recording unit for, upon determining that a target object exists in the safe area of ​​the first environmental image, recording first position information of the target object in the first environmental image, acquiring the environmental image around the vehicle again as a second environmental image, and recording second position information of the target object in the second environmental image; a position change information determination unit for determining position change information of the target object relative to the warning line based on the first position information and the second position information; and an alarm unit for triggering an alarm operation of the vehicle against the target object upon determining that the position change information satisfies an alarm condition.

[0006] According to a third aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0007] According to a fourth aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method described in the first aspect are implemented.

[0008] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0009] In an embodiment of the present application, by dividing the safe area and risk area by the warning line in the acquired environmental image, it is possible to simply determine whether a target object exists in the safe area with less resource computing power; and when the target object exists, the corresponding first position information and the second position information corresponding to the target object in the next acquired environmental image can be recorded, and the position change information of the target object relative to the warning line can be determined based on the first position information and the second position information. Since the position change information involves information from multiple different environmental images, the target object can be continuously identified and detected on a longer time scale. Therefore, the solution of the present application can more comprehensively detect complex vehicle environments and can perform warning operations in a timely manner. Other aspects can be understood after reading and understanding the drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] To more clearly illustrate the technical solution of this application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0011] FIG1 is a schematic diagram of the architecture of a vehicle environment warning system shown in an exemplary embodiment of the present application.

[0012] FIG2 is a schematic diagram of a logic execution flow of a vehicle environment warning system according to an exemplary embodiment of the present application.

[0013] FIG3 is a flow chart of a vehicle environment warning method according to an exemplary embodiment of the present application.

[0014] FIG4 is a schematic diagram of an environment image related to abnormal water level, shown in an exemplary embodiment of the present application.

[0015] FIG5 is another schematic diagram of an environment image related to abnormal water level, shown in an exemplary embodiment of the present application.

[0016] FIG6 is a schematic diagram of an environmental image related to a special water accumulation situation, shown in an exemplary embodiment of the present application.

[0017] FIG. 7 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application.

[0018] FIG8 is a schematic structural diagram of a vehicle environment warning device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of the present application.

[0020] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0022] With the changes in climate conditions in recent years, cities have begun to suffer from more frequent and severe weather and climate disasters, making most vehicles parked throughout the city more vulnerable to damage. Since users often cannot rush to the scene in the first time to prevent or reduce the impact on the vehicles, how to ensure that the vehicles are in a safe environment has become a difficult problem that needs to be solved urgently in the industry.

[0023] Typically, a vehicle's sentry mode is used to identify and alert people or other vehicles near the vehicle. However, this method makes it difficult to promptly and correctly identify and respond to special situations such as vehicles submerged in water due to urban flooding or vehicles covered in snow due to extreme weather, resulting in certain risks to users' property safety.

[0024] The following describes in detail an embodiment of the vehicle environment warning method of the present application with reference to the accompanying drawings.

[0025] FIG1 is a schematic diagram of the architecture of a vehicle environment warning system according to an exemplary embodiment of the present application. As shown in FIG1 , the system may include the following components:

[0026] The Around View Monitor (AVM) system 11 can capture images of the vehicle in different directions using multiple corresponding ultra-wide-angle fisheye lenses. It can also perform distortion correction and splicing on these images using a special algorithm to form a panoramic image of the vehicle's surroundings. This panoramic image can be presented based on multiple sets of images or videos, which is not described in detail in this application. During operation, the AVM system 11 can capture real-time video streams, such as those captured by four fisheye cameras on the vehicle, and transmit these video streams to the driving recorder 12 via a network, for example, using Ethernet sockets.

[0027] The Digital Video Recorder (DVR) 12 is a device that records images, sounds, and other related information while the vehicle is in motion. During system operation, the DVR 12 receives the video stream from the panoramic surveillance imaging system 11 and transmits it to the driving information and entertainment host 13 via a socket.

[0028] The Digital Cockpit Head Unit (DHU) 13, as the vehicle's cockpit host, may include a system on chip (SOC) 131 for detecting target objects and a microcontroller unit (MCU) 132 for synchronizing vehicle signals. The system on chip 131 may implement data transmission to the microcontroller unit 132 based on a serial bus protocol such as SPI (Serial Peripheral Interface). During operation of the system-on-chip, the system-on-chip 131 can determine whether a target object exists in the video stream transmitted by the travel recorder 12 based on visual recognition algorithms such as YOLO, ResNet, Viola-Jones, DPM, Transform, R-CNN, Fast R-CNN, and Faster R-CNN, thereby generating a corresponding alarm event and transmitting the alarm event to the microcontroller unit 132. At the same time, the image and related information of the target object can be intuitively fed back to the user to the central control panel 14 via a serial interface such as a Gigabit Multimedia Serial Link (GMSL), and the video stream of the corresponding time period can be saved to the external storage device 15 as a backup. After receiving the alarm event, the microcontroller unit 132 can adjust various vehicle parameters based on the alarm event, such as the parameters of the lights, horn, or chassis, and transmit the alarm event to the vehicle body gateway module 16 via, for example, a controller area network (CAN) bus. At the same time, the length of the image can be limited to a preset time period before and after the presence of the target object to avoid subsequent processing of invalid images. The system on chip 131 may include one or more layers of operating systems to process the algorithm detection calculation process for the video stream. The operating system may be an in-vehicle system such as QNX, Android, Linux or Windows CE, which is not limited in this application.

[0029] The touch-screen 14 is equipped with a display screen to display images of the target object to the user, and can also provide an input interface for the user to send corresponding information and instructions to the on-chip system 131 through the touch-screen 14. Specifically, the information and instructions can be transmitted between different components based on a serial communication protocol such as an inter-integrated circuit (I2C).

[0030] The external storage device 15 is a persistent memory for storing and backing up the image of the target object received in the on-chip system 131, such as a USB flash disk, a mobile hard disk, a Secure Digital Card (SD card), etc., which is not limited in this application.

[0031] The vehicle gateway module (VGM) 16 can connect various components in the vehicle to the network. During the operation of the system, the vehicle gateway module 16 can send the alarm event to the user terminal via the Internet after receiving the image, avoiding the user from not being able to understand the occurrence of abnormal situations in time because they are not near the vehicle.

[0032] The user terminal 17 is an electronic device capable of accessing the vehicle. During operation of the system, the vehicle is bound to at least one user terminal, so that the alarm event can be sent to the bound user terminal via the vehicle body gateway module 16. Furthermore, the user terminal can be a mobile phone, tablet device, laptop computer, PDA (Personal Digital Assistant), or wearable device (such as smart glasses, smart watches, etc.), and one or more embodiments of the present application are not limited thereto.

[0033] Based on the component architecture of Figure 1, the objects involved in the vehicle environment alarm method described below can be divided into system services, media services, algorithm services, vehicle control services, storage services, gateways, and user terminals based on the execution logic. The execution process of each of the above objects can be described by Figure 2. Figure 2 is a schematic diagram of the logic execution flow of a vehicle environment alarm system shown in an exemplary embodiment of the present application. As shown in Figure 2, the logic execution flow can include steps S201 to S212.

[0034] S201, initializing system services.

[0035] S202: Start the media service.

[0036] In one embodiment, the system service, as a basic service of the vehicle system, can be used to start and call the media service so that the media service can obtain the corresponding video stream through, for example, a four-way fisheye camera, and the video stream can correspond to the environment in which the vehicle is located.

[0037] S203: The media service sends the video stream to the algorithm service.

[0038] In one embodiment, the media service can perform secondary processing on the acquired video stream, such as frame control, splicing, and abnormal frame operations, and send the video stream to the relevant components of the algorithm service after processing. The component can be the SOC (system on chip) 131, wherein the video stream processing operation in the media service can be performed by the panoramic monitoring imaging system 11 and / or the driving recorder 12. This application does not limit the operational division of labor of each component.

[0039] S204: The algorithm service determines whether an alarm event is triggered.

[0040] In one embodiment, the component corresponding to the algorithm service identifies and analyzes different frame images in the received video stream based on a preset visual algorithm, and then determines whether there is a target object in the vehicle environment shown in the video stream.

[0041] S205: The algorithm service returns the judgment result to the media service.

[0042] S206: The media service returns the determination result to the system service.

[0043] In one embodiment, the algorithm service may return the judgment result obtained by the visual algorithm to the media service, and the media service may return it to the system service.

[0044] S207: The system service sends a vehicle control request based on the judgment result.

[0045] In one embodiment, it is assumed that the system service does not perform special processing for normal vehicle environments, but synchronizes the entire vehicle signal for abnormal vehicle environments. Therefore, if the judgment result returned by the media service indicates that the vehicle environment is abnormal, the system service may send a vehicle control request to the relevant component of the vehicle control service, where the relevant component of the vehicle control service may be the microcontroller unit (MCU) 132.

[0046] S208 , the vehicle control service notifies the corresponding component to execute the vehicle control request.

[0047] In one embodiment, assuming that the vehicle control request requires the vehicle to: turn on the hazard lights, repeatedly sound the horn, illuminate the center console screen, and raise the chassis, the vehicle control service can make corresponding adjustments through the corresponding electronic control unit (ECU) in the vehicle.

[0048] S209: The vehicle control service sends the image to be saved to the storage service.

[0049] S210: The vehicle control service receives the saving result returned by the storage service.

[0050] In one embodiment, the vehicle control service may send a video containing at least the target object to a storage service for storage backup, and the storage service may return the corresponding storage result to the vehicle control service. The storage device corresponding to the storage service may be a built-in persistent storage device in the vehicle, an external persistent storage device such as a USB flash drive, or a cloud server, and this application does not limit this.

[0051] S211, the vehicle control service sends a notification request to the gateway.

[0052] S212: The gateway sends an alarm notification to the user terminal.

[0053] In one embodiment, the vehicle control service can send a notification request to the gateway in the vehicle, such as the body gateway module 16, for displaying an alarm message on the user terminal. After receiving the notification request, the gateway can send the alarm message to the corresponding user terminal for display, thereby ultimately realizing information interaction between the vehicle and the user terminal.

[0054] FIG3 is a flow chart of a vehicle environment warning method according to an exemplary embodiment of the present application. As shown in FIG3 , the method may include the following steps S301 to S304 .

[0055] S301 : Acquire an environmental image around a vehicle as a first environmental image, wherein the environmental image includes warning lines for dividing the environmental image into at least a safe area and a risk area.

[0056] When a vehicle is parked or in a stopped state, the vehicle can obtain an image of the environment around the vehicle through the panoramic monitoring imaging system. The environmental image can be, for example, a collection of frame images from a video captured by the four fisheye cameras in the four directions of the front, rear, left, and right of the vehicle, or a frame image of a panoramic video captured directly by a 360-degree camera. The positional relationship between the frame image and the video in which it is located can be determined by the image screenshot interval, time point, or custom rules pre-set by the vehicle. Of course, in addition to indirectly obtaining the image of the environment around the vehicle through the video stream, the panoramic monitoring imaging system can also directly capture the environment around the vehicle to obtain an image, thereby reducing the computing resources consumed by the panoramic monitoring imaging system in the scenario where the vehicle alerts the vehicle environment.

[0057] The environmental image can be directly drawn with a warning line in a manner of being directly superimposed on the environmental image during further processing after acquisition, or the warning line in the environmental image can be independently represented in the form of an additional file associated with the environmental image; the warning line can be a straight line, a broken line or a curve in the environmental image, used to divide the environmental image into at least two parts, a safe area and a risk area, and the two can be determined based on the relative position relationship between the safety area and the warning line. Specifically, the safety area can be the area in the environmental image that is close to the vehicle relative to the warning line, and the risk area can be the area in the environmental image that is away from the vehicle relative to the warning line; or, the safety area can be the area in the environmental image that is away from the vehicle relative to the warning line, and the risk area can be the area in the environmental image that is close to the vehicle relative to the warning line.

[0058] Take a warning line that crosses the environmental image as an example. The warning line intuitively divides the environmental image into two parts, an upper part and an lower part. Assuming that the upper part is far from the vehicle and the lower part is close to the vehicle, the vehicle can determine the relative position relationship between the safe area, the risk area and the warning line based on the preset relative position relationship. For example, the area close to the vehicle side of the warning line is the risk area, and the area away from the vehicle side of the warning line is the safe area by default; or the warning line and / or the safe area and the risk area can be dynamically switched according to the identified target object. For example, in the case where the target object is a pool in the terrain, the area close to the vehicle side relative to the warning line can be set as the safe area, and the area away from the vehicle side relative to the warning line can be set as the risk area; in the case where the target object is a fire source in a fire, the area close to the vehicle side relative to the warning line can be set as the risk area, and the area away from the vehicle side relative to the warning line can be set as the safe area. The specific setting rules will be introduced in the embodiments below, and this application will not be repeated here.

[0059] It can be understood that there can be multiple warning lines, so that there are more than two areas in the environmental image. In addition to the safe area and the risk area, the additional area can serve as a transition area between the safe area and the risk area, and perform additional operations different from the safe area or risk area in combination with the location of the target object, thereby providing a technical basis for the scalability of the warning capability of abnormal vehicle environments.

[0060] S302, when it is determined that there is a target object in the safety area of ​​the first environmental image, record the first position information of the target object in the first environmental image, obtain the environmental image around the vehicle again as a second environmental image, and record the second position information of the target object in the second environmental image.

[0061] After acquiring the environmental image, the vehicle can inspect target objects that may be causing abnormalities in the vehicle environment. These target objects can include pedestrians, vehicles, and key features of abnormal disasters, such as smoke and flames in a fire, snow accumulation in a blizzard, and water accumulation in a flood. Accordingly, the target objects can be associated with first position information in the environmental image to quantify their spatial positions and facilitate the determination of position change information in subsequent steps.

[0062] Different from the application of traditional visual algorithms, after acquiring any environmental image, the vehicle in the embodiment of the present application not only needs to determine the warning line that divides the environmental image into a safe area and a risk area and check whether there is a target object in the environmental image according to the target recognition algorithm, but also needs to determine whether the target object is located in the safe area, and determine the first position information of the target object in the environmental image if the judgment result is yes.

[0063] In one embodiment, the environmental image can be constructed with a plane coordinate system, and the warning line intersects with the edge of the environmental image at least at a first coordinate point and a second coordinate point in the plane coordinate system; then the vehicle can first detect whether the target object exists in the environmental image, and if the detection result is yes, determine the relative position relationship between the target coordinate point of the target object in the environmental image and the warning line, and if the relative position relationship indicates that the target coordinate point is located on the target side of the warning line, determine that the target object exists in the safe area of ​​the environmental image. The specific content of the first position information may include: the coordinates of the target coordinate point and\or the vertical distance between the target coordinate point and the warning line; the target side can be determined based on the preset feature change trend of the target object, and the preset feature change trend can represent the spreading trend of the target object in the environmental image. Considering that the preset feature change trend is given in detail below in this application, it will not be described in detail here.

[0064] Taking Figure 4 as an example, the environmental image shown in Figure 4 is a collection of four pictures taken by the vehicle's four fisheye cameras in the four directions of the front, rear, left and right of the vehicle, respectively. Among them, the corresponding pictures in each direction can be respectively configured with a plane coordinate system, and the warning line of each picture corresponds to two endpoints. Therefore, the four pictures have a total of eight endpoints. If the eight endpoints in the four coordinate systems are connected end to end in three-dimensional space, a rectangle containing the vehicle body can be formed. The specific size of the rectangle is related to the vehicle model.

[0065] Specifically, before a car model is released, the manufacturer can perform four-corner calibration on the actual vehicle to determine the coordinates of each endpoint on each picture; taking the sub-picture in the upper left corner of Figure 4 as an example, the sub-picture is a shot of the front of the vehicle, where A1 and A2 are the first coordinate point and the second coordinate point in the image, and the two are the intersection points of the warning line and the edge of the image. Assuming that the four-corner calibration stipulates that the warning line formed by connecting A1 and A2 is a virtual line segment 1 meter away from the front of the vehicle and parallel to the front of the vehicle, then in the corresponding plane coordinate system, the target object whose target coordinate point is lower than the warning line can be regarded as less than 1 meter away from the front of the vehicle, and the target object whose target coordinate point is higher than the warning line can be regarded as more than 1 meter away from the front of the vehicle (taking into account the perspective and distortion of the environmental image, the above distance conversion method can be made more accurate by adding additional warning lines, correcting the curvature of the warning lines, etc.).

[0066] In scenarios where disasters such as floods occur, vehicles need to be alert to abnormal situations of being submerged by accumulated water. In the process of the vehicle being submerged by accumulated water, according to the characteristics of accumulated water accumulating on the ground, the preset characteristic change trend of the water level is to spread from the bottom to the top of the image taken by the vehicle (that is, the target coordinate point of the water level continues to rise in the corresponding coordinate system). Therefore, the side of the environmental image that is close to the vehicle relative to the warning line can be used as the target side, and the area on the target side can be set as the safe area of ​​the environmental image, and the area on the side far from the vehicle relative to the warning line in the environmental image can be set as the risk area. When the position of the target coordinate point of the water level in the environmental image is below the warning line, it can be determined that there is a target object of accumulated water in the safe area; when the position of the target coordinate point of the water level in the environmental image is above the warning line, it can be determined that the water level is too high and has reached the risk area.

[0067] In the event of a fire or other disaster, a vehicle needs to be alert to the abnormal situation of being surrounded by flames or smoke. In the process of being surrounded by flames or smoke, according to the characteristics of the flames or smoke accumulating in the air, the preset characteristic change trend of the flames or smoke is to spread from the top to the bottom of the vehicle image (that is, the target coordinate point of the flame or smoke continues to descend in the corresponding coordinate system). Therefore, the side of the environmental image that is far away from the vehicle relative to the warning line can be used as the target side, and the area of ​​the target side is set as the safe area of ​​the environmental image, and the area of ​​the environmental image that is close to the vehicle relative to the warning line is set as the risk area. When the position of the target coordinate point of the flame or smoke in the environmental image is above the warning line, it can be determined that there is a target object of fire in the safe area; when the position of the target coordinate point in the environmental image is below the warning line, it can be determined that the fire is too close to the vehicle and has reached the risk area. In addition, the target side changes according to the determined target object. When there are multiple target objects in the environmental image, the target side corresponding to each target object can be independent of each other, thereby ensuring the accuracy of the alarm.

[0068] Among them, the target coordinate point of the water level can be abstracted as a coordinate point with the largest vertical coordinate in the plane coordinate system of the line segment formed by the water level, such as aa in Figure 4; or a coordinate point corresponding to the average value of the vertical coordinate of the line segment formed by the water level in the plane coordinate system, or a coordinate point corresponding to the center point of the entire water accumulation. The target coordinate points of target objects such as snow and smoke are processed in a similar way to the water level, and this application will not go into details here.

[0069] It should be noted that if there is no target object in the environmental image, then the vehicle environment is normal. In other words, the vehicle does not need to record the first position information or perform any subsequent steps before obtaining the next environmental image and determining the target object; if there is any target object in the environmental image, and any target object is located in the risk area, then the vehicle can directly trigger the alarm operation for the target object in S304 according to the preset rules, so as to ensure that the user can receive the corresponding alarm information as soon as the vehicle is in an abnormal vehicle environment.

[0070] S303: Determine position change information of the target object relative to the warning line based on the first position information and the second position information.

[0071] As previously mentioned, the first position information includes the coordinates of the target point of the target object and / or the vertical distance between the target point and the warning line, thereby representing the specific position of the target object in the currently acquired environmental image. Similarly, for the specific position of the target object in the next acquired environmental image, the second position information of the target object can also be obtained in the same manner as the first position information. Obviously, by comparing the difference between the first position information and the second position information, it is possible to quickly determine the position change information used to represent the change in the target object during the time between the two environmental images. The acquisition strategy corresponding to the "next environmental image" can be actively or automatically adjusted by the user or vehicle based on the actual scenario. For example, the environmental image acquired 10 seconds after the last environmental image can be used as the next environmental image, thereby ensuring timely warning of vehicle environmental safety conditions while avoiding excessive resource consumption. For another example, the next frame of the environmental image after the last environmental image can be used as the next environmental image, thereby increasing the detection frequency of abnormal vehicle environments, so that the user receives corresponding warning information as soon as the vehicle is in an abnormal vehicle environment. In some embodiments, if there is any target object in the current environmental image (also referred to as the first environmental image) and there is no target object in the environmental images before the current environmental image, the shooting time interval between the next environmental image (also referred to as the second environmental image) and the current environmental image can be 5 minutes. Taking the target object as the water level as an example, if it is detected that the water level has not changed within these 5 minutes, the shooting time interval between the next environmental image (also referred to as the third environmental image) and the second environmental image can be 5 minutes; if it is detected that the water level has changed within these 5 minutes, the shooting time interval between the third environmental image and the second environmental image can be 1 minute.

[0072] Taking Figure 5 as an example, Figure 5 is the next environmental image acquired by the vehicle after Figure 4. In the event of a disaster such as a flood, assuming that the vertical coordinate value of the target water level coordinate point aa` has further increased compared to Figure 4, a classification discussion can be conducted based on the positional relationship between the target coordinate point and the warning line: that is, if the target coordinate point aa` is in the risk area, then the subsequent warning operation can be directly triggered; if the target coordinate point aa` is still in the safe area, then the position change information can be used to determine whether to trigger the subsequent warning operation. Specifically, the position change information can be determined in the following different ways.

[0073] In one embodiment, when the first position information includes the target coordinate point, the difference in distance between the target coordinate point corresponding to the first position information and the target coordinate point corresponding to the second position information can be converted into an actual change distance, and the actual change distance can be used as the position change information, wherein the position change information can refer to the conversion of the displacement of the horizontal coordinate x and the vertical coordinate y of the target coordinate point on the plane coordinate system into an actual displacement in three-dimensional space in units such as pixels. Assuming that in the two environmental images corresponding to the first position information and the second position information of the target object, the translation vector between the target coordinate points of the two environmental images corresponding to the same direction (for example, in front of the vehicle) is Δ1 (Δx1, Δy1), then the displacement of the target object in the three-dimensional space is Δ2 (Δx2, Δy2, Δz2). According to the perspective projection transformation formula, the displacement Δ2 (Δx2, Δy2, Δz2) of the target object in the three-dimensional space can be expressed as: Δx2 = Δx1 / tan(fov / 2)×z Δy2 = Δy1 / tan(fov / 2)×z Δz2 = 0

[0074] Among them, fov is the field of view, which represents the viewing angle of the corresponding camera observing the scene. In this application, it can be assumed that the camera viewing angle is fixed, so fov can be regarded as a constant. By solving the set of equations, the displacement Δ2 (Δx2, Δy2, Δz2) of the target object in 3D space can be determined. Since the two-dimensional environmental image cannot show the depth of the image, it can be assumed that z=1m and Δz remains unchanged. For example: the coordinate value of the target coordinate point aa in Figure 4 is (200, 100), the coordinate value of the target coordinate point aa` in Figure 5 is (150, 300), fov is 120°, z=1 meter, then Δ1 (Δx1, Δy1) can be (-50, 200), and the difference in distance between the two coordinate points can be represented as:

[0075] Through the modular formula of three-dimensional space The actual change distance before and after the water level rise can be obtained, and its distance unit can be finally determined according to the preset proportional coefficient k between the pixel and the actual change distance.

[0076] In another embodiment, when the first position information and the second position information include the vertical distance between the target coordinate point and the warning line, the difference between the vertical distance corresponding to the first position information and the vertical distance corresponding to the second position information can be converted into the actual change distance, and the actual change distance is used as the position change information of the target object. Compared to the previous embodiment, in this embodiment, the actual change distance of the target object can be obtained by simply multiplying the difference between the vertical distances of the first position information and the second position information by a preset proportionality factor j representing the pixel and the actual change distance.

[0077] S304: When it is determined that the position change information satisfies an alarm condition, trigger an alarm operation of the vehicle for the target object.

[0078] Once the position change information indicates that the target object actually endangers the safety of the vehicle, it can trigger the vehicle's alarm operation for the target object, where the situation can be determined by a preset alarm condition. For example, if the actual change distance in the position change information is greater than a preset change distance threshold, it indicates that the target object is approaching the vehicle too fast within a certain period of time (taking accumulated water as an example, it shows a trend of continuous rapid rise), and therefore it is expected that the vehicle environment will be abnormal; or, the target object exists continuously in the environmental images obtained multiple times for a time period greater than a preset safety time threshold, or other conditions.

[0079] The alarm operation may be that the vehicle sends an alarm in the form of images, text, etc. to the user's mobile phone or other terminal through the network. The alarm operation may no longer be executed when the number of triggers reaches the predicted number or the user actively starts the vehicle, thereby avoiding the user from receiving repeated information.

[0080] Of course, the target objects in the environmental images of Figures 4 and 5 approach the vehicle in an all-round manner. In special circumstances, such as uneven ground and water surfaces, target objects may also approach the vehicle from a certain direction as shown in Figure 6. The vehicle can monitor the images in different directions in the environmental images separately to identify the target objects in time.

[0081] FIG7 is a schematic structural diagram of an electronic device in an exemplary embodiment. Referring to FIG7 , at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other required hardware. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an alarm device for the vehicle environment at the logical level. Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0082] Corresponding to the embodiment of the aforementioned vehicle environment alarm method, the present application also provides an embodiment of a vehicle environment alarm device.

[0083] Please refer to FIG8 , which is a schematic diagram of a vehicle environment warning device according to an exemplary embodiment. As shown in FIG8 , the device may include: an image acquisition unit 801 for acquiring an environmental image of the vehicle surroundings as a first environmental image, wherein the environmental image includes a warning line for dividing the environmental image into at least a safe area and a risk area; a position information recording unit 802 for, upon determining that a target object exists in the safe area of ​​the first environmental image, recording first position information of the target object in the first environmental image, acquiring the environmental image of the vehicle surroundings as a second environmental image again, and recording second position information of the target object in the second environmental image; a position change information determination unit 803 for determining position change information of the target object relative to the warning line based on the first position information and the second position information; and an alarm unit 804 for triggering an alarm operation of the vehicle for the target object upon determining that the position change information satisfies an alarm condition.

[0084] In some embodiments, the environmental image is constructed with a plane coordinate system; the position change information determination unit 803 is used to: detect whether the target object exists in the environmental image; if the result of the detection is yes, determine the relative position relationship between the target coordinate point of the target object in the plane coordinate system of the environmental image and the warning line; if the relative position relationship indicates that the target coordinate point is located on the target side of the warning line, determine that the target object exists in the safe area of ​​the environmental image.

[0085] In some embodiments, the target side is the side close to the vehicle or the side away from the vehicle; the device also includes: a target side determination unit 805, which is used to determine the corresponding target side according to the characteristics of the target object, and the characteristics are used to characterize the change trend of the target object in the environmental image.

[0086] In some embodiments, the position change information determination unit 803 is used to: when the first position information and the second position information respectively include the target coordinate point, convert the difference in distance between the target coordinate point corresponding to the first position information and the target coordinate point corresponding to the second position information into an actual change distance, and use the actual change distance as the position change information; and / or when the first position information and the second position information respectively include the vertical distance between the target coordinate point and the warning line, convert the difference in vertical distance corresponding to the first position information and the vertical distance corresponding to the second position information into an actual change distance, and use the actual change distance as the position change information.

[0087] In some embodiments, the alarm condition includes: the actual change distance is greater than a preset change distance threshold.

[0088] In some embodiments, when the preset characteristic change trend of the target object is from near to far, the safe area is the area in the environmental image that is close to the vehicle side relative to the warning line, and the risk area is the area in the environmental image that is far from the vehicle side; or, when the preset characteristic change trend characteristic of the target object is from far to near, the safe area is the area in the environmental image that is far from the vehicle side relative to the warning line, and the risk area is the area in the environmental image that is close to the vehicle side relative to the warning line.

[0089] In some embodiments, the target object includes at least one of the following: a pedestrian, a vehicle, accumulated water, accumulated snow, smoke, and fire.

[0090] In some embodiments, when it is determined that a target object exists in the risk area of ​​the environment image, an alarm operation of the vehicle for the target object is triggered.

[0091] The implementation process of the functions and effects of each unit in the device is specifically described in the implementation process of the corresponding steps in the method, which will not be repeated here.

[0092] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0093] Embodiments of the subject matter and functional operations described in this application may be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this application and their structural equivalents, or a combination of one or more of them. Embodiments of the subject matter described in this application may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier to be executed by a data processing device or to control the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information and transmit it to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0094] The processes and logic flows described herein can be performed by one or more programmable computers executing one or more computer programs to perform the corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0095] Computers suitable for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to such mass storage devices to receive data from them or to transmit data to them, or both. However, a computer does not necessarily have such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0096] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0097] Although the present application includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of a particular invention. Certain features described in multiple embodiments of the present application can also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment can also be implemented separately in multiple embodiments or implemented in any suitable sub-combination. In addition, although features can work in some combinations as described above and even initially claim protection, one or more features from the claimed combination can be removed from the combination in some cases, and the claimed combination can point to a sub-combination or a variation of the sub-combination.

[0098] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or that all illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0099] Thus, specific embodiments of the subject matter have been described. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0100] The above descriptions are some embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An alarm method for a vehicle environment, comprising: Obtaining an environmental image around the vehicle as a first environmental image, where the environmental image includes an alarm line for at least dividing the environmental image into a safe area and a risk area; When it is determined that there is a target object in the safe area of the first environmental image, recording the first position information of the target object in the first environmental image, obtaining the environmental image around the vehicle again as a second environmental image, and recording the second position information of the target object in the second environmental image; Determining the position change information of the target object relative to the alarm line according to the first position information and the second position information; When it is determined that the position change information meets the alarm condition, triggering an alarm operation of the vehicle for the target object.

2. The method according to claim 1, wherein, A plane coordinate system is constructed in the environmental image, and the environmental image includes the first environmental image and / or the second environmental image; Determining that there is the target object in the safe area of the environmental image includes: Detecting whether there is the target object in the environmental image; When the result of the detection is yes, determining the relative position relationship between the target coordinate point of the target object in the plane coordinate system of the environmental image and the alarm line; When the relative position relationship indicates that the target coordinate point is located on the target side of the alarm line, determining that there is the target object in the safe area of the environmental image.

3. The method according to claim 2, wherein The target side is the side close to the vehicle or the side far from the vehicle; the method further includes: Determining the target side corresponding to the target object according to the preset feature change trend of the target object.

4. The method according to claim 2, wherein The determining the position change information of the target object relative to the alarm line includes: When the first position information and the second position information respectively include the target coordinate point, converting the distance difference between the target coordinate point corresponding to the first position information and the target coordinate point corresponding to the second position information into an actual change distance, and using the actual change distance as the position change information; and / or When the first position information and the second position information respectively include the vertical distance between the target coordinate point and the alarm line, converting the difference between the vertical distance corresponding to the first position information and the vertical distance corresponding to the second position information into an actual change distance, and using the actual change distance as the position change information.

5. The method according to claim 4, wherein The alarm condition includes: The actual change distance is greater than a preset change distance threshold.

6. The method according to claim 1, wherein When the preset feature change trend of the target object is from near to far, The safe area is the area in the environmental image that is on the side close to the vehicle relative to the alarm line, The risk area is the area in the environmental image that is on the side far from the vehicle relative to the alarm line.

7. The method according to claim 1, wherein When the preset feature change trend characteristic of the target object is from far to near, The safe area is the area in the environmental image that is on the side far from the vehicle relative to the alarm line, The risk area is the area on the side of the vehicle relative to the warning line in the environmental image.

8. The method according to claim 1, wherein, The method further includes: When it is determined that the target object exists in the risk area of the environmental image, triggering an alarm operation of the vehicle for the target object, where the environmental image includes the first environmental image and / or the second environmental image.

9. An alarm device for a vehicle environment, comprising: An image acquisition unit, configured to acquire an environmental image around the vehicle as a first environmental image, where the environmental image includes a warning line for at least dividing the environmental image into a safe area and a risk area; A position information recording unit, configured to record first position information of the target object in the first environmental image when it is determined that the target object exists in the safe area of the first environmental image, acquire the environmental image around the vehicle again as a second environmental image, and record second position information of the target object in the second environmental image; A position change information determination unit, configured to determine position change information of the target object relative to the warning line according to the first position information and the second position information; An alarm unit, configured to trigger an alarm operation of the vehicle for the target object when it is determined that the position change information meets the alarm condition.

10. A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in any one of claims 1 to 8 is implemented.

11. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the method described in any one of claims 1 to 8 is implemented.

Images