Vehicle abnormal area determination method and device, equipment and medium

Abstract

The embodiment of the invention discloses a vehicle abnormal area determination method and device, equipment and a medium, and the method comprises the steps: obtaining a vehicle environment video of a current vehicle, and determining a candidate watching focus and a candidate gesture focus of a candidate pedestrian; determining a target pedestrian from the candidate pedestrians according to the vehicle boundary interval and a preset focus distance threshold; determining the pedestrian number of the target pedestrians, and when the pedestrian number reaches a preset pedestrian number threshold value, determining an abnormal region center of the current vehicle according to the target watching focus and the target gesture focus of each target pedestrian; based on the vehicle environment video, determining candidate fixation time and a candidate sight line included angle of each target pedestrian, and determining a target fixation time length and a target sight line included angle; and determining the radius of an abnormal area according to the number of pedestrians, the target watching duration and the target sight line included angle, and determining the vehicle abnormal area of the current vehicle based on the radius of the abnormal area and the center of the abnormal area. And the accuracy of determining the abnormal area of the vehicle is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of automobile technology, and in particular to a method, device, equipment and medium for determining an abnormal area of ​​a vehicle. Background Art

[0002] With the development of science and technology, vehicles are frequently used in daily life. In order to improve the safety of vehicles, it is necessary to diagnose vehicle faults. The existing technology usually relies on in-vehicle sensors and diagnostic tools to determine the abnormal area of ​​the vehicle, but the accuracy is low. Summary of the invention

[0003] The present invention provides a method, device, equipment and medium for determining an abnormal area of ​​a vehicle, so as to improve the accuracy of the determined abnormal area of ​​the vehicle.

[0004] According to one aspect of the present invention, a method for determining an abnormal area of ​​a vehicle is provided, comprising:

[0005] Acquire a vehicle environment video of a current vehicle, and determine candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video;

[0006] According to the candidate gaze focus, the candidate gesture focus, the vehicle boundary interval and the preset focus distance threshold, a target gaze focus and a target gesture focus are selected from the candidate gaze focus and the candidate gesture focus, and the candidate pedestrians corresponding to the target gaze focus and the target gesture focus are selected as target pedestrians;

[0007] Determine the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determine the center of the abnormal area of ​​the current vehicle according to the target gaze focus and the target gesture focus of each target pedestrian;

[0008] Determine candidate gaze times and candidate sight angles of each target pedestrian based on the vehicle environment video, determine target gaze duration based on each candidate gaze time, and determine target sight angle based on each candidate sight angle;

[0009] The radius of the abnormal area is determined according to the number of pedestrians, the target gaze duration and the target sight angle, and the vehicle abnormal area of ​​the current vehicle is determined based on the radius of the abnormal area and the center of the abnormal area.

[0010] According to another aspect of the present invention, there is provided a vehicle abnormal area determination device, comprising:

[0011] A focus determination module, used to obtain a vehicle environment video of a current vehicle, and determine candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video;

[0012] a target pedestrian determination module, configured to select a target gaze focus and a target gesture focus from the candidate gaze focuses and the candidate gesture focuses according to the candidate gaze focuses, the candidate gesture focuses, the vehicle boundary interval and a preset focus distance threshold, and select the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as the target pedestrian;

[0013] an abnormal area center determination module, used to determine the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determine the abnormal area center of the current vehicle according to the target gaze focus and target gesture focus of each target pedestrian;

[0014] A target data determination module, for determining candidate gaze times and candidate sight angles of each target pedestrian based on the vehicle environment video, and determining a target gaze duration based on each candidate gaze time, and determining a target sight angle based on each candidate sight angle;

[0015] The abnormal area determination module is used to determine the radius of the abnormal area according to the number of pedestrians, the target gaze duration and the target sight angle, and determine the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center.

[0016] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0017] one or more processors;

[0018] A memory for storing one or more programs;

[0019] When one or more programs are executed by one or more processors, the one or more processors can execute any one of the vehicle abnormal area determination methods provided by the embodiments of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a processor to implement any vehicle abnormal area determination method provided by an embodiment of the present invention when executed.

[0021] An embodiment of the present invention provides a vehicle abnormal area determination solution, which obtains a vehicle environment video of a current vehicle, and determines candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video; selects a target gaze focus and a target gesture focus from the candidate gaze focuses and candidate gesture focuses according to the candidate gaze focuses, the candidate gesture focuses, the vehicle boundary interval and a preset focus distance threshold, and uses the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as target pedestrians; determines the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determines the abnormal area center of the current vehicle according to the target gaze focus and the target gesture focus of each target pedestrian; determines the candidate gaze time and the candidate sight angle of each target pedestrian based on the vehicle environment video, and determines the target gaze time based on each candidate gaze time, and determines the target sight angle based on each candidate sight angle; determines the abnormal area radius according to the number of pedestrians, the target gaze time and the target sight angle, and determines the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center. The above scheme determines the abnormal area of ​​the vehicle by introducing candidate gaze focus and candidate gesture focus of candidate pedestrians, realizes the determination of the abnormal area of ​​the vehicle based on the recognition of the movements of pedestrians around the vehicle, realizes the rapid positioning of the abnormal area of ​​the vehicle, and improves the efficiency of determining the abnormal area of ​​the vehicle; at the same time, by introducing a preset pedestrian number threshold, it realizes the use of target gaze focus and target gesture focus of multiple people, and comprehensively considers the factors of target gaze duration and target line of sight angle, thereby improving the accuracy and reliability of the determined abnormal area of ​​the vehicle; and by analyzing the behavioral data of people outside the vehicle, the human-computer interaction capability is enhanced.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a method for determining an abnormal area of ​​a vehicle provided in Embodiment 1 of the present invention;

[0025] Figure 2 is a flow chart of a method for determining an abnormal area of ​​a vehicle provided in Embodiment 2 of the present invention;

[0026] Figure 3 is a structural schematic diagram of a vehicle abnormal area determination device provided by Embodiment 3 of the present invention;

[0027] Figure 4 It is a structural schematic diagram of an electronic device for implementing a method for determining an abnormal area of ​​a vehicle provided in a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0029] Embodiment 1

[0030] Figure 1 It is a flow chart of a vehicle abnormal area determination method provided in Example 1 of the present invention. This embodiment is applicable to the situation of determining abnormal areas in a vehicle. The method can be executed by a vehicle abnormal area determination device, which can be implemented in software and / or hardware and can be configured in an electronic device that carries the vehicle abnormal area determination function.

[0031] See also Figure 1 The method for determining an abnormal area of ​​a vehicle shown includes:

[0032] S110, obtaining a vehicle environment video of the current vehicle, and determining candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video.

[0033] Among them, the current vehicle refers to the vehicle that needs to determine the abnormal area at the current moment. The current vehicle can be in a stationary state or a running state. The vehicle environment video refers to the surrounding environment video where the current vehicle is located. The embodiment of the present invention does not impose any restrictions on the method of obtaining the vehicle environment video, which can be set by a technician based on experience. Exemplarily, it can be obtained through a camera installed on the outside of the current vehicle, and it is only necessary to ensure that the camera covers the entire surrounding area of ​​the current vehicle. The sensor installed on the outside of the current vehicle can identify the gaze direction (which may include the gaze angle and the gaze starting point) and the gesture direction (which may include the gesture angle and the gesture starting point) of the pedestrian in each frame image of the vehicle environment video. The sensor can cover the entire surrounding area of ​​the current vehicle. The sensor may include a gesture recognition sensor and a line of sight recognition sensor. The embodiment of the present invention does not impose any restrictions on the time length of the vehicle environment video, which can be set by a technician based on experience.

[0034] The initial vehicle environment image refers to any frame image in the vehicle environment video. Exemplarily, the initial vehicle environment image may be the first frame image in the vehicle environment video. The candidate pedestrian refers to the pedestrian in the initial vehicle environment image. The candidate pedestrian may be understood as a person outside the current vehicle.

[0035] The candidate gaze focus refers to the focus point of the candidate pedestrian's line of sight, and the candidate gesture focus refers to the indication focus point of the candidate pedestrian's hand.

[0036] Exemplarily, the initial vehicle environment image includes at least one candidate person. For any candidate person, the gaze direction of the candidate person is determined from the initial vehicle environment image. Based on the facial feature point detection algorithm (such as OpenFace), the gaze direction is analyzed and calculated according to the area where the current vehicle is located, and the candidate gaze focus of the candidate person is determined; the gesture direction of the candidate person is determined from the initial vehicle environment image. Based on the gesture recognition algorithm (such as OpenPose), the gesture direction is analyzed and calculated according to the area where the current vehicle is located, and the candidate gesture focus of the candidate person is determined. Among them, the area where the vehicle is located refers to the area where the current vehicle is located. The embodiment of the present invention does not specifically limit the method for determining the area where the vehicle is located, which can be set by technical personnel based on experience. Exemplarily, it can be determined by a high-precision map.

[0037] For example, according to the gaze starting point and gaze angle in the gaze direction of the candidate, a gaze prediction extension line is made, and the intersection of the gaze prediction extension line and the area where the vehicle is located is used as the candidate gaze focus of the candidate. Correspondingly, according to the gesture starting point and gesture angle in the gesture direction of the candidate, a gesture prediction extension line is made, and the intersection of the gesture prediction extension line and the area where the vehicle is located is used as the candidate gesture focus of the candidate. Among them, the gaze prediction extension line refers to the virtual line of sight of the candidate predicted by the facial feature detection algorithm. The gesture prediction extension line refers to the virtual direction of the candidate predicted by the gesture recognition algorithm.

[0038] It should be noted that a candidate person should correspond to a candidate gaze focus and a candidate gesture focus, that is, a candidate person has a candidate gaze focus and a candidate gesture focus; the candidate person with only one candidate gaze focus or only one candidate gesture focus will be deleted to achieve the update of the candidate person.

[0039] S120. Select a target gaze focus and a target gesture focus from the candidate gaze focuses and the candidate gesture focuses according to the candidate gaze focuses, the candidate gesture focuses, the vehicle boundary interval and the preset focus distance threshold, and select the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as the target pedestrians.

[0040] The vehicle boundary interval refers to a predetermined boundary range of the current vehicle. The embodiment of the present invention does not impose any limitation on the value of the preset focal distance threshold, which can be set by technicians based on experience or repeatedly determined through a large number of experiments.

[0041] The target gaze focus refers to the candidate gaze focus on the current vehicle surface. The target gesture focus refers to the candidate gesture focus on the current vehicle surface. The target pedestrian refers to the candidate pedestrian corresponding to the target gaze focus and the target gesture focus.

[0042] S130: Determine the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determine the abnormal area center of the current vehicle according to the target gaze focus and target gesture focus of each target pedestrian.

[0043] The number of pedestrians refers to the number of target pedestrians. The embodiment of the present invention does not impose any limitation on the size of the preset pedestrian number threshold, which can be set by a technician based on experience or repeatedly determined through a large number of experiments. For example, the preset pedestrian number threshold can be 2. The center of the abnormal area refers to the center point of the vehicle abnormal area.

[0044] In an optional embodiment, the center of the abnormal area of ​​the current vehicle is determined based on the target gaze focus and target gesture focus of each target pedestrian, including: determining the gaze weight of the target gaze focus, and determining the gesture weight of the target gesture focus; determining the center of the abnormal area of ​​the current vehicle based on the gaze weight, the target gaze focus, the gesture weight and the target gesture focus.

[0045] Among them, the gaze weight can be used to quantify the importance of the target gaze focus. The gesture weight can be used to quantify the importance of the target gesture focus. The embodiment of the present invention does not impose any restrictions on the size of the gaze weight and / or gesture weight, which can be set by a technician based on experience, or repeatedly determined through a large number of experiments. It can be understood that the gaze weight and gesture weight can be used to balance the relative importance of the target gaze focus and the target gesture focus in determining the center of the abnormal area of ​​the current vehicle.

[0046] Exemplarily, the center of the abnormal area can be determined by the following formula:

[0047]

[0048] Where M represents the center of the abnormal area; n represents the number of target pedestrians; i represents the i-th target pedestrian; w i represents the attention weight; Z i Indicates the target's gaze focus; w i ' represents the gesture weight; S i Indicates the target gesture focus.

[0049] It can be understood that by introducing gaze weights and gesture weights, the center of the abnormal area of ​​the current vehicle is determined by gaze weights, target gaze focus, gesture weights and target gesture focus, thereby improving the accuracy of the determined center of the abnormal area.

[0050] It should be noted that if the number of target pedestrians in the initial vehicle environment image is less than a preset pedestrian number threshold, any frame image other than the initial vehicle environment image is selected from the vehicle environment video as a new initial vehicle environment image.

[0051] S140, based on the vehicle environment video, determine the candidate gaze time and candidate sight angle of each target pedestrian, and determine the target gaze duration based on each candidate gaze time, and determine the target sight angle based on each candidate sight angle.

[0052] The candidate gaze time refers to the time when the target pedestrian gazes at the current vehicle. Exemplarily, the candidate gaze time may include a candidate gaze start time and a candidate gaze end time. The candidate gaze start time refers to the time point when the target pedestrian starts to gaze at the current vehicle; correspondingly, the candidate gaze end time refers to the time point when the target pedestrian stops looking at the current vehicle. The candidate sight angle refers to the angle between the gaze foci of any two target pedestrians.

[0053] The target gaze duration refers to the gaze duration that can be used to determine the radius of the abnormal area. The target sight angle refers to the candidate sight angle that can be used to determine the radius of the abnormal area.

[0054] In an optional embodiment, the candidate gaze time of each target pedestrian is determined based on the vehicle environment video, including: tracking and monitoring the target pedestrian based on the vehicle environment video, and determining the reference gaze focus of each target pedestrian in each reference vehicle environment image in the vehicle environment video; judging whether each reference gaze focus is within a preset gaze focus interval, and determining the candidate gaze time of each target pedestrian based on the judgment result.

[0055] The reference vehicle environment image refers to other frame images in the vehicle environment video except the initial vehicle environment image. The reference gaze focus refers to the gaze focus of the target pedestrian determined according to the reference vehicle environment image.

[0056] The preset focus interval can be understood as a deviation interval of the reference focus. For example, with the target focus as a reference, a certain range around the target focus is used as the preset focus interval. The embodiment of the present invention does not impose any limitation on the size of the certain range, which can be set by technicians based on experience.

[0057] Exemplarily, whether each reference gaze focus is within a preset gaze focus interval is judged, and based on the judgment result, the candidate gaze time of each target pedestrian is determined, including: for any reference gaze focus, if the reference gaze focus is within the preset gaze focus interval, then the reference vehicle environment image where the reference gaze focus is located is valid; if the reference gaze focus is not within the preset gaze focus interval, then the reference vehicle environment image where the reference gaze focus is located is invalid; based on the time point corresponding to the initial vehicle environment image and the time point of the valid reference vehicle environment image, the candidate gaze time of the target pedestrian is determined.

[0058] It should be noted that if any reference vehicle environment image is invalid for any reference gaze focus, then the reference vehicle environment image may be valid for another reference gaze focus; that is, the valid reference vehicle environment images corresponding to different target persons may be different.

[0059] In an embodiment of the present invention, gaze time (or staring time) is a key indicator for evaluating the importance of an object or event. The longer the gaze time, the higher the pedestrian's attention to the location, and the greater the possibility of abnormality. In addition, a longer gaze time usually indicates that the pedestrian is more certain about the location of concern. Pedestrians confirm that there is an abnormality at the location by staring for a long time. In an embodiment of the present invention, if a person outside the vehicle stares at a certain location for a long time, it means that the abnormality at the location deserves in-depth investigation. Therefore, the longer the gaze time, the radius of the abnormal area should be increased accordingly to ensure comprehensive detection of abnormalities.

[0060] It can be understood that by tracking and monitoring the target pedestrian based on the vehicle environment video, the candidate gaze time of the target pedestrian is determined, thereby improving the accuracy of the determined candidate gaze time; at the same time, in this scheme, by introducing a preset gaze focus interval, the reference gaze focus is filtered, thereby avoiding the influence of invalid reference gaze focus on the determination of the candidate gaze time, thereby improving the accuracy of the determined candidate gaze time.

[0061] In an optional embodiment, determining the target gaze duration based on each candidate gaze time includes: determining the gaze intersection time of the target pedestrian according to the candidate gaze time of each target pedestrian; and determining the target gaze duration according to the gaze intersection time.

[0062] Among them, the gaze intersection time refers to the time that each target pedestrian gazes at the current vehicle together. The target gaze duration refers to the time that the target pedestrian gazes at the current vehicle together.

[0063] For example, if the candidate gaze time of target person A is from 1s to 5s, the candidate gaze time of target person B is from 2s to 4s, and the candidate gaze time of target person C is from 3s to 7s, then the gaze intersection time is from 3s to 4s, and the target gaze duration is 1s.

[0064] It can be understood that by introducing the gaze intersection time to determine the target gaze duration, the accuracy of the determined target gaze duration is improved.

[0065] In an optional embodiment, the candidate sight angles of each target pedestrian are determined, including: determining an intersection vehicle environment image from a vehicle environment video based on the gaze intersection time; determining the candidate sight angles between the target pedestrians in the intersection vehicle environment image based on the intersection gaze focus of each target pedestrian in the intersection vehicle environment image; accordingly, determining the target sight angle based on each candidate sight angle, including: comparing the candidate sight angles, and taking the candidate sight angle with the largest value as the target sight angle.

[0066] The intersection vehicle environment image refers to an image determined from the vehicle environment video according to the gaze intersection time. Exemplarily, the intersection vehicle environment image may include an initial vehicle environment image and / or a reference vehicle environment image.

[0067] The intersection gaze focus refers to the gaze focus of each target pedestrian in the intersection vehicle environment image. Exemplarily, the intersection gaze focus may include a target gaze focus and / or a reference gaze focus.

[0068] In an embodiment of the present invention, the sight angle can reflect the position distribution information of multiple pedestrians. By calculating the sight angle between pedestrians, their degree of attention to the same position can be evaluated. The larger the angle, the more dispersed the pedestrian distribution is, the higher the independence is, and each confirms that there is an abnormality at the position from a different angle. When the sight angle is large, it means that pedestrians from different positions are paying attention to the same position, and the result of this independent verification is more convincing. In practical applications, by calculating the sight angle of multiple pedestrians, it can be evaluated whether their gazes are concentrated at the same position. The larger the sight angle, the more pedestrians at different positions have noticed this information, and the greater the possibility of an abnormality. Therefore, the larger the sight angle, the larger the radius of the abnormal area needs to be set.

[0069] It can be understood that by introducing the intersection vehicle environment image and the intersection gaze focus to determine the candidate sight line angle, the accuracy of the determined candidate sight line angle is improved, thereby improving the accuracy of the determined target sight line angle.

[0070] S150: determining the radius of the abnormal area according to the number of pedestrians, the target gaze duration, and the target sight angle, and determining the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center.

[0071] The abnormal area radius refers to the radius value of the vehicle abnormal area. The vehicle abnormal area refers to the area where there is a fault or abnormality in the current vehicle.

[0072] In an embodiment of the present invention, more pedestrians' gazes and gestures point to the same location, which usually means that there is a higher possibility of abnormality in this location. This collective attention can be regarded as a manifestation of "collective wisdom", and the conclusions independently drawn by multiple pedestrians are often more reliable. In practical applications, if multiple people outside the vehicle look at or point to a certain location at the same time, it means that the abnormal situation at this location deserves high attention. Therefore, the more people there are, the larger the radius of the abnormal area needs to be set to ensure that all possible abnormal points are covered.

[0073] Exemplarily, the radius of the abnormal area is determined based on the number of pedestrians, the target gaze duration and the target sight angle, including: determining the pedestrian weight of the number of pedestrians, determining the duration weight of the target gaze duration, and determining the angle weight of the target sight angle; determining the radius of the abnormal area based on the pedestrian weight, duration weight, angle weight, the number of pedestrians, the target gaze duration and the target sight angle.

[0074] Among them, the pedestrian weight can be used to quantify the importance of the number of pedestrians in determining the radius of the abnormal area. The duration weight can be used to quantify the importance of the target gaze duration in determining the radius of the abnormal area. The angle weight can be used to quantify the importance of the target line of sight angle in determining the radius of the abnormal area. It can be understood that the pedestrian weight, duration weight and angle weight can be used to balance the relative importance of the number of pedestrians, target gaze duration and target line of sight angle in determining the radius of the abnormal area. The embodiment of the present invention does not impose any limitation on the size of the pedestrian weight, duration weight and angle weight, which can be set by technicians based on experience or needs, or determined repeatedly through a large number of experiments.

[0075] For example, the radius of the abnormal area can be determined by the following formula:

[0076] R=αN+βT+γ·θ max ;

[0077] Among them, R represents the radius of the abnormal area; N represents the number of pedestrians; T represents the target gaze duration; θ max represents the target sight angle; α represents the pedestrian weight; β represents the duration weight; γ represents the angle weight.

[0078] An embodiment of the present invention provides a vehicle abnormal area determination solution, which obtains a vehicle environment video of a current vehicle, and determines candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video; selects a target gaze focus and a target gesture focus from the candidate gaze focuses and candidate gesture focuses according to the candidate gaze focuses, the candidate gesture focuses, the vehicle boundary interval and a preset focus distance threshold, and uses the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as target pedestrians; determines the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determines the abnormal area center of the current vehicle according to the target gaze focus and the target gesture focus of each target pedestrian; determines the candidate gaze time and the candidate sight angle of each target pedestrian based on the vehicle environment video, and determines the target gaze time based on each candidate gaze time, and determines the target sight angle based on each candidate sight angle; determines the abnormal area radius according to the number of pedestrians, the target gaze time and the target sight angle, and determines the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center. The above scheme determines the abnormal area of ​​the vehicle by introducing candidate gaze focus and candidate gesture focus of candidate pedestrians, realizes the determination of the abnormal area of ​​the vehicle based on the recognition of the movements of pedestrians around the vehicle, realizes the rapid positioning of the abnormal area of ​​the vehicle, and improves the efficiency of determining the abnormal area of ​​the vehicle; at the same time, by introducing a preset pedestrian number threshold, it realizes the use of target gaze focus and target gesture focus of multiple people, and comprehensively considers the factors of target gaze duration and target line of sight angle, thereby improving the accuracy and reliability of the determined abnormal area of ​​the vehicle; and by analyzing the behavioral data of people outside the vehicle, the human-computer interaction capability is enhanced.

[0079] On the basis of the above technical solution, the embodiment of the present invention can determine whether the vehicle has an abnormality based on the abnormal area of ​​the vehicle in the following ways. For example, the image of the determined abnormal area of ​​the vehicle can be displayed on a display screen that the driver can see, and the driver can make a judgment on whether it is abnormal by careful observation; or, the image of the abnormal area of ​​the vehicle is sent to the vehicle abnormality judgment model, and the model obtains the result of abnormality through analysis; or, the abnormal area of ​​the vehicle is detected in detail using vehicle diagnostic tools and sensors to confirm whether there is a fault. Among them, the vehicle abnormality judgment model is used to determine whether there is an abnormality or fault in the abnormal area of ​​the vehicle. The vehicle abnormality judgment model can be a deep learning network model.

[0080] Embodiment 2

[0081] Figure 2It is a flow chart of a method for determining an abnormal area of ​​a vehicle provided in the second embodiment of the present invention. Based on the above embodiments, this embodiment further refines the operation of "selecting a target gaze focus and a target gesture focus from the candidate gaze focus and the candidate gesture focus according to the candidate gaze focus, the candidate gesture focus, the vehicle boundary interval and the preset focus distance threshold" into "for any candidate pedestrian, determine whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are both within the vehicle boundary interval; if so, determine the candidate focus distance of the candidate pedestrian according to the candidate gaze focus and the candidate gesture focus; determine whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are the target gaze focus and the target gesture focus according to the candidate focus distance and the preset focus distance threshold" to improve the target focus determination mechanism. It should be noted that for the parts not described in detail in the embodiments of the present invention, reference may be made to the descriptions of other embodiments.

[0082] See also Figure 2 The method for determining an abnormal area of ​​a vehicle shown includes:

[0083] S210: Acquire a vehicle environment video of the current vehicle, and determine candidate gaze foci and candidate gesture foci of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video.

[0084] S220: For any candidate pedestrian, determine whether a candidate gaze focus and a candidate gesture focus of the candidate pedestrian are both within a vehicle boundary interval.

[0085] Exemplarily, the vehicle boundary interval may include a first direction boundary interval and a second direction boundary interval. Among them, the first direction boundary interval may be an x-direction boundary interval; the second direction boundary interval may be a y-direction boundary interval. The embodiment of the present invention does not impose any limitation on the method of determining the vehicle boundary interval, which may be set by a technician based on experience. Exemplarily, the vehicle boundary interval may be determined based on the vehicle three-dimensional model of the current vehicle and the position of the current vehicle based on a high-precision map, and the vehicle boundary interval may be determined based on the vehicle three-dimensional model and the position of the current vehicle.

[0086] For example, if the candidate's candidate focus is (x g ,y g ), the candidate gesture focus is (x h ,y h ), the first direction boundary interval is [x min ,x max ], the second direction boundary interval is [y min ,y max ], if and only if x g In [x min ,x max ], x h In [xmin ,x max ], and y g In [y min ,y max ] within, y h In [y min ,y max ], determine the candidate focal distance of the candidate pedestrian.

[0087] S230: If yes, determine the candidate focus distance of the candidate pedestrian according to the candidate gaze focus and the candidate gesture focus.

[0088] The candidate focus distance refers to the distance between the candidate gaze focus and the candidate gesture focus. Exemplarily, for any candidate pedestrian, the candidate focus distance between the candidate gaze focus and the candidate gesture focus of the candidate pedestrian is determined.

[0089] Exemplarily, the candidate focal distance may be determined by the following formula:

[0090]

[0091] Where L represents the candidate focal distance; x g Indicates the first direction coordinate of the candidate gaze focus; x h Indicates the first direction coordinate of the candidate gesture focus; y g The second direction coordinate of the candidate gaze focus; y h Indicates the second direction coordinate of the candidate gesture focus.

[0092] S240: Determine whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are the target gaze focus and the target gesture focus according to the candidate focus distance and the preset focus distance threshold.

[0093] The embodiment of the present invention does not impose any limitation on the size of the preset focal distance threshold, which can be set by a technician based on experience or repeatedly determined through a large number of experiments.

[0094] In an optional embodiment, based on the candidate focus distance and the preset focus distance threshold, it is determined whether the candidate gaze focus and candidate gesture focus of the candidate pedestrian are the target gaze focus and target gesture focus, including: if the candidate focus distance is less than or equal to the preset focus distance threshold, then the candidate gaze focus and candidate gesture focus of the candidate pedestrian are determined to be the target gaze focus and target gesture focus; if the candidate focus distance is greater than the preset focus distance threshold, then the candidate gaze focus and candidate gesture focus of the candidate pedestrian are prohibited from being used as the target gaze focus and target gesture focus.

[0095] It can be understood that by comparing the candidate focus distance with the preset focus distance threshold to determine the target gaze focus and the target gesture focus, the accuracy of the determined target gaze focus and the target gesture focus is improved.

[0096] S250: taking the candidate pedestrian corresponding to the target gaze focus and the target gesture focus as the target pedestrian.

[0097] S260: Determine the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determine the abnormal area center of the current vehicle according to the target gaze focus and target gesture focus of each target pedestrian.

[0098] S270: Determine candidate gaze times and candidate sight angles for each target pedestrian based on the vehicle environment video, determine the target gaze duration based on each candidate gaze time, and determine the target sight angle based on each candidate sight angle.

[0099] S280: Determine the radius of the abnormal area according to the number of pedestrians, the target gaze duration, and the target sight angle, and determine the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center.

[0100] The embodiment of the present invention provides a vehicle abnormal area determination scheme, which selects the target gaze focus and target gesture focus operations from the candidate gaze focus and candidate gesture focus according to the candidate gaze focus, candidate gesture focus, vehicle boundary interval and preset focus distance threshold, and refines it into, for any candidate pedestrian, determining whether the candidate gaze focus and candidate gesture focus of the candidate pedestrian are both within the vehicle boundary interval; if so, determining the candidate focus distance of the candidate pedestrian according to the candidate gaze focus and candidate gesture focus; determining whether the candidate gaze focus and candidate gesture focus of the candidate pedestrian are the target gaze focus and target gesture focus according to the candidate focus distance and the preset focus distance threshold, thereby improving the target focus determination mechanism. The above scheme improves the accuracy of the determined target gaze focus and target gesture focus by introducing the candidate focus distance, comparing the candidate focus distance with the preset focus distance threshold, and determining the target gaze focus and target gesture focus.

[0101] During vehicle detection and diagnosis, the gaze focus and gesture focus of people outside the vehicle (such as pedestrians or bystanders) can provide important indication information. Traditional methods usually rely on in-vehicle sensors and diagnostic tools, while ignoring the gaze and gesture information of people outside the vehicle. By combining this information, abnormal vehicle conditions can be more accurately located and diagnosed.

[0102] The vehicle abnormal area determination scheme provided by the embodiment of the present invention can be implemented by a vehicle detection system. This scheme realizes the rapid positioning of the vehicle abnormal area, that is, by analyzing the target gaze focus and target gesture focus of the person outside the vehicle (i.e., the target pedestrian), the possible vehicle abnormal area is quickly determined; the detection accuracy is improved, that is, the line of sight and gesture information of multiple people outside the vehicle are combined to reduce false alarms and improve the detection accuracy; the distance and time factors are comprehensively considered, that is, the key focus area is determined through the comprehensive analysis of the line of sight angle and the gaze time.

[0103] On the basis of the above technical solution, the vehicle abnormal area determination method provided in the embodiment of the present invention can be applied to the scenario of determining whether the tires of the vehicle are abnormal or faulty.

[0104] Embodiment 3

[0105] Figure 3 This is a schematic diagram of the structure of a vehicle abnormal area determination device provided in Embodiment 3 of the present invention. This embodiment is applicable to the case of determining an abnormal area in a vehicle, and the method can be performed by a vehicle abnormal area determination device, which can be implemented in software and / or hardware, and can be configured in an electronic device that carries the vehicle abnormal area determination function.

[0106] like Figure 3 As shown, the device includes: a focus determination module 310, a target pedestrian determination module 320, an abnormal area center determination module 330, a target data determination module 340 and an abnormal area determination module 350. Among them,

[0107] A focus determination module 310 is used to obtain a vehicle environment video of a current vehicle, and determine a candidate gaze focus and a candidate gesture focus of a candidate pedestrian around the current vehicle from an initial vehicle environment image in the vehicle environment video;

[0108] a target pedestrian determination module 320, configured to select a target gaze focus and a target gesture focus from the candidate gaze focuses and the candidate gesture focuses according to the candidate gaze focuses, the candidate gesture focuses, the vehicle boundary interval and a preset focus distance threshold, and select the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as the target pedestrian;

[0109] The abnormal area center determination module 330 is used to determine the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determine the abnormal area center of the current vehicle according to the target gaze focus and target gesture focus of each target pedestrian;

[0110] A target data determination module 340 is used to determine candidate gaze times and candidate sight angles of each target pedestrian based on the vehicle environment video, and to determine a target gaze duration based on each candidate gaze time, and to determine a target sight angle based on each candidate sight angle;

[0111] The abnormal area determination module 350 is used to determine the abnormal area radius according to the number of pedestrians, the target gaze duration and the target sight angle, and determine the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center.

[0112] An embodiment of the present invention provides a vehicle abnormal area determination solution, which obtains a vehicle environment video of a current vehicle, and determines candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video; selects a target gaze focus and a target gesture focus from the candidate gaze focuses and candidate gesture focuses according to the candidate gaze focuses, the candidate gesture focuses, the vehicle boundary interval and a preset focus distance threshold, and uses the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as target pedestrians; determines the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determines the abnormal area center of the current vehicle according to the target gaze focus and the target gesture focus of each target pedestrian; determines the candidate gaze time and the candidate sight angle of each target pedestrian based on the vehicle environment video, and determines the target gaze time based on each candidate gaze time, and determines the target sight angle based on each candidate sight angle; determines the abnormal area radius according to the number of pedestrians, the target gaze time and the target sight angle, and determines the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center. The above scheme determines the abnormal area of ​​the vehicle by introducing candidate gaze focus and candidate gesture focus of candidate pedestrians, realizes the determination of the abnormal area of ​​the vehicle based on the recognition of the movements of pedestrians around the vehicle, realizes the rapid positioning of the abnormal area of ​​the vehicle, and improves the efficiency of determining the abnormal area of ​​the vehicle; at the same time, by introducing a preset pedestrian number threshold, it realizes the use of target gaze focus and target gesture focus of multiple people, and comprehensively considers the factors of target gaze duration and target line of sight angle, thereby improving the accuracy and reliability of the determined abnormal area of ​​the vehicle; and by analyzing the behavioral data of people outside the vehicle, the human-computer interaction capability is enhanced.

[0113] Optionally, the target pedestrian determination module 320 includes:

[0114] a focus determination unit, configured to determine, for any candidate pedestrian, whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are both within the vehicle boundary interval;

[0115] a candidate focus distance determining unit, configured to determine a candidate focus distance of the candidate pedestrian according to the candidate gaze focus and the candidate gesture focus, if any;

[0116] A target focus determination unit is used to determine whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are the target gaze focus and the target gesture focus according to the candidate focus distance and the preset focus distance threshold.

[0117] Optionally, the target focus determination unit is specifically used for:

[0118] If the candidate focus distance is less than or equal to the preset focus distance threshold, determining the candidate gaze focus and the candidate gesture focus of the candidate pedestrian as the target gaze focus and the target gesture focus;

[0119] If the candidate focus distance is greater than the preset focus distance threshold, it is prohibited to use the candidate gaze focus and the candidate gesture focus of the candidate pedestrian as the target gaze focus and the target gesture focus.

[0120] Optionally, the abnormal region center determination module 330 includes:

[0121] a weight determination unit, configured to determine a gaze weight of the target gaze focus and a gesture weight of the target gesture focus;

[0122] The abnormal area center determination unit is used to determine the abnormal area center of the current vehicle according to the gaze weight, the target gaze focus, the gesture weight and the target gesture focus.

[0123] Optionally, the target data determination module 340 includes:

[0124] A reference gaze focus determination unit, configured to track and monitor the target pedestrian based on the vehicle environment video, and determine a reference gaze focus of each target pedestrian in each reference vehicle environment image in the vehicle environment video;

[0125] The candidate gaze time determination unit is used to determine whether each of the reference gaze focuses is within a preset gaze focus interval, and determine the candidate gaze time of each of the target pedestrians according to the determination result.

[0126] Optionally, the target data determination module 340 includes:

[0127] A gaze intersection time determination unit, configured to determine a gaze intersection time of the target pedestrian according to the candidate gaze times of each of the target pedestrians;

[0128] The target gaze duration determining unit is used to determine the target gaze duration according to the gaze intersection time.

[0129] Optionally, the target data determination module 340 includes:

[0130] An intersection image determining unit, configured to determine an intersection vehicle environment image from the vehicle environment video according to the gaze intersection time;

[0131] A candidate sight angle determination unit, configured to determine a candidate sight angle between target pedestrians in the intersection vehicle environment image according to an intersection gaze focus of each target pedestrian in the intersection vehicle environment image;

[0132] Accordingly, the target data determination module 340 further includes:

[0133] The target sight line angle determination unit is used to compare the candidate sight line angles and take the largest candidate sight line angle as the target sight line angle.

[0134] The vehicle abnormal area determination device provided in the embodiment of the present invention can execute the vehicle abnormal area determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing each vehicle abnormal area determination method.

[0135] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of vehicle environment videos, vehicle boundary intervals and preset focal distance thresholds are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0136] Embodiment 4

[0137] Figure 4 It is a structural schematic diagram of an electronic device for implementing a method for determining a vehicle abnormal area provided by Embodiment 4 of the present invention. Electronic device 410 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0138] like Figure 4As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411 in communication, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0139] Multiple components in the electronic device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0140] The processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 411 executes the various methods and processes described above, such as a vehicle abnormal area determination method.

[0141] In some embodiments, the vehicle abnormal area determination method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the vehicle abnormal area determination method described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured to execute the vehicle abnormal area determination method in any other appropriate manner (e.g., by means of firmware).

[0142] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0144] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0147] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0149] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining an abnormal area of ​​a vehicle, characterized in that: include: Acquire a vehicle environment video of a current vehicle, and determine candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video; According to the candidate gaze focus, the candidate gesture focus, the vehicle boundary interval and the preset focus distance threshold, selecting a target gaze focus and a target gesture focus from the candidate gaze focus and the candidate gesture focus, and taking the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as the target pedestrian; Determine the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determine the center of the abnormal area of ​​the current vehicle according to the target gaze focus and the target gesture focus of each target pedestrian; Determine candidate gaze times and candidate sight angles of each target pedestrian based on the vehicle environment video, determine target gaze duration based on each candidate gaze time, and determine target sight angle based on each candidate sight angle; The radius of the abnormal area is determined according to the number of pedestrians, the target gaze duration and the target sight angle, and the vehicle abnormal area of ​​the current vehicle is determined based on the radius of the abnormal area and the center of the abnormal area.

2. The method according to claim 1, characterized in that The selecting a target gaze focus and a target gesture focus from the candidate gaze focus and the candidate gesture focus according to the candidate gaze focus, the candidate gesture focus, the vehicle boundary interval and the preset focus distance threshold comprises: For any candidate pedestrian, determining whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are both within the vehicle boundary interval; If yes, determining a candidate focus distance of the candidate pedestrian according to the candidate gaze focus and the candidate gesture focus; According to the candidate focus distance and the preset focus distance threshold, it is determined whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are the target gaze focus and the target gesture focus.

3. The method according to claim 2, characterized in that The determining, according to the candidate focus distance and the preset focus distance threshold, whether the candidate gaze focus and the candidate gesture focus of the candidate pedestrian are target gaze focus and target gesture focus, comprises: If the candidate focus distance is less than or equal to the preset focus distance threshold, determining the candidate gaze focus and the candidate gesture focus of the candidate pedestrian as the target gaze focus and the target gesture focus; If the candidate focus distance is greater than the preset focus distance threshold, it is prohibited to use the candidate gaze focus and the candidate gesture focus of the candidate pedestrian as the target gaze focus and the target gesture focus.

4. The method according to claim 1, characterized in that The determining the abnormal area center of the current vehicle according to the target gaze focus and the target gesture focus of each target pedestrian includes: Determining a gaze weight of the target gaze focus, and determining a gesture weight of the target gesture focus; The abnormal area center of the current vehicle is determined according to the gaze weight, the target gaze focus, the gesture weight and the target gesture focus.

5. The method according to claim 1, characterized in that The determining, based on the vehicle environment video, the candidate gaze time of each target pedestrian includes: Based on the vehicle environment video, the target pedestrian is tracked and monitored to determine a reference gaze focus of each target pedestrian in each reference vehicle environment image in the vehicle environment video; It is determined whether each of the reference gaze focuses is within a preset gaze focus interval, and based on the determination result, a candidate gaze time of each of the target pedestrians is determined.

6. The method according to claim 5, characterized in that The determining the target gaze duration based on each of the candidate gaze times comprises: Determining the gaze intersection time of the target pedestrian according to the candidate gaze time of each target pedestrian; The target gaze duration is determined according to the gaze intersection time.

7. The method according to claim 6, characterized in that The determining of the candidate sight line angles of each target pedestrian includes: determining an intersection vehicle environment image from the vehicle environment video according to the gaze intersection time; Determining candidate sight angles between target pedestrians in the intersection vehicle environment image according to intersection gaze focuses of target pedestrians in the intersection vehicle environment image; Accordingly, determining the target sight angle based on each of the candidate sight angles includes: The candidate sight line angles are compared, and the largest candidate sight line angle is used as the target sight line angle.

8. A vehicle abnormal area determination device, characterized in that: include: A focus determination module, used to obtain a vehicle environment video of a current vehicle, and determine candidate gaze focuses and candidate gesture focuses of candidate pedestrians around the current vehicle from an initial vehicle environment image in the vehicle environment video; a target pedestrian determination module, configured to select a target gaze focus and a target gesture focus from the candidate gaze focuses and the candidate gesture focuses according to the candidate gaze focuses, the candidate gesture focuses, the vehicle boundary interval and a preset focus distance threshold, and select the candidate pedestrians corresponding to the target gaze focus and the target gesture focus as the target pedestrian; an abnormal area center determination module, used to determine the number of target pedestrians, and when the number of pedestrians reaches a preset pedestrian number threshold, determine the abnormal area center of the current vehicle according to the target gaze focus and target gesture focus of each target pedestrian; A target data determination module, for determining candidate gaze times and candidate sight angles of each target pedestrian based on the vehicle environment video, and determining a target gaze duration based on each candidate gaze time, and determining a target sight angle based on each candidate sight angle; The abnormal area determination module is used to determine the radius of the abnormal area according to the number of pedestrians, the target gaze duration and the target sight angle, and determine the vehicle abnormal area of ​​the current vehicle based on the abnormal area radius and the abnormal area center.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a vehicle abnormal area determination method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for determining an abnormal area of ​​a vehicle as described in any one of claims 1 to 7 is implemented.

Images