Method and apparatus for detecting failure to yield to pedestrians of vehicle

Abstract

The present disclosure relates to the technical field of artificial intelligence such as intelligent transportation and computer vision, and provides a method and apparatus for detecting failure to yield to pedestrians of a vehicle, an electronic device, and a readable storage medium. The method for detecting failure to yield to pedestrians of a vehicle comprises: acquiring a marked crosswalk area on the basis of surveillance images in a surveillance video stream; for each surveillance image frame in the surveillance video stream, acquiring the intersection over union between a target vehicle and the marked crosswalk area in the surveillance image frame, and selecting multiple surveillance image frames having intersection over union greater than a preset intersection over union threshold as target surveillance images corresponding to the target vehicle; and sequentially performing human detection on each target surveillance image frame in descending order of intersection over union, and when it is determined that a human detection result corresponding to a current target surveillance image satisfies a preset requirement, determining that the target vehicle is a regulation-violating vehicle which does not yield to pedestrians. The present disclosure can expand the detection scenario, reduce the computing resources required for detection, and improve the detection efficiency while detecting whether vehicles yield to pedestrians.

Description

Method and device for detecting vehicles that fail to yield to pedestrians

[0001] This application claims priority to a Chinese patent application filed on November 21, 2023, with application number 202311558128.5 and titled “Method and device for detecting vehicles failing to give way to pedestrians.” Technical Field

[0002] The present disclosure relates to the field of computer technology, and in particular to artificial intelligence technologies such as intelligent transportation and computer vision. A method, device, electronic device, and readable storage medium are provided for detecting vehicles that fail to yield to pedestrians. Background Art

[0003] As people's living standards continue to improve, the number of motor vehicles in cities is also increasing. The conflict between people, vehicles, and roads is becoming increasingly prominent, and the need for civilized participation in traffic is becoming increasingly urgent. At zebra crossings, pedestrians have the right of way, and motor vehicles are required to stop and yield to pedestrians. However, many vehicles do not slow down or yield at zebra crossings, and the competition between vehicles and pedestrians can easily lead to traffic accidents.

[0004] Existing technologies typically detect whether a vehicle is yielding to a pedestrian based on whether the pedestrian's trajectory intersects with the vehicle's. However, this technology requires capturing the vehicle's trajectory based on the vehicle's front angle, which places certain demands on the monitoring angle. Furthermore, capturing the complete pedestrian's trajectory is difficult at night and at long distances, which can easily lead to detection failures.

[0005] Summary of the Invention

[0006] According to a first aspect of the present disclosure, a method for detecting a vehicle that does not give way to pedestrians is provided, comprising: obtaining a zebra crossing area based on a surveillance image in a surveillance video stream; obtaining, for each surveillance image frame in the surveillance video stream, an intersection-and-union (IoU) between a target vehicle in the surveillance image frame and the zebra crossing area, and selecting a plurality of surveillance images whose IoUs are greater than a preset IoU threshold as target surveillance images corresponding to the target vehicle; performing human body detection on each target surveillance image frame in descending order of IoUs, and determining that the target vehicle is an illegal vehicle that does not give way to pedestrians upon determining that the human body detection result corresponding to the current target surveillance image meets preset requirements.

[0007] According to a second aspect of the present disclosure, a device for detecting vehicles that do not give way to pedestrians is provided, comprising: an acquisition unit for acquiring a zebra crossing area based on a surveillance image in a surveillance video stream; a processing unit for acquiring, for each surveillance image frame in the surveillance video stream, an intersection-and-union ratio between a target vehicle and the zebra crossing area in the surveillance image frame, and selecting a plurality of surveillance images whose intersection-and-union ratios are greater than a preset intersection-and-union ratio threshold as target surveillance images corresponding to the target vehicle; a detection unit for performing human body detection on each target surveillance image frame in descending order of the intersection-and-union ratios, and determining that the target vehicle is an illegal vehicle that does not give way to pedestrians when it is determined that the human body detection result corresponding to the current target surveillance image meets preset requirements.

[0008] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0009] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method as described above.

[0010] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described above when executed by a processor.

[0011] It can be seen from the above technical solutions that the present disclosure detects whether a vehicle gives way to pedestrians based on the zebra crossing area in the surveillance image, does not limit the surveillance angle of the surveillance video stream, and can expand the applicable scenarios when detecting whether a vehicle gives way to pedestrians. Moreover, the present disclosure only performs human body detection on the target surveillance image selected according to the intersection-union ratio to determine whether the target vehicle violates the rules. There is no need to perform human body detection on each frame of the surveillance image in the surveillance video stream, which can reduce the computing resources required for detecting whether a vehicle gives way to pedestrians and improve the efficiency of detecting whether a vehicle gives way to pedestrians.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0014] FIG1 is a schematic diagram of a first embodiment of the present disclosure;

[0015] FIG2 is a schematic diagram of a second embodiment of the present disclosure;

[0016] FIG3 is a schematic diagram of a third embodiment of the present disclosure;

[0017] FIG4 is a block diagram of an electronic device for implementing the method for detecting a vehicle that fails to yield to pedestrians according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, and various details of the embodiments of the present disclosure are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and mechanisms are omitted in the following description.

[0019] FIG1 is a schematic diagram of a first embodiment of the present disclosure. As shown in FIG1 , the method for detecting a vehicle failing to yield to pedestrians in this embodiment specifically includes the following steps:

[0020] S101, obtaining a zebra crossing area according to a surveillance image in a surveillance video stream;

[0021] S102: for each monitoring image frame in the monitoring video stream, obtaining an intersection-and-union (IoU) ratio between the target vehicle and the zebra crossing area in the monitoring image frame, and selecting multiple monitoring images with IoU ratios greater than a preset IoU threshold as target monitoring images corresponding to the target vehicle;

[0022] S103. Perform human body detection on each frame of the target monitoring image in descending order of intersection-over-union ratio, and determine that the target vehicle is an illegal vehicle that does not give way to pedestrians when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements.

[0023] The method for detecting a vehicle that does not give way to pedestrians in this embodiment first selects multiple frames of target monitoring images from the monitoring video stream based on the obtained intersection-in-union ratio between the zebra crossing area and the target vehicle, and then performs human body detection on each frame of the target monitoring image in descending order of the intersection-in-union ratio. Finally, when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements, the target vehicle is determined to be an illegal vehicle. This embodiment detects whether a vehicle gives way to pedestrians based on the zebra crossing area in the monitoring image, does not limit the monitoring angle of the monitoring video stream, and can expand the applicable scenarios when detecting whether a vehicle gives way to pedestrians. Moreover, this embodiment only performs human body detection on the target monitoring image selected according to the intersection-in-union ratio to determine whether the target vehicle violates the rules. There is no need to perform human body detection on each frame of the monitoring image in the monitoring video stream, which can reduce the computing resources required for detecting whether a vehicle gives way to pedestrians and improve the efficiency of detecting whether a vehicle gives way to pedestrians.

[0024] The intersection over union (IoU) is the ratio of the intersection and union of two detection frames. Specifically, in this embodiment, the IoU between the target vehicle in the monitoring image and the zebra crossing area is the ratio of the intersection and union of the detection frame of the target vehicle in the monitoring image and the detection frame of the zebra crossing area in the monitoring image.

[0025] When executing S101 to obtain the zebra crossing area based on the surveillance image in the surveillance video stream, this embodiment can first select a frame of surveillance image from the multiple frames of surveillance image contained in the surveillance video stream, then perform zebra crossing detection on the frame of surveillance image, and finally obtain the zebra crossing area contained in the surveillance scene based on the detection result. The detection result in this embodiment can include a zebra crossing detection score and a zebra crossing detection frame; wherein the zebra crossing detection score is used to indicate the confidence that the area in the zebra crossing detection frame obtained by detection is a real zebra crossing, and the zebra crossing detection frame is a frame that surrounds the zebra crossing area in the surveillance image.

[0026] Since the detection of arbitrary-shaped quadrilaterals is more consistent with zebra crossings than rectangular detection, and the arbitrary-shaped quadrilaterals will not frame out redundant areas in the image and cause false detection of the edges of the zebra crossing areas, this embodiment can use the arbitrary-shaped quadrilateral detection model to perform zebra crossing detection on the surveillance image when executing S101, thereby obtaining the zebra crossing detection frame in the surveillance image and its corresponding detection score, that is, using the arbitrary-shaped quadrilateral frame to identify the zebra crossing area in the image.

[0027] When executing S101, this embodiment determines that the zebra crossing detection score is greater than or equal to the preset score threshold, and uses the area corresponding to the zebra crossing detection frame in the monitoring image as the zebra crossing area; if it determines that the zebra crossing detection score is less than the preset score threshold, this embodiment will obtain another frame of monitoring image from the monitoring video stream, continue to perform zebra crossing detection on the newly obtained monitoring image, and then obtain the zebra crossing area in the monitoring image based on the detection result.

[0028] When executing S101 to obtain the zebra crossing area based on the surveillance image in the surveillance video stream, this embodiment can also adopt the following method: obtain the surveillance image from the surveillance video stream according to the preset time interval, that is, this embodiment will continuously obtain the surveillance image according to the preset time interval; perform zebra crossing detection on the obtained surveillance image, and obtain the zebra crossing area based on the detection result.

[0029] That is to say, this embodiment continuously obtains monitoring images from the monitoring video stream at preset time intervals. Even if the monitoring camera rotates, the zebra crossing area will be obtained based on the monitoring images collected by the monitoring camera after the rotation, which can improve the accuracy of the obtained zebra crossing area.

[0030] After executing S101 to obtain the zebra crossing area, this embodiment executes S102 to obtain the intersection-and-union ratio between the target vehicle and the zebra crossing area in each frame of the monitoring image in the monitoring video stream, and selects multiple frames of monitoring images with an intersection-and-union ratio greater than a preset intersection-and-union ratio threshold as the target monitoring images corresponding to the target vehicle.

[0031] In this embodiment, the target vehicles are all vehicles (motor vehicles) included in each frame of monitoring image. One frame of monitoring image may include one target vehicle or multiple target vehicles.

[0032] If a frame of monitoring image contains multiple target vehicles, this embodiment will obtain the intersection-and-union ratio between the current target vehicle and the zebra crossing area in each monitoring image for each target vehicle when executing S102, and then select multiple frames of target monitoring images corresponding to the current target vehicle based on the obtained intersection-and-union ratio.

[0033] In this embodiment, when executing S102 to obtain the intersection over union (IoU) between the target vehicle and the zebra crossing area in each frame of surveillance image in the surveillance video stream, the following implementation method may be adopted: for each frame of surveillance image, vehicle detection is performed on the frame of surveillance image to obtain the vehicle area of ​​the target vehicle in the frame of surveillance image, where the vehicle area is the area in the surveillance image corresponding to the detection frame of the target vehicle, and the detection frame of the target vehicle is the frame surrounding the target vehicle in the surveillance image; based on the obtained vehicle area and zebra crossing area, the intersection over union (IoU) between the target vehicle and the zebra crossing area in the frame of surveillance image is obtained.

[0034] After executing S102 to obtain the IoU between the target vehicle and the zebra crossing area in each frame of the monitoring image, this embodiment selects multiple frames of monitoring images with IoU greater than a preset IoU threshold as target monitoring images corresponding to the target vehicle.

[0035] It can be understood that if there are multiple target vehicles, executing S102 in this embodiment will obtain multiple frames of target monitoring images corresponding to each target vehicle. For example, different target vehicles correspond to different image groups, and each image group contains multiple frames of target monitoring images corresponding to the current target vehicle.

[0036] The preset intersection-over-union ratio threshold in this embodiment may be 0, that is, when executing S102 , this embodiment may use all monitoring images when the target vehicle overlaps with the zebra crossing area as the multi-frame target monitoring images corresponding to the target vehicle.

[0037] The preset intersection-over-union ratio threshold in this embodiment can also be a value greater than 0 and less than 1, such as 0.1, 0.2, 0.5, etc., that is, when executing S102 in this embodiment, the partial monitoring image when the target vehicle overlaps with the zebra crossing area, such as the monitoring image when there is a large overlap, can be used as a multi-frame target monitoring image corresponding to the target vehicle.

[0038] After executing S102 to obtain multiple frames of target monitoring images corresponding to the target vehicle, this embodiment executes S103 to perform human body detection on each frame of the target monitoring image in descending order of intersection-over-union ratio, and when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements, the target vehicle is determined to be an illegal vehicle that does not give way to pedestrians.

[0039] When executing S103 in this embodiment, the human body detection result obtained by performing human body detection on the target monitoring image includes whether the zebra crossing area in the target monitoring image contains pedestrians, that is, whether the human body detection frame of the pedestrian in the target monitoring image is located in the zebra crossing area. The human body detection frame of the pedestrian is a frame that surrounds the pedestrian in the monitoring image; the human body detection result obtained by executing S103 in this embodiment may further include the orientation information and / or movement direction of the pedestrian contained in the target monitoring image.

[0040] In this embodiment, when executing S103, human body detection is performed on each frame of the target monitoring image in sequence according to the intersection-union ratio from large to small. If it is determined that the human body detection result corresponding to the current target monitoring image does not meet the preset requirements, human body detection is performed on the next target monitoring image in the order of the intersection-union ratio from large to small, and this process is continued until the human body detection of all target monitoring images is completed or it is determined that the human body detection result corresponding to the next target monitoring image meets the preset requirements.

[0041] When executing S103 to determine whether the human body detection result corresponding to the current target monitoring image meets the preset requirements, this embodiment may include the following content: when it is determined that the human body detection result shows that the zebra crossing area contains pedestrians, determining that the human body detection result corresponding to the current target monitoring image meets the preset requirements.

[0042] That is to say, when this embodiment determines that the pedestrian in the current target monitoring image is located in the zebra crossing area, it can determine that the human body detection result corresponding to the current target monitoring image meets the preset requirements, and then determine the target vehicle as an illegal vehicle that does not give way to pedestrians.

[0043] When executing S103 to determine that the human body detection result corresponding to the current target monitoring image meets the preset requirements, this embodiment may also include the following contents: when it is determined that the human body detection result is that the zebra crossing area contains pedestrians, the orientation information and / or movement direction of the pedestrians contained in the zebra crossing area is obtained; when it is determined that the pedestrians contained in the zebra crossing area are close to the target vehicle based on the obtained orientation information and / or movement direction, it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements.

[0044] That is to say, this embodiment combines the orientation information and / or movement direction of pedestrians in the zebra crossing area to determine whether the human body detection result corresponding to the current target monitoring image meets the preset requirements, which can further improve the accuracy of determining whether the target vehicle is an illegal vehicle that does not give way to pedestrians.

[0045] After executing S103 to determine that the target vehicle is an illegal vehicle that does not give way to pedestrians, this embodiment can also save the target monitoring image whose human body detection results meet the preset requirements for subsequent use; in addition, this embodiment can also obtain the license plate information of the illegal vehicle and push the obtained license plate information to the downstream service, so that the downstream service can notify the target vehicle of the illegal behavior of not giving way to pedestrians based on the license plate information.

[0046] Figure 2 is a schematic diagram of the second embodiment of the present disclosure. Figure 2 shows a flow chart for detecting vehicles that do not give way to pedestrians in this embodiment: S201, obtain a frame of monitoring image from the monitoring video stream; S202, obtain the zebra crossing area based on the monitoring image; S203, for each frame of monitoring image in the monitoring video stream, obtain the intersection-and-union ratio between the target vehicle and the zebra crossing area in the frame of monitoring image; S204, select multiple frames of monitoring images with an intersection-and-union ratio greater than a preset intersection-and-union ratio threshold as target monitoring images corresponding to the target vehicle; S205, perform human body detection on the target monitoring images in descending order of the intersection-and-union ratio; S206, when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements, determine that the target vehicle is an illegal vehicle; S207, obtain the license plate information of the target vehicle and push it to the downstream service.

[0047] FIG3 is a schematic diagram of a third embodiment of the present disclosure. As shown in FIG3 , a detection device 300 for detecting a vehicle failing to yield to pedestrians in this embodiment includes:

[0048] An acquisition unit 301 is configured to acquire a zebra crossing area based on a surveillance image in a surveillance video stream;

[0049] The processing unit 302 is configured to obtain, for each monitoring image frame in the monitoring video stream, an intersection-and-union ratio between the target vehicle and the zebra crossing area in the monitoring image frame, and select multiple monitoring images whose intersection-and-union ratios are greater than a preset intersection-and-union ratio threshold as target monitoring images corresponding to the target vehicle;

[0050] The detection unit 303 is used to perform human body detection on each frame of the target monitoring image in order of intersection-union ratio from large to small, and when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements, determine that the target vehicle is an illegal vehicle that does not give way to pedestrians.

[0051] When the acquisition unit 301 obtains the zebra crossing area based on the surveillance image in the surveillance video stream, it can first select a frame of surveillance image from the multiple frames of surveillance image contained in the surveillance video stream, then perform zebra crossing detection on the frame of surveillance image, and finally obtain the zebra crossing area contained in the surveillance scene based on the detection result. The detection result in this embodiment can include a zebra crossing detection score and a zebra crossing detection frame.

[0052] Since the detection of arbitrary-shaped quadrilaterals is more consistent with zebra crossings than rectangular detection, and the arbitrary-shaped quadrilaterals will not frame redundant areas in the image and cause false edge detection, the acquisition unit 301 can use the arbitrary-shaped quadrilateral detection model to perform zebra crossing detection on the monitoring image.

[0053] If the acquisition unit 301 determines that the zebra crossing detection score is greater than or equal to the preset score threshold, the area corresponding to the zebra crossing detection frame in the monitoring image is used as the zebra crossing area; if the acquisition unit 301 determines that the zebra crossing detection score is less than the preset score threshold, this embodiment will obtain another frame of monitoring image from the monitoring video stream, continue to perform zebra crossing detection on the newly obtained monitoring image, and then obtain the zebra crossing area in the monitoring image based on the detection result.

[0054] When the acquisition unit 301 acquires the zebra crossing area based on the surveillance image in the surveillance video stream, it can also adopt the following method: acquire the surveillance image from the surveillance video stream at a preset time interval, that is, this embodiment will continuously acquire the surveillance image according to the preset time interval; perform zebra crossing detection on the acquired surveillance image, and acquire the zebra crossing area based on the detection result.

[0055] That is to say, the acquisition unit 301 continuously acquires surveillance images from the surveillance video stream at preset time intervals. Even when the surveillance camera rotates, the zebra crossing area is acquired based on the surveillance images captured by the surveillance camera after the rotation, thereby improving the accuracy of the acquired zebra crossing area.

[0056] In this embodiment, after the acquisition unit 301 acquires the zebra crossing area, the processing unit 302 obtains the intersection-and-union ratio between the target vehicle and the zebra crossing area in each frame of the monitoring image in the monitoring video stream, and selects multiple frames of monitoring images with an intersection-and-union ratio greater than a preset intersection-and-union ratio threshold as the target monitoring images corresponding to the target vehicle.

[0057] In this embodiment, the target vehicles are all the vehicles included in each frame of monitoring image. One frame of monitoring image may include one target vehicle or multiple target vehicles.

[0058] If a frame of surveillance image contains multiple target vehicles, the processing unit 302 will obtain the intersection-and-union ratio between the current target vehicle and the zebra crossing area in each surveillance image for each target vehicle, and then select multiple frames of target surveillance images corresponding to the current target vehicle based on the obtained intersection-and-union ratio.

[0059] When the processing unit 302 obtains the intersection over union (IoU) between the target vehicle and the zebra crossing area in each frame of surveillance image in the surveillance video stream, the implementation method that can be adopted is: for each frame of surveillance image, vehicle detection is performed on the frame of surveillance image to obtain the vehicle area of ​​the target vehicle in the frame of surveillance image, where the vehicle area is the area in the surveillance image corresponding to the detection frame of the target vehicle; based on the obtained vehicle area and zebra crossing area, the intersection over union (IoU) between the target vehicle and the zebra crossing area in the frame of surveillance image is obtained.

[0060] After obtaining the IoU ratio between the target vehicle and the zebra crossing area in each frame of the monitoring image, the processing unit 302 selects multiple frames of monitoring images with IoU ratios greater than a preset IoU ratio threshold as target monitoring images corresponding to the target vehicle.

[0061] It is understandable that if there are multiple target vehicles, the processing unit 302 will obtain multiple frames of target monitoring images corresponding to each target vehicle. For example, different target vehicles correspond to different image groups, and each image group contains multiple frames of target monitoring images corresponding to the current target vehicle.

[0062] The preset intersection-over-union ratio threshold in this embodiment may be 0, that is, the processing unit 302 may use all monitoring images when the target vehicle overlaps with the zebra crossing area as the multi-frame target monitoring images corresponding to the target vehicle.

[0063] The preset intersection-over-union ratio threshold in this embodiment can also be a value greater than 0 and less than 1, such as 0.1, 0.2, 0.5, etc., that is, the processing unit 302 can use the partial monitoring image when the target vehicle overlaps with the zebra crossing area, such as the monitoring image when there is a large overlap, as a multi-frame target monitoring image corresponding to the target vehicle.

[0064] In this embodiment, after the processing unit 302 obtains multiple frames of target monitoring images corresponding to the target vehicle, the detection unit 303 performs human body detection on each frame of the target monitoring image in descending order of intersection-over-union ratio, and determines that the target vehicle is an illegal vehicle that does not give way to pedestrians when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements.

[0065] The human detection result obtained by the detection unit 303 through human detection on the target monitoring image includes whether the zebra crossing area in the target monitoring image contains pedestrians, that is, whether the human body detection frame of the pedestrian in the target monitoring image is located in the zebra crossing area; the human body detection result obtained by the detection unit 303 may further include the orientation information and / or movement direction of the pedestrian contained in the target monitoring image.

[0066] When the detection unit 303 performs human body detection on each frame of the target monitoring image in sequence according to the intersection-union ratio from large to small, if it is determined that the human body detection result corresponding to the current target monitoring image does not meet the preset requirements, it then performs human body detection on the next target monitoring image in sequence according to the intersection-union ratio from large to small, and continues in this manner until the human body detection of all target monitoring images is completed or it is determined that the human body detection result corresponding to the next target monitoring image meets the preset requirements.

[0067] When the detection unit 303 determines that the human detection result corresponding to the current target monitoring image meets the preset requirements, it may include the following content: when it is determined that the human detection result is that there are pedestrians in the zebra crossing area, it is determined that the human detection result corresponding to the current target monitoring image meets the preset requirements.

[0068] That is to say, when the detection unit 303 determines that the pedestrian in the current target monitoring image is located in the zebra crossing area, it can determine that the human body detection result corresponding to the current target monitoring image meets the preset requirements, and then determine the target vehicle as an illegal vehicle that does not give way to pedestrians.

[0069] When the detection unit 303 determines that the human body detection result corresponding to the current target monitoring image meets the preset requirements, it may also include the following contents: when it is determined that the human body detection result is that the zebra crossing area contains pedestrians, the orientation information and / or movement direction of the pedestrians contained in the zebra crossing area is obtained; when it is determined that the pedestrians contained in the zebra crossing area are close to the target vehicle based on the obtained orientation information and / or movement direction, it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements.

[0070] That is to say, the detection unit 303 combines the orientation information and / or movement direction of pedestrians in the zebra crossing area to determine whether the human body detection result corresponding to the current target monitoring image meets the preset requirements, which can further improve the accuracy of determining whether the target vehicle is an illegal vehicle that does not give way to pedestrians.

[0071] After determining that the target vehicle is an illegal vehicle that does not give way to pedestrians, the detection unit 303 can also save the target monitoring image whose human body detection results meet the preset requirements for subsequent use; in addition, the detection unit 303 can also obtain the license plate information of the illegal vehicle and push the obtained license plate information to the downstream service, so that the downstream service can notify the target vehicle of the illegal behavior of not giving way to pedestrians based on the license plate information.

[0072] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0073] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0074] As shown in FIG4 , a block diagram of an electronic device for detecting a vehicle failing to yield to pedestrians according to an embodiment of the present disclosure is provided. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0075] As shown in Figure 4, device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. Various programs and data required for the operation of device 400 can also be stored in RAM 403. Computing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0076] Various components in device 400 are connected to I / O interface 405, including an input unit 406, such as a keyboard, mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, optical disk, etc.; and a communication unit 409, such as a network card, modem, wireless communication transceiver, etc. Communication unit 409 allows device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0077] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as a method for detecting a vehicle that does not give way to pedestrians. For example, in some embodiments, the method for detecting a vehicle that does not give way to pedestrians can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 408.

[0078] In some embodiments, part or all of the computer program can be loaded and / or installed onto device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by computing unit 401, one or more steps of the above-described method for detecting a vehicle failing to yield to pedestrians can be performed. Alternatively, in other embodiments, computing unit 401 can be configured to execute the method for detecting a vehicle failing to yield to pedestrians via any other suitable means (e.g., via firmware).

[0079] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0080] The program code used to implement the methods of the present disclosure can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable vehicle pedestrian incivility detection device, such that, when executed by the processor or controller, the program code implements the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0081] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0082] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0083] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0084] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is established by computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service system that addresses the management difficulties and poor business scalability of traditional physical hosts and VPS services ("Virtual Private Servers," or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0085] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0086] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for detecting a vehicle that fails to yield to pedestrians, comprising: Obtaining a zebra crossing area according to a surveillance image in a surveillance video stream; For each monitoring image frame in the monitoring video stream, an intersection-and-union ratio between the target vehicle and the zebra crossing area in the monitoring image frame is obtained, and a plurality of monitoring images whose intersection-and-union ratios are greater than a preset intersection-and-union ratio threshold are selected as target monitoring images corresponding to the target vehicle; Human body detection is performed on each frame of the target monitoring image in order of intersection-union ratio from large to small, and when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements, the target vehicle is determined to be an illegal vehicle that does not give way to pedestrians.

2. The method according to claim 1, wherein: The obtaining, for each monitoring image frame in the monitoring video stream, an intersection-and-combination ratio between the target vehicle and the zebra crossing area in the monitoring image frame includes: For each frame of monitoring image, performing vehicle detection on the frame of monitoring image, and obtaining a vehicle area of ​​the target vehicle in the frame of monitoring image; According to the vehicle area and the zebra crossing area, an intersection-and-union ratio between the target vehicle and the zebra crossing area in the frame monitoring image is obtained.

3. The method according to claim 1, wherein: Determining that the human body detection result corresponding to the current target monitoring image meets the preset requirements includes: When it is determined that the human body detection result indicates that the zebra crossing area contains pedestrians, it is determined that the human body detection result corresponding to the current target monitoring image meets a preset requirement.

4. The method according to claim 1, wherein: Determining that the human body detection result corresponding to the current target monitoring image meets the preset requirements includes: When it is determined that the human body detection result indicates that the zebra crossing area contains pedestrians, obtaining orientation information and / or movement direction of the pedestrians contained in the zebra crossing area; In the case where it is determined according to the orientation information and / or the movement direction that the pedestrian contained in the zebra crossing area is close to the target vehicle, it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirement.

5. The method according to claim 1, further comprising: When it is determined that the human body detection result corresponding to the current target monitoring image does not meet the preset requirements, the next target monitoring image is subjected to human body detection in the order of intersection-over-union ratio from large to small. Test; This process continues until the human body detection of all target monitoring images is completed or it is determined that the human body detection result corresponding to the next target monitoring image meets the preset requirements.

6. The method according to claim 1, wherein: The step of obtaining the zebra crossing area according to the monitoring image in the monitoring video stream includes: Acquiring surveillance images from the surveillance video stream at preset time intervals; Perform zebra crossing detection on the monitoring image, and obtain the zebra crossing area according to the detection result.

7. A detection device for vehicles not yielding to pedestrians, comprising: An acquisition unit, used for acquiring a zebra crossing area according to a monitoring image in a monitoring video stream; A processing unit is used to obtain, for each monitoring image frame in the monitoring video stream, an intersection-and-union ratio between the target vehicle and the zebra crossing area in the monitoring image frame, and select multiple monitoring images whose intersection-and-union ratios are greater than a preset intersection-and-union ratio threshold as target monitoring images corresponding to the target vehicle; The detection unit is used to perform human body detection on each frame of the target monitoring image in order of intersection-union ratio from large to small, and determine that the target vehicle is an illegal vehicle that does not give way to pedestrians when it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirements.

8. The device according to claim 7, wherein: When the processing unit obtains the intersection-and-combination ratio between the target vehicle and the zebra crossing area in each frame of the monitoring image in the monitoring video stream, the processing unit specifically performs: For each frame of monitoring image, performing vehicle detection on the frame of monitoring image, and obtaining a vehicle area of ​​the target vehicle in the frame of monitoring image; According to the vehicle area and the zebra crossing area, an intersection-and-union ratio between the target vehicle and the zebra crossing area in the frame monitoring image is obtained.

9. The device according to claim 7, wherein: When the detection unit determines that the human body detection result corresponding to the current target monitoring image meets the preset requirements, the detection unit specifically performs: When it is determined that the human body detection result indicates that the zebra crossing area contains pedestrians, it is determined that the human body detection result corresponding to the current target monitoring image meets a preset requirement.

10. The device according to claim 7, wherein: When the detection unit determines that the human body detection result corresponding to the current target monitoring image meets the preset requirements, the detection unit specifically performs: When it is determined that the human body detection result indicates that the zebra crossing area contains pedestrians, Obtaining orientation information and / or movement direction of pedestrians in the zebra crossing area; In the case where it is determined according to the orientation information and / or the movement direction that the pedestrian contained in the zebra crossing area is close to the target vehicle, it is determined that the human body detection result corresponding to the current target monitoring image meets the preset requirement.

11. The device according to claim 7, wherein: The detection unit is also used to perform: When it is determined that the human body detection result corresponding to the current target monitoring image does not meet the preset requirements, human body detection is performed on the next target monitoring image in descending order of intersection-over-union ratio; This process continues until the human body detection of all target monitoring images is completed or it is determined that the human body detection result corresponding to the next target monitoring image meets the preset requirements.

12. The device according to claim 7, wherein: When the acquisition unit acquires the zebra crossing area according to the monitoring image in the monitoring video stream, the acquisition unit specifically performs: Acquiring surveillance images from the surveillance video stream at preset time intervals; Perform zebra crossing detection on the monitoring image, and obtain the zebra crossing area according to the detection result.

13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images