Pedestrian red light running detection method and system and electronic equipment

Abstract

The invention relates to a pedestrian red light running detection method, and the method comprises the steps: obtaining a pedestrian image frame sequence of a target traffic scene, obtaining a continuous track point and a bounding box of a target pedestrian according to the pedestrian image frame sequence based on a YOLO target detection model and a DeepSort multi-target tracking algorithm, and obtaining a pedestrian red light running detection result. Obtaining a current coordinate and an instantaneous speed of the target pedestrian according to the bounding box and the continuous track point, determining a current state of the target pedestrian based on the current coordinate, the instantaneous speed and a preset stop line, determining a detection target according to the current state, responding to the detection target for traffic light detection, obtaining and identifying a traffic light image, and if the traffic light image is identified to be in a red light state, outputting the traffic light image. And if so, triggering red light running early warning. Through the method, the problem of inaccurate pedestrian red light running detection is solved, YOLO detection and DeepSort tracking are combined, the tracking stability under the scenes of shielding, short leaving of the visual field and the like is improved, the coordinates of the midpoint of the bottom of the bounding box serve as the coordinates of the pedestrian, and the situation that the pedestrian is easily interfered by the posture change of the pedestrian is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic traffic, in particular to a pedestrian red light running detection method, system and electronic device. BACKGROUND

[0002] With the rapid advancement of urbanization, traffic safety problems are increasingly prominent, and pedestrian red light running is common, which is one of the important factors threatening traffic order and pedestrian safety.

[0003] The traditional pedestrian red light running detection method determines whether the target crosses the preset boundary by the center position of the target bounding box, which is easily disturbed by the change of the pedestrian posture (such as turning sideways and bending), resulting in misjudgment or omission; and only based on the position of a single frame or a few continuous frames to determine the crossing line, which is easy to misjudge the instantaneous touch line as the actual crossing behavior. SUMMARY

[0004] The embodiments of the present application provide a pedestrian red light running detection method, system, electronic device and storage medium to at least solve the problem of inaccurate pedestrian red light running detection in related technologies.

[0005] In a first aspect, the embodiments of the present application provide a pedestrian red light running detection method, which comprises: Obtaining a pedestrian image frame sequence of a target traffic scene, based on a YOLO target detection model and a DeepSort multi-target tracking algorithm, obtaining a continuous trajectory point of a target pedestrian and a bounding box of the target pedestrian in each frame according to the pedestrian image frame sequence; According to the bounding box and the continuous trajectory point, obtaining a current coordinate of the target pedestrian and an instantaneous speed, the current coordinate being a coordinate of a bottom center point of a bounding box corresponding to a current frame; Based on the current coordinate, the instantaneous speed and a preset stop line, determining a current state of the target pedestrian, and determining a detection target according to the current state; In response to the detection target being a red-green light detection, obtaining a red-green light image, identifying the red-green light image, determining whether the current red-green light is in a red light state according to the identification result, and if so, triggering a red light running warning.

[0006] In some embodiments, the first reference point and the second reference point are provided on the preset stop line; and the determination of the current state of the target pedestrian based on the current coordinate, the instantaneous speed and the preset stop line comprises: According to the current coordinate, the coordinates of the first reference point and the second reference point, determining the positional relationship between the target pedestrian and the preset stop line, and the relative distance from the current coordinate of the target pedestrian to the preset stop line; determine a dynamic distance threshold according to the instantaneous speed; determine a current state of the target pedestrian based on the position relationship, the relative distance and the dynamic distance threshold, the state of the target pedestrian including a safe state and a pending right-of-way state.

[0007] In some embodiments, the determining the current state of the target pedestrian based on the position relationship, the relative distance and the dynamic distance threshold includes: determining whether the target pedestrian has crossed a line according to the position relationship when the relative distance is greater than the dynamic distance threshold, considering the current state of the target pedestrian as the pending right-of-way state if the target pedestrian has crossed the line, considering the current state of the target pedestrian as the safe state if the target pedestrian has not crossed the line.

[0008] In some embodiments, the determining the detection target according to the current state includes: setting the detection target as a traffic light detection when the current state is the pending right-of-way state, setting the detection target as continuing to detect the state of the target pedestrian when the current state is the safe state.

[0009] In some embodiments, the obtaining the continuous trajectory points of the target pedestrian and the bounding box corresponding to each image frame based on the YOLO target detection model and the DeepSort multi-target tracking algorithm according to the pedestrian image frame sequence includes: analyzing the pedestrian images in the pedestrian image frame sequence based on the YOLO target detection model to obtain the bounding box data of the pedestrians in each pedestrian image, obtaining the continuous trajectory points of the target pedestrian and the bounding box corresponding to the target pedestrian in the bounding box data based on the DeepSort multi-target tracking algorithm according to the pedestrian image frame sequence.

[0010] In some embodiments, the identifying the traffic light image and determining whether the current traffic light is in a red light state according to the identification result includes: converting the traffic light image to an HSV color space, performing saturation equalization on the converted traffic light image to obtain a preprocessed image, performing denoising processing on the preprocessed image based on a morphological opening operation method to obtain a continuous red region in the image, determining an image confidence based on a ratio of an area of the continuous red region to a total area of the traffic light image, determining whether the current traffic light is in a red light state based on the image confidence and a confidence threshold.

[0011] In some embodiments, determining whether the current traffic light is in a red light state based on the image confidence and a confidence threshold comprises: adjusting the confidence threshold dynamically according to historical confidence data of the traffic light; determining that the current traffic light is in a red light state if the image confidence is greater than or equal to the confidence threshold, and triggering a red light running warning; determining that the current traffic light is not in a red light state if the image confidence is less than the confidence threshold, and continuing to detect the state of the target pedestrian.

[0012] In some embodiments, triggering the red light running warning comprises: recording relevant data of the crossing event, the relevant data comprising a unique ID of the pedestrian, an initial crossing time, a sequence of pedestrian image frames and the continuous trajectory point based on a DeepSort multi-target tracking algorithm; reporting the relevant data to a traffic management platform.

[0013] In a second aspect, the embodiments of the present application provide a pedestrian red light running detection system, the system comprising: an image acquisition module configured to obtain a sequence of pedestrian image frames of a target traffic scene, and obtain a continuous trajectory point of a target pedestrian and a bounding box of the target pedestrian in each image frame based on a YOLO target detection model and a DeepSort multi-target tracking algorithm; a parameter analysis module configured to obtain a current coordinate and an instantaneous speed of the target pedestrian based on the bounding box and the continuous trajectory point, the current coordinate being a coordinate of a bottom midpoint of a current frame corresponding bounding box; a state judgment module configured to determine a current state of the target pedestrian based on the current coordinate, the instantaneous speed and a preset stop line, and determine a detection target according to the current state; a warning module configured to, in response to the detection target being a traffic light detection, acquire a traffic light image, identify the traffic light image, determine whether a current traffic light is in a red light state according to an identification result, and trigger a red light running warning if the current traffic light is in a red light state.

[0014] In a third aspect, the embodiments of the present application provide a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the pedestrian red light running detection method of the first aspect when executing the computer program.

[0015] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the pedestrian red light running detection method in the first aspect.

[0016] Compared with the related art, the pedestrian red light running detection method provided by the embodiments of the present application combines the high-precision target detection of YOLO and the long-time multi-target tracking of DeepSort, solves the problem that the traditional single-frame detection cannot associate continuous behaviors, improves the tracking stability in scenes such as occlusion and temporary leaving of the field of view, takes the coordinate of the bottom midpoint of the bounding box as the pedestrian coordinate, avoids the interference of pedestrian posture changes (such as turning sideways and bending), and thus avoids misjudgment or omission, and solves the problem of inaccurate pedestrian red light running detection. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings: Figure 1 is a flowchart of the pedestrian red light running detection method according to the embodiments of the present application; Figure 2 is a stop line schematic diagram according to the embodiments of the present application; Figure 3 is a flowchart of a current state determination of a target pedestrian according to the embodiments of the present application; Figure 4 is a flowchart of a confidence calculation method according to the embodiments of the present application; Figure 5 is a flowchart of a pedestrian red light running detection method according to the embodiments of the present application; Figure 6 is a structural block diagram of a pedestrian red light running detection system according to the embodiments of the present application; Figure 7 is an internal structure schematic diagram of an electronic device according to the embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application is described and explained below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0019] It is apparent that the drawings in the following description merely show some examples or embodiments of the present application, and the present application can be applied to other similar situations without creative efforts by those skilled in the art based on these drawings. In addition, it can be understood that, although the efforts made in the development process can be complex and lengthy, some modifications, such as design, manufacture or production, made on the basis of the technical content disclosed in the present application by those skilled in the art related to the content disclosed in the present application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the present application.

[0020] Reference to "an embodiment" in this application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is explicitly contemplated that embodiments described herein can be combined with other embodiments in a non- conflicting manner.

[0021] Unless otherwise defined, technical terms or scientific terms used in the present application should be understood as their common meanings to those skilled in the art. The terms "a", "an", "one", "this", and similar terms in the present application do not represent quantity limitation, but can represent singular or plural. The terms "include", "contain", "have", and any variations thereof in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or units, but can further include steps or units not listed or can further include other steps or units inherent to the process, method, product or device. The terms "connect", "connected", "couple", and similar terms in the present application are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The term "multiple" in the present application means two or more. The term "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The terms "first", "second", "third", and the like in the present application are merely to distinguish similar objects, and do not represent a specific order of the objects.

[0022] The embodiment provides a pedestrian red light running detection method. Figure 1 The flow chart of the pedestrian red light running detection method according to the embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 1 The flow chart of the pedestrian red light running detection method according to the embodiment of the present application is shown in FIG. 1, which includes the following steps: Step S101, obtain a pedestrian image frame sequence of a target traffic scene, and based on a YOLO target detection model and a DeepSort multi-target tracking algorithm, obtain continuous trajectory points of a target pedestrian and a bounding box of the target pedestrian in each frame of image according to the pedestrian image frame sequence.

[0023] A high-definition camera is deployed at an intersection to collect traffic scene images in real time. The camera supports multi-angle coverage and clear imaging in low-light environments.

[0024] In some embodiments, step S101 specifically includes: Step S1012, based on the YOLO target detection model, analyze the pedestrian image in the pedestrian image frame sequence to obtain the bounding box data of the pedestrian in each frame of pedestrian image.

[0025] Based on the YOLO target detection model, the pedestrian target is detected in real time and the coordinates and confidence of the bounding box are output.

[0026] Step S1012, based on the DeepSort multi-target tracking algorithm, obtain continuous trajectory points of a target pedestrian and a bounding box corresponding to the target pedestrian in the bounding box data according to the pedestrian image frame sequence.

[0027] Based on the DeepSort multi-target tracking algorithm, set tracking parameters (such as maximum lost frame number and initialization required frame number) to track the pedestrian target across frames, output the unique ID of the pedestrian and the continuous trajectory points with time stamp. Optionally, the maximum lost frame number in the tracking parameter is set to 30 to adapt to the scene of temporary occlusion of pedestrians (such as being occluded by other pedestrians or vehicles) and avoid false termination of tracking; the initialization required frame number is set to 2 to quickly start tracking of new pedestrians entering the scene and reduce delay.

[0028] The YOLO algorithm can quickly locate pedestrians in complex scenes (such as occlusion, light change, and target density) and output accurate bounding boxes, class labels, and confidence. This high-precision detection provides accurate input of the initial target state for the tracking of DeepSort, avoids tracking drift or identity confusion caused by detection errors (such as missed detection, false detection, and bounding box offset), and reduces the possibility of tracking errors from the source.

[0029] DeepSort combines deep learning-based appearance feature extraction with Kalman filter-based motion prediction, is good at maintaining tracking stability of multiple targets in long time series, and can track multiple pedestrians at the same time.

[0030] Combining the high-precision target detection of YOLO and the long-time multi-target tracking of DeepSort, the problem that traditional single-frame detection cannot associate continuous behavior is solved, and the tracking stability in scenes such as occlusion and temporary out of sight is improved.

[0031] In step S102, the current coordinates and instantaneous speed of the target pedestrian are obtained according to the bounding box and the continuous trajectory point. The current coordinates are the coordinates of the midpoint of the bottom of the bounding box corresponding to the current frame.

[0032] The physical existence of the pedestrian is supported by the feet, and the position of the feet directly corresponds to the actual coordinates of the pedestrian on the ground. Therefore, the midpoint of the bottom of the bounding box is highly consistent with the actual landing position of the feet of the pedestrian, and can truly reflect the position of the pedestrian in the ground coordinate system. The motion trajectory of the bottom midpoint directly corresponds to the walking path of the pedestrian, and can be accurately used to determine whether the pedestrian runs a red light (whether the feet cross the line), avoiding additional fluctuations caused by the swinging of the pedestrian's limbs (such as waving arms) or changes in posture (such as bending), which may deviate from the actual walking path and easily lead to behavior judgment errors.

[0033] In step S103, the current state of the target pedestrian is determined based on the current coordinates, the instantaneous speed and the preset stop line, and the detection target is determined according to the current state.

[0034] The preset stop line is a virtual line in the image and is a vector from the first reference point A (x1, y1) to the second reference point B (x2, y2) . Figure 2 It is a stop line diagram according to an embodiment of the present application, as Figure 2 shown, when the pedestrian does not cross the stop line, point A is on the right side of the pedestrian and point B is on the left side of the pedestrian, that is, the pedestrian is on the left side of the stop line at this time, and after the pedestrian crosses the stop line, the pedestrian is on the right side of the stop line.

[0035] In some embodiments, the first reference point and the second reference point are provided on the preset stop line; and in step S103, determining the current state of the target pedestrian based on the current coordinates, the instantaneous speed and the preset stop line comprises: In step S1031, the position relationship between the target pedestrian and the preset stop line, and the relative distance from the current coordinates of the target pedestrian to the preset stop line are determined according to the current coordinates, the coordinates of the first reference point and the coordinates of the second reference point.

[0036] The current coordinates of the target pedestrian are P (x, y), and the target point P is preliminarily judged on which side of the stop line: The cross product of the vector (x2-x1,y2-y1) and the vector (x-x1,y-y1) is calculated, ​cross = (x2 - x1)(y - y1) - (y2 - y1)(x - x1) If cross > 0, it is preliminarily determined that the target point is behind the stop line (not crossing the line), and if cross < 0, it is preliminarily determined that the target point is in front of the stop line (crossing the line).

[0037] The relative distance d of the target point to the stop line is calculated as follows:

[0038] In step S1032, the dynamic distance threshold is determined according to the instantaneous speed.

[0039] The dynamic distance threshold threshold(v) is a threshold of the distance of the target point P to the stop line, which is dynamically adjusted according to the instantaneous speed v of the pedestrian. The calculation method of the dynamic distance threshold is as follows: threshold(v) = kv + b wherein k is a speed coefficient, and b is a basic threshold distance. Optionally, the initial value of k is set to 0.2 m / s, and the initial value of b is set to 0.3 meters, which are fitted and adjusted based on historical pedestrian step frequency data in actual application.

[0040] In step S1033, the current state of the target pedestrian is determined based on the positional relationship, the relative distance, and the dynamic distance threshold. The state of the target pedestrian includes a safe state and a pending right-of-way state.

[0041] In some embodiments, step S1033 specifically includes: In step S201, if the relative distance is greater than the dynamic distance threshold, it is determined whether the target pedestrian has crossed the line according to the positional relationship.

[0042] In step S202, if the target pedestrian has crossed the line, it is considered that the current state of the target pedestrian is the pending right-of-way state.

[0043] In step S203, if the target pedestrian has not crossed the line, it is considered that the current state of the target pedestrian is the safe state.

[0044] When the distance d of the target point to the stop line is greater than or equal to threshold(v), it is finally determined which side of the stop line the target point is on according to the positive or negative of When the target point is on the left side of the stop line, it is considered that the pedestrian is currently in a safe state, and when the target point is on the right side of the stop line, it is considered that the pedestrian is currently in a pending right-of-way state, and needs to enter the next step of red light detection to determine whether the pedestrian has run a red light.

[0045] If the distance d of the target point to the stop line is less than threshold(v), it is considered that the current state of the pedestrian is unstable, and no state judgment is performed.

[0046] In some embodiments, determining the detection target according to the current state in step S103 includes: Step S1034, in the case where the current state is the pending right-of-way state, setting the detection target as the red light detection.

[0047] Step S1035, in the case where the current state is the safe state, setting the detection target as the state of continuing to detect the pedestrian.

[0048] If the final state of the target pedestrian is the pending right-of-way state (has crossed the line), the red light detection is entered, in the case where the red light state is detected, it is determined that the target pedestrian runs the red light to trigger the early warning, in the case where the non-red light state is detected, the crossing line detection on the target pedestrian is continued; if the final state of the target pedestrian is the safe state (has not crossed the line), the crossing line detection on the target pedestrian is continued. Table 1 is a state transition logic of a pedestrian according to an embodiment of the present application. Figure 3 FIG. 1 is a flowchart of determining the current state of a target pedestrian according to an embodiment of the present application.

[0049] Table 1

[0050] Setting a return re-determination mechanism solves the missing judgment problem of the traditional method “not monitoring again after crossing the line once” and supports twice crossing line determination Step S104, in response to the detection target being the red light detection, acquiring a red light image, identifying the red light image, determining whether the current red light is in the red light state according to the identification result, and if so, triggering the red light running early warning.

[0051] In some embodiments, identifying the red light image in step S104 includes: Step S1041, converting the red light image to the HSV color space, performing saturation equalization on the converted red light image to obtain a preprocessed image.

[0052] The numerical ranges of the hue (H), saturation (S) and brightness (V) channels are dynamically set to clearly define the identification boundary of the red color in the HSV color space, and to adapt to the red color detection under different light changes.

[0053] The color image is converted to the HSV color space to separate the color information (H), brightness information (V) and saturation information (S), which facilitates red color extraction. Saturation equalization is performed on the converted red light image to enhance the saturation contrast of the red color region and reduce low saturation interference (such as faded signal lights).

[0054] Step S1042, based on the morphological opening operation method, performing denoising processing on the preprocessed image to obtain the continuous red color region in the image.

[0055] Remove noise (e.g. small area interference points) by morphological opening operation, and keep the continuous red area.

[0056] Step S1043, determine the image confidence based on the ratio of the area of the continuous red area to the total area of the traffic light image.

[0057] Determine the confidence by counting the proportion of the number of non-zero pixels (red area) to the total number of pixels (traffic light area), and quantify the significance of the red signal. Figure 4 is a flowchart of a confidence calculation method according to an embodiment of the present application.

[0058] Step S1044, determine whether the current traffic light is in red light state based on the image confidence and the confidence threshold.

[0059] In some embodiments, step S1044 specifically includes: Step S301, dynamically adjust the confidence threshold according to the historical confidence data of the traffic light.

[0060] Statistical historical confidence distribution of the same traffic light area, and dynamically adjust the threshold. For example, set the threshold to 0.6 during the day and 0.4 at night.

[0061] Step S302, if the image confidence is greater than or equal to the confidence threshold, determine that the current traffic light is in red light state, and trigger the red light running warning.

[0062] Step S303, if the image confidence is less than the confidence threshold, determine that the current traffic light is not in red light state, and continue to detect the state of the target pedestrian.

[0063] Set the confidence threshold (e.g. ≥ 0.7), and only accept results that exceed the threshold, to reduce false positives caused by noise or interference. Figure 5 is a flowchart of a pedestrian red light running detection according to an embodiment of the present application.

[0064] In some embodiments, triggering the red light running warning includes: Record the relevant data of the crossing event, including the unique ID of the pedestrian based on the DeepSort multi-target tracking algorithm, the initial crossing time, the sequence of the pedestrian image frames and the continuous trajectory points; Report the relevant data to the traffic management platform.

[0065] Record the crossing event (unique ID of the target pedestrian, initial crossing time, trajectory segment, evidence image), trigger the alarm and upload to the traffic management platform. Support event playback and multi-channel video linkage, which is convenient for manual review.

[0066] Through the above steps, the high-precision target detection of YOLO and the long-time sequence multi-target tracking of DeepSort are combined, the problem that the traditional single-frame detection cannot be associated with continuous behavior is solved, and the tracking stability in scenes such as occlusion and temporary leaving the field of view is improved. Through the multi-dimensional conditions of “initial area→target area + time + distance”, combined with a dynamic threshold, the substantive line-crossing behavior is accurately identified. Through the re-judgment mechanism of turning back, the missed judgment problem of the traditional method “once crossing the line no longer monitoring” is solved, and the secondary line-crossing judgment is supported.

[0067] It should be noted that the steps shown in the above flow or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0068] The embodiment also provides a pedestrian red light running detection system, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the terms “module”, “unit”, “sub-unit” and the like can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware, or a combination of software and hardware implementation is also possible and contemplated.

[0069] Figure 6 is a structural block diagram of a pedestrian red light running detection system according to the embodiment of the application, as shown in Figure 6 The system comprises: An image acquisition module 61 is configured to acquire a pedestrian image frame sequence of a target traffic scene, and obtain continuous trajectory points of a target pedestrian and a bounding box of the target pedestrian in each frame of image based on a YOLO target detection model and a DeepSort multi-target tracking algorithm according to the pedestrian image frame sequence.

[0070] A parameter analysis module 62 is configured to obtain a current coordinate and an instantaneous speed of the target pedestrian according to the bounding box and the continuous trajectory points, and the current coordinate is a coordinate of a bottom midpoint of the bounding box corresponding to a current frame.

[0071] A state judgment module 63 is configured to determine a current state of the target pedestrian based on the current coordinate, the instantaneous speed and a preset stop line, and determine a detection target according to the current state.

[0072] A warning module 64 is configured to, in response to the detection target being a red-green light detection, acquire a red-green light image, identify the red-green light image, determine whether a current red-green light is in a red light state according to an identification result, and trigger a red light running warning if the current red-green light is in the red light state.

[0073] In some embodiments, the preset stop line is provided with a first reference point and a second reference point; the state determining module 63 comprises: The position analysis module is configured to determine a position relationship between the target pedestrian and the preset stop line and a relative distance from the target pedestrian to the preset stop line according to the current coordinates, the coordinates of the first reference point and the coordinates of the second reference point.

[0074] The threshold determination module is configured to determine a dynamic distance threshold according to the instantaneous speed.

[0075] The state determination module is configured to determine a current state of the target pedestrian based on the position relationship, the relative distance and the dynamic distance threshold, the state of the target pedestrian including a safe state and a pending right-of-way state.

[0076] In some embodiments, the state determination module is configured to determine whether the target pedestrian has crossed the line according to the position relationship if the relative distance is greater than the dynamic distance threshold, and if the target pedestrian has crossed the line, the current state of the target pedestrian is determined as the pending right-of-way state, and if the target pedestrian has not crossed the line, the current state of the target pedestrian is determined as the safe state.

[0077] In some embodiments, the state determining module 63 comprises: The first detection module is configured to set the detection target as a traffic light detection if the current state is the pending right-of-way state.

[0078] The second detection module is configured to set the detection target as a continuous state detection of the target pedestrian if the current state is the safe state.

[0079] In some embodiments, the image acquisition module 61 comprises: The calibration module is configured to analyze the pedestrian images in the pedestrian image frame sequence based on a YOLO target detection model to obtain the bounding box data of the pedestrians in each pedestrian image.

[0080] The tracking module is configured to obtain the continuous trajectory points of the target pedestrian and the bounding box corresponding to the target pedestrian in the bounding box data based on a DeepSort multi-target tracking algorithm according to the pedestrian image frame sequence.

[0081] In some embodiments, the warning module 64 comprises: The preprocessing module is configured to convert the traffic light image to an HSV color space, perform saturation equalization on the converted traffic light image to obtain a preprocessed image.

[0082] The denoising module is configured to perform denoising processing on the preprocessed image based on a morphological opening operation method to obtain a continuous red region in the image.

[0083] The confidence determination module is configured to determine the image confidence based on a ratio of the continuous red area and a total area of the traffic light image.

[0084] The red light determination module is configured to determine whether the current traffic light is in a red light state based on the image confidence and a confidence threshold.

[0085] In some embodiments, the red light determination module includes: The threshold adjustment module is configured to dynamically adjust the confidence threshold according to historical confidence data of the traffic light.

[0086] The first determination module is configured to determine that the current traffic light is in the red light state and trigger the red light running warning if the image confidence is greater than or equal to the confidence threshold.

[0087] The second determination module is configured to determine that the current traffic light is not in the red light state and continue to detect the state of the target pedestrian if the image confidence is less than the confidence threshold.

[0088] In some embodiments, the warning module 64 includes: The recording module is configured to record the relevant data of the line-crossing event, and the relevant data includes a unique ID of the pedestrian, an initial line-crossing time, a sequence of pedestrian image frames, and a continuous trajectory point, which are output based on the DeepSort multi-target tracking algorithm.

[0089] The reporting module is configured to report the relevant data to a traffic management platform.

[0090] Through the above system, the high-precision target detection of YOLO and the long-time multi-target tracking of DeepSort are combined to solve the problem that the traditional single-frame detection cannot be associated with continuous behavior, improve the tracking stability in scenes such as occlusion and temporary leaving of the field of view, use the coordinate of the bottom midpoint of the bounding box as the pedestrian coordinate to avoid interference caused by changes in the posture of the pedestrian (such as turning sideways and bending over), which leads to misjudgment or omission, and solve the problem of inaccurate detection of red light running by pedestrians.

[0091] It should be noted that each of the above modules can be a functional module or a program module, and can be implemented by software or hardware. For the modules implemented by hardware, each of the above modules can be located in the same processor; or each of the above modules can also be located in different processors in any combination.

[0092] The embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the method embodiments.

[0093] Optionally, the electronic device described above can further include a transmission device connected with the processor, and an input and output device connected with the processor.

[0094] Optionally, in the embodiment, the processor can be configured to execute the following steps by means of a computer program: S1, obtaining a pedestrian image frame sequence of a target traffic scene, and obtaining continuous trajectory points of a target pedestrian and a bounding box of the target pedestrian in each frame of image based on a YOLO target detection model and a DeepSort multi-target tracking algorithm according to the pedestrian image frame sequence.

[0095] S2, obtaining a current coordinate and an instantaneous speed of the target pedestrian according to the bounding box and the continuous trajectory points, and the current coordinate being a coordinate of a bottom midpoint of the bounding box corresponding to a current frame.

[0096] S3, determining a current state of the target pedestrian based on the current coordinate, the instantaneous speed and a preset stop line, and determining a detection target according to the current state.

[0097] S4, in response to the detection target being a traffic light detection, obtaining a traffic light image, identifying the traffic light image, determining whether a current traffic light is in a red light state according to an identification result, and triggering a red light running warning if so.

[0098] It should be noted that the specific examples in the embodiment can refer to the examples described in the above embodiments and optional implementation manners, and the embodiment will not be described here.

[0099] In one embodiment, Figure 7 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, as Figure 7 indicated, an electronic device is provided, which can be a server, and the internal structure diagram thereof can be as Figure 7 indicated. The electronic device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a pedestrian red light running detection method.

[0100] Those skilled in the art can understand that, Figure 7The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0101] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM) and the like.

[0102] Those skilled in the art should understand that each technical feature of the above-mentioned embodiments can be combined arbitrarily, and in order to make the description simple, not all possible combinations of each technical feature in the above-mentioned embodiments are described, however, as long as the combination of these technical features does not exist contradictory, it should be considered as the scope of the present application.

[0103] The above-mentioned embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for detecting pedestrians running red lights, characterized in that, The method comprises: obtaining a sequence of pedestrian image frames of a target traffic scene, and based on a YOLO target detection model and a DeepSort multi-target tracking algorithm, obtaining continuous trajectory points of a target pedestrian and a bounding box of the target pedestrian in each frame of image according to the sequence of pedestrian image frames; obtaining a current coordinate and an instantaneous speed of the target pedestrian according to the bounding box and the continuous trajectory points, wherein the current coordinate is a coordinate of a bottom midpoint of a bounding box corresponding to a current frame; determining a current state of the target pedestrian based on the current coordinate, the instantaneous speed and a preset stop line, and determining a detection target according to the current state; in response to the detection target being a red light detection, obtaining a red light image, identifying the red light image, and determining whether a current red light is in a red light state according to an identification result, and if so, triggering a red light running warning.

2. The method of claim 1, wherein, The first reference point and the second reference point are arranged on the preset stop line; and determining the current state of the target pedestrian based on the current coordinate, the instantaneous speed and the preset stop line comprises: determining a positional relationship between the target pedestrian and the preset stop line, and a relative distance from the current coordinate of the target pedestrian to the preset stop line according to the current coordinate, a coordinate of the first reference point and a coordinate of the second reference point; determining a dynamic distance threshold according to the instantaneous speed; determining the current state of the target pedestrian based on the positional relationship, the relative distance and the dynamic distance threshold, wherein the state of the target pedestrian comprises a safe state and a pending right-of-way state.

3. The method of claim 2, wherein, The determining the current state of the target pedestrian based on the positional relationship, the relative distance and the dynamic distance threshold comprises: in a case where the relative distance is greater than the dynamic distance threshold, determining whether the target pedestrian has crossed a line according to the positional relationship, if the target pedestrian has crossed the line, considering that the current state of the target pedestrian is the pending right-of-way state, if the target pedestrian has not crossed the line, considering that the current state of the target pedestrian is the safe state.

4. The method of claim 2, wherein, The determining the detection target according to the current state comprises: in a case where the current state is the pending right-of-way state, setting the detection target as the red light detection; in a case where the current state is the safe state, setting the detection target as continuing to detect the state of the target pedestrian.

5. The method of claim 1, wherein, The obtaining the continuous trajectory points of the target pedestrian and the bounding box corresponding to the target pedestrian in each frame of image based on the sequence of pedestrian image frames comprises: analyzing pedestrian images in the sequence of pedestrian image frames based on the YOLO target detection model to obtain bounding box data of pedestrians in each pedestrian image; obtaining the continuous trajectory points of the target pedestrian and the bounding box corresponding to the target pedestrian in the bounding box data based on the DeepSort multi-target tracking algorithm according to the sequence of pedestrian image frames.

6. The method of claim 1, wherein, The identifying the red light image and determining whether the current red light is in the red light state according to an identification result comprises: Converting the traffic light image to an HSV color space, performing saturation equalization on the converted traffic light image, and obtaining a preprocessed image; Performing denoising processing on the preprocessed image based on a morphological opening operation method, and obtaining a continuous red region in the image; Determining an image confidence based on a ratio of an area of the continuous red region to a total area of the traffic light image; Determining whether the current traffic light is in a red light state based on the image confidence and a confidence threshold.

7. The method of claim 6, wherein, The determination whether the current traffic light is in the red light state based on the image confidence and the confidence threshold includes: Adjusting the confidence threshold dynamically according to historical confidence data of the traffic light; In a case where the image confidence is greater than or equal to the confidence threshold, determining that the current traffic light is in the red light state, and triggering a red light running warning; In a case where the image confidence is less than the confidence threshold, determining that the current traffic light is not in the red light state, and continuing to detect a state of the target pedestrian.

8. The method of claim 1, wherein, The triggering of the red light running warning includes: Recording related data of the crossing event, the related data including a pedestrian unique ID, an initial crossing time, a sequence of pedestrian images, and continuous trajectory points, which are output based on a DeepSort multi-target tracking algorithm; Reporting the related data to a traffic management platform.

9. A system for detecting jaywalking, the system comprising: The system includes: An image acquisition module configured to obtain a sequence of pedestrian image frames of a target traffic scene, and obtain continuous trajectory points of a target pedestrian and a bounding box of the target pedestrian in each image frame based on a YOLO target detection model and a DeepSort multi-target tracking algorithm; A parameter analysis module configured to obtain a current coordinate and an instantaneous speed of the target pedestrian based on the bounding box and the continuous trajectory points, the current coordinate being a coordinate of a bottom midpoint of a current frame corresponding bounding box; A state judgment module configured to determine a current state of the target pedestrian based on the current coordinate, the instantaneous speed, and a preset stop line, and determine a detection target according to the current state; An early warning module configured to, in response to the detection target being a traffic light detection, acquire a traffic light image, identify the traffic light image, determine whether a current traffic light is in a red light state according to an identification result, and trigger a red light running warning if the current traffic light is in the red light state.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the pedestrian red light running detection method according to any one of claims 1 to 8.

Images