Battery car detection method and device and electronic equipment

Abstract

The embodiment of the invention provides a battery car detection method and device and electronic equipment. According to the embodiment of the invention, whether the battery car is suspected to exist in the current image frame or not is determined based on the target detection result corresponding to the current image frame for the collected current image frame, and then when it is determined that the battery car is suspected to exist in the current image frame, the battery car is determined to be suspected to exist in the current image frame. And according to the target detection result corresponding to each image frame acquired within a period of time starting from the acquisition ending moment of the current image frame, determining whether the battery car exists in the specified scene again so as to determine whether to output a battery car alarm instruction, thereby realizing battery car detection in the specified scene through automation. According to the battery car detection system, the situation of missing detection of the battery car caused by a manual mode is avoided, whether the battery car exists or not is detected secondarily, the accuracy of battery car detection is improved, and the situation of false alarm of the battery car is effectively avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and electronic equipment for detecting electric battery vehicles. Background Art

[0002] In recent years, accidents caused by electric bicycles entering elevators, such as those in elevators where electric bicycles are prohibited from entering, have become common. For example, fires caused by electric bicycles spontaneously combusting inside elevators have occurred. Therefore, to improve safety, it is necessary to detect and manage the occurrence of electric bicycles entering elevators.

[0003] Currently, it is usually necessary to manually check the elevator monitoring screen to determine whether there is an electric bicycle entering the elevator. However, this method has high labor costs and may miss the electric bicycle detection. Summary of the Invention

[0004] In view of this, the present application provides a battery vehicle detection method, device and electronic equipment to realize the automation of battery vehicle detection and avoid the situation where battery vehicles are missed in detection.

[0005] The present invention provides a method for detecting an electric vehicle battery, the method comprising:

[0006] Get the current image frame captured by the camera installed in the specified scene; the specified scene is set to prohibit battery vehicles from entering;

[0007] Perform target detection on the current image frame using the trained target detection model to obtain the target detection result corresponding to the current image frame;

[0008] If, based on the target detection result corresponding to the current image frame, it is determined that an electric bicycle is suspected to exist in the current image frame, then each image frame captured by the camera within a first specified time period is obtained; the first specified time period is a period of time starting from the moment when the capture of the current image frame ends; target detection is performed on each image frame within the first specified time period using the trained target detection model to obtain target detection results corresponding to each image frame within the first specified time period;

[0009] If it is determined that there is an electric battery vehicle in the specified scene based on the target detection results corresponding to each image frame within the first specified time period, an electric battery vehicle alarm indication is output.

[0010] The present application also provides a battery vehicle detection device, which includes:

[0011] The acquisition module is used to obtain the current image frame captured by the camera installed in the specified scene; the specified scene is set to prohibit battery vehicles from entering;

[0012] The detection module is used to perform target detection on the current image frame using the trained target detection model to obtain the target detection result corresponding to the current image frame;

[0013] A determination module is configured to, if it is determined based on the target detection result corresponding to the current image frame that a battery vehicle is suspected to exist in the current image frame, obtain each image frame captured by the camera within a first specified time period; the first specified time period is a period of time starting from the end time of capture of the current image frame; perform target detection on each image frame within the first specified time period using a trained target detection model to obtain target detection results corresponding to each image frame within the first specified time period;

[0014] The alarm module is used to output an electric battery vehicle alarm indication if it is determined that there is an electric battery vehicle in the specified scene based on the target detection results corresponding to each image frame within the first specified time period.

[0015] An embodiment of the present application further provides an electronic device, comprising:

[0016] processor; and

[0017] A computer-readable storage medium stores computer program instructions, which, when executed by a processor, cause the processor to execute the steps of the above method.

[0018] An embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor is caused to execute the steps in the above method.

[0019] It can be seen from the above technical solution that in the embodiment of the present application, first, for the current image frame collected, based on the target detection result corresponding to the current image frame, it is determined whether there is a suspected electric vehicle in the current image frame. Afterwards, when it is determined that there is a suspected electric vehicle in the current image frame, based on the target detection results corresponding to each image frame collected within a period of time starting from the end time of collection of the current image frame, it is determined again whether there is an electric vehicle in the specified scene to determine whether to output an electric vehicle alarm indication. This not only realizes the electric vehicle detection in the specified scene through automation, avoiding the situation of missed electric vehicle detection caused by manual methods, but also improves the accuracy of electric vehicle detection by secondary detection of whether there is an electric vehicle, effectively avoiding the situation of false alarm of electric vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] Figure 1A flow chart of the method provided in an embodiment of the present application.

[0022] Figure 2 Another method flow chart provided in an embodiment of the present application.

[0023] Figure 3 Another method flow chart provided in an embodiment of the present application.

[0024] Figure 4 A schematic diagram of the device structure provided in an embodiment of the present application.

[0025] Figure 5 A schematic diagram of the electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0027] See also Figure 1 , Figure 1 This is a flow chart of a method provided in an embodiment of the present application. The method is applied to an electronic device. As an embodiment, the electronic device may be, for example, a camera installed in a specified scene, or a computer communicatively connected to a camera installed in a specified scene, etc., which is not specifically limited in this embodiment.

[0028] like Figure 1 As shown, the process may include the following steps:

[0029] Step 101: obtain a current image frame captured by a camera installed in a specified scene; the specified scene is set to prohibit electric bicycles from entering.

[0030] In this embodiment, the designated scene may refer to a scene that is set to prohibit electric vehicles from entering, such as an elevator scene, which is not specifically limited in this embodiment.

[0031] Step 102: Perform target detection on the current image frame using the trained target detection model to obtain a target detection result corresponding to the current image frame.

[0032] In this embodiment, the target detection model refers to a model used for target detection. This embodiment does not specifically limit the target detection model used, and can be flexibly set based on actual needs. For example, it can include a YOLO algorithm model, a Single Shot MultiBox Detector (SSD) algorithm model, and a Faster Region-Based Convolutional Neural Network (Faster R-CNN) algorithm model. The target detection result of the current image frame refers to the detection result output by the target detection model after the current image frame is input as input data to the trained target detection model.

[0033] In this embodiment, as an embodiment, the target detection results of any image frame may include, for example, N preset category labels (N is greater than or equal to 1), objects in the any image frame that belong to at least one preset category label, the confidence that each object matches at least one preset category label, the size of each object, and the position of each object, etc. This embodiment does not make specific limitations.

[0034] Here, belonging to a preset category label can be understood as being classified into the preset category label. Based on this, the confidence level of an object matching a preset category label can be understood as the degree of certainty that the object is attributed to the preset category label (or the degree of certainty that the object is classified into the preset category label).

[0035] The size of the object can be understood as the size of the object in the corresponding image frame. The position of the object can be understood as the position of the object in the corresponding image frame. In this embodiment, there are no specific limitations on the size and position of the object. For example, in a specific implementation, the position of the object can be represented by a bounding box, and the size of the object can be represented by the size of the bounding box; the bounding box here can be determined by the coordinate information of the object in the image.

[0036] As for how to train the target detection model in this step 102, an example will be given below and will not be described here.

[0037] In step 103, if it is determined that there is a suspected electric vehicle in the current image frame based on the target detection result corresponding to the current image frame, the image frames captured by the camera within the first specified time period are obtained; target detection is performed on the image frames within the first specified time period through the trained target detection model to obtain the target detection results corresponding to the image frames within the first specified time period.

[0038] In this embodiment, if it is determined that there is a suspected battery vehicle in the current image frame based on the target detection result corresponding to the current image frame, then in order to improve the accuracy of battery vehicle detection and avoid the false alarm of the battery vehicle caused by detection error, the image frames captured by the camera within the first specified time period can be obtained at this time, and the target detection model trained above is used to perform target detection on each image frame within the first specified time period to obtain the target detection result corresponding to each image frame within the first specified time period, so as to be used for the subsequent determination of whether there is a battery vehicle in the above-mentioned specified scene based on the target detection result corresponding to each image frame within the first specified time period. The first specified time period here refers to a period of time starting from the moment when the acquisition of the current image frame ends.

[0039] As for how to determine the suspected presence of an electric vehicle in the current image frame based on the target detection result corresponding to the current image frame in step 103, an example will be given below and will not be described here in detail.

[0040] Step 104 : If it is determined that an electric vehicle exists in the designated scene based on the target detection results corresponding to the image frames within the first designated time period, an electric vehicle alarm indication is output.

[0041] In this embodiment, after obtaining the target detection results corresponding to each image frame within the first specified time period, a determination can be made for each image frame within the first specified time period regarding whether a battery vehicle is present in the image frame based on the target detection results corresponding to the image frame. Subsequently, a determination can be made regarding whether a battery vehicle is present in the specified scene based on the determination results regarding whether a battery vehicle is present in the specified scene. If a battery vehicle is present in the specified scene, a battery vehicle alarm indication can be output. The specific method for determining the presence of a battery vehicle in the specified scene based on the target detection results corresponding to the image frames within the first specified time period will be described below with examples and will not be elaborated upon here.

[0042] As an embodiment, the above-mentioned output of the electric vehicle alarm indication can be implemented in many ways. For example, it can be a sound alarm method, such as triggering a buzzer to emit a loud alarm sound; or it can be a display screen prompt method, such as displaying a warning message such as "electric vehicle detected" on a display screen installed in a specified scene; or it can be a warning light alarm method, such as flashing a warning light installed in a specified scene to indicate that an electric vehicle has been detected; or it can be a combination of two or more of the above alarm methods, and so on.

[0043] So far, completed Figure 1 The process shown.

[0044] pass Figure 1As can be seen from the shown process, in the embodiment of the present application, the current image frame collected is firstly determined based on the target detection result corresponding to the current image frame to determine whether there is a suspected electric vehicle in the current image frame. Afterwards, when it is determined that there is a suspected electric vehicle in the current image frame, it is determined again whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each image frame collected within a period of time starting from the end time of acquisition of the current image frame to determine whether to output an electric vehicle alarm indication. This not only realizes the electric vehicle detection in the specified scene through automation, avoiding the situation of missed electric vehicle detection caused by manual methods, but also improves the accuracy of electric vehicle detection by secondary detection of whether there is an electric vehicle, effectively avoiding the situation of false alarm of electric vehicle.

[0045] The following describes how to train the target detection model:

[0046] As an embodiment, during the training stage of the target detection model, a training data set for training the target detection model is first prepared, and a label is established for each image in the training data set, such as by manual labeling; then, the target detection model is trained based on the prepared training data set. As for how to train the target detection model based on the training data set, this embodiment does not make any specific restrictions.

[0047] In this embodiment, the training data set includes at least a plurality of battery-powered bicycle images, a plurality of other vehicle images, and a plurality of human body images. This embodiment does not specifically limit the number of each type of image in the training data set. The other vehicles here refer to vehicles similar to the battery-powered bicycle model, such as at least baby carriages, bicycles, scooters, and motorcycles. This embodiment does not specifically limit them. Accordingly, the established labels may include battery-powered bicycle head labels, battery-powered bicycle body labels, battery-powered bicycle body labels, electric vehicle windshield labels, baby carriage labels, bicycle labels, scooter labels, motorcycle labels, and human body labels.

[0048] This embodiment enhances the target detection performance of the target detection model by adding images of other vehicles similar to the electric bicycle model to the training data set of the target detection model, and can effectively avoid interference from other vehicles similar to the electric bicycle model when detecting the electric bicycle.

[0049] As an embodiment, since the number of images corresponding to different preset category labels in the training data set is different, the more images a preset category label has, the better the target detection model's detection performance for objects belonging to the preset category label, such as higher detection rate and precision. Based on this, in order to avoid the difference in the target detection model's detection performance for objects with different preset category labels due to the different number of images corresponding to different preset category labels, a corresponding confidence factor can be set in advance for each preset category label based on the number of images corresponding to each preset category label in the training data set, so that when the trained target detection model outputs the target detection result in the application stage, the product of the confidence corresponding to the detected object belonging to each preset category label and the confidence factor corresponding to the preset category label is used as the final confidence to be output.

[0050] In this example, the detection rate, also known as the recall rate, refers to the ratio of the number of samples correctly identified as electric bicycles to the total number of electric bicycle samples. The precision rate, also known as the accuracy rate, refers to the ratio of the number of samples correctly identified as electric bicycles to the total number of samples identified as electric bicycles.

[0051] In this embodiment, there is no specific limitation on how to set the confidence factors corresponding to different preset category labels. For example, as an embodiment, the confidence factor corresponding to the preset category label with the largest number of images in the training data set can be set to 1, and the confidence factors corresponding to other preset category labels are based on the confidence factors corresponding to the preset category labels. For any other preset category label, the product of the confidence factor corresponding to the preset category label and the specified coefficient corresponding to the other preset category label is used as the confidence factor corresponding to the other preset category label; here, the specified coefficient corresponding to the other preset category label refers to the ratio between the number of images corresponding to the specified preset category label and the number of images corresponding to the preset category label in the training data set.

[0052] The following describes how to determine that a suspected battery vehicle exists in the current image frame based on the target detection result corresponding to the current image frame in step 103 above:

[0053] As an embodiment, based on the target detection results described in step 103 above, it can be seen that the target detection result corresponding to the current image frame indicates at least N preset category labels, objects in the current image frame that belong to at least one preset category label, and the confidence level that the objects match the at least one preset category label. Here, the N preset category labels include at least an electric vehicle body label. Furthermore, in practical applications, due to angle issues, the front or rear of an electric vehicle can easily be obscured after entering a designated scene. To avoid the problem of missing electric vehicles, the presence of an electric vehicle body can be detected in the image frame to determine whether an electric vehicle is suspected to be present in the current image frame. Based on this, there are many specific implementation methods for determining the suspected presence of an electric vehicle in the current image frame based on the target detection results corresponding to the current image frame. For example, in a specific implementation, the following method can be used: first, based on the target detection results corresponding to the current image frame, determine whether a set condition is met, wherein the set condition includes at least the presence of an object matching the electric vehicle body label in the current image frame; then, if the set condition is met, determine that an electric vehicle is suspected to be present in the current image frame; and, if the set condition is not met, determine that an electric vehicle is not present in the current image frame.

[0054] In this embodiment, the existence of an object matching the battery vehicle body label in the current image frame means: the existence of an object belonging to the battery vehicle body label in the current image frame, the confidence level of the object matching the battery vehicle body label is greater than the confidence level of the object matching other preset category labels other than the battery vehicle body label, and the confidence level of the object matching the battery vehicle body label is greater than or equal to a set confidence threshold. Here, the confidence threshold can be flexibly set based on actual needs, such as 80%, 85%, or 90%, and is not specifically limited in this embodiment.

[0055] Exemplarily, assuming that N preset category labels are electric bicycle body labels and bicycle labels, the target detection result of the current image frame is: object 1, object 1 belongs to the electric bicycle body label and the bicycle label, the confidence that object 1 matches the electric bicycle body label is 83%, and the confidence that object 1 matches the bicycle label is 35%; it can be seen that the confidence that object 1 matches the electric bicycle body label in the current image frame is greater than the confidence that object 1 matches the bicycle label, and the confidence that object 1 matches the electric bicycle body label is greater than the set confidence threshold of 80%. Therefore, based on the target detection result of the current image frame, it can be determined that there is an object matching the electric bicycle body label in the current image frame, that is, there is suspected to be an electric bicycle in the current image frame.

[0056] As an embodiment, in order to avoid the situation where pets, handheld objects, and electric bicycle posters are mistakenly reported as electric bicycles in a specified scene, the size of the object in the image frame can also be used to filter out objects in the specified scene that are easily mistakenly reported as electric bicycles. For example, a set size threshold can be preset to further identify the object that matches the electric bicycle body label in the current image frame. If the size of the object that matches the electric bicycle body label is greater than or equal to the set size threshold, it can be determined that there is a suspected electric bicycle in the current image frame. If the size of the object that matches the electric bicycle body label is less than the set size threshold, it can be determined that there is no electric bicycle in the current image frame.

[0057] As an embodiment, under normal circumstances, human participation is required for an electric vehicle to enter a designated scene. Therefore, after determining that there is an object matching the electric vehicle body label in the current image frame, it is possible to further determine whether there is an electric vehicle in the current image frame by detecting whether there is a human body in the designated scene. In other words, it is possible to further determine whether there is an electric vehicle in the current image frame by determining whether there is an object matching the human body label in the current image frame; if there is an object matching the human body label in the current image frame, it can be determined that there is a suspected electric vehicle in the current image frame, otherwise it is determined that there is no electric vehicle in the current image frame.

[0058] Based on the above description, the N preset category labels may also include a human body label; the target detection result also indicates the size of the object in the image frame that belongs to at least one preset category label. Based on this, as an embodiment, the above-mentioned set conditions may also include: the size of the object matching the battery vehicle body label in the current image frame is greater than or equal to a set size threshold, and / or, the existence of an object matching the human body label in the current image frame. As for how to determine whether there is an object matching the human body label in the current image frame, it is similar to the above-mentioned method of determining whether there is an object matching the battery vehicle body label in the current image frame, and will not be repeated here.

[0059] This embodiment provides a method for detecting whether an electric battery vehicle is suspected to exist in an image frame by combining a human body. The following describes a specific implementation of this method with an example:

[0060] See also Figure 2 As shown, the method may include the following steps when implemented:

[0061] Step 201: input the obtained current image frame into a trained target detection model to obtain a target detection result of the current image frame.

[0062] As for the specific content of the target detection result, please refer to the relevant description in the above step 102, which will not be repeated here.

[0063] Step 202 , determining whether there is an object belonging to the battery vehicle body tag in the current image frame based on the target detection result of the current image frame; if yes, executing step 203 ; if not, executing step 207 .

[0064] Step 203 , determine whether the confidence level of the object matching the battery vehicle body label is greater than the confidence level of the object matching other preset category labels except the battery vehicle body label; if yes, execute step 204 , if not, execute step 207 .

[0065] Step 204 , checking whether there is an object matching the human body label in the current image frame; if yes, executing step 205 ; if not, executing step 207 .

[0066] Step 205 , determining whether an electric vehicle is suspected to exist in the current image frame; and proceeding to step 206 .

[0067] Step 206, obtain each image frame captured by the camera within the first specified time period, and perform target detection on each image frame within the first specified time period through the trained target detection model to obtain the target detection results corresponding to each image frame within the first specified time period. If it is determined that there is an electric vehicle in the specified scene based on the target detection results corresponding to each image frame within the first specified time period, an electric vehicle alarm indication is output.

[0068] Step 207 : Set the next image frame of the current image frame as the current image frame, and return to step 201 .

[0069] The following describes how to determine whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each image frame within the first specified time period in the above step 104:

[0070] As an embodiment, the N preset category labels also include a battery vehicle front label and a battery vehicle rear label; based on this, there are many specific implementation methods for determining the presence of a battery vehicle in a specified scene based on the target detection results corresponding to each image frame within the first specified time period, such as referring to Figure 3 As shown, the specific implementation may include the following steps:

[0071] Step 301, for each image frame within the first specified time period, check whether there is an object matching the battery vehicle body label in the image frame based on the target detection result corresponding to the image frame; if yes, execute step 302, if not, execute step 303.

[0072] In this embodiment, the above-mentioned target detection result corresponding to the image frame is used to check whether there is an object matching the battery vehicle body label in the image frame, which is similar to the determination method of whether there is an object matching the battery vehicle body label in the current image frame, and will not be repeated here.

[0073] Step 302 , checking whether the object and the object matching the battery vehicle body label in the previous image frame meet the object similarity requirement. If so, executing step 304 .

[0074] In this embodiment, the object similarity requirement means that the distance between the positions of two objects meets the set distance requirement, and the difference between the sizes of the two objects meets the set difference requirement. Based on the object positions and object sizes described above, the distance between the positions of two objects can refer to the distance between the center point coordinates of the two objects (such as the center point coordinates of the bounding box representing the object positions); correspondingly, the size of the object can refer to the size of the bounding box representing the object positions.

[0075] This embodiment can pre-set the distance requirement and the difference requirement based on actual needs. For example, the distance requirement can be greater than or equal to a first distance threshold and less than or equal to a second distance threshold, where the first distance threshold is less than the second distance threshold. The first and second distance thresholds can be flexibly set based on actual needs and are not specifically limited in this embodiment. The difference requirement can be less than or equal to the difference threshold, where the difference threshold is also flexibly set based on actual needs and is not specifically limited in this embodiment.

[0076] In this embodiment, it is understandable that, for a moving object, the position of the object in different image frames (that is, the position of the object) will change slightly, while the size of the object in different image frames (that is, the size of the object) is basically unchanged. Based on this, after determining that there is an object matching the battery vehicle body label in the image frame, this embodiment further determines whether it is necessary to set a mark for the image frame by checking whether the object and the object matching the battery vehicle body label in the previous image frame meet the object similarity requirements. This can achieve tracking of the same object in different image frames (that is, target tracking), thereby avoiding stationary objects in specified scenes (such as battery vehicle posters, etc.) from being mistakenly reported as battery vehicles, thereby improving the accuracy of battery vehicle detection.

[0077] Step 303 , checking whether there is an object matching the front tag of the battery vehicle, or an object matching the rear tag of the battery vehicle in the image frame. If yes, executing step 304 .

[0078] Step 304 , set a mark for the image frame; then proceed to step 305 .

[0079] Step 305 : When it is determined that the battery vehicle detection requirement is met based on each image frame with a mark, it is determined that there is a battery vehicle in the designated scene.

[0080] As for how to determine whether the battery vehicle detection requirements are met based on each marked image frame, an example will be given below and will not be elaborated here.

[0081] In this embodiment, the target detection result also indicates the position of an object in the image frame that belongs to at least one preset category label. Based on this, as an embodiment, when it is checked that there is no object matching the front label of the electric vehicle and the object matching the rear label of the electric vehicle in the image frame, the method further includes: if the object belonging to the electric vehicle body label in the image frame is less than a set confidence threshold, at this time, in order to avoid the situation where the object belonging to the electric vehicle body label in the image frame is less than the set confidence threshold due to the increase in the range of the electric vehicle blocked by the human body, it can be further checked whether there is an object matching the human body label in the image frame, and whether the first overlapping range is larger than the second overlapping range. When it is checked that there is an object matching the human body label in the image frame, and the first overlapping range is larger than the second overlapping range, it can be indicated that there is an object matching the electric vehicle in the image frame, but it is just that the corresponding confidence is too low due to the excessive range blocked by the human body, so a mark can be set for the image frame.

[0082] The first overlapping range refers to the overlapping range between the position of the object matching the human body tag in the image frame and the position of the object belonging to the battery vehicle body tag. The second overlapping range refers to the overlapping range between the position of the object matching the human body tag in the image frame before the image frame and the position of the reference object.

[0083] The reference object here refers to an object that meets the object similarity requirements between the object matching the battery vehicle body label in the previous image frame of the image frame and the object belonging to the battery vehicle body label in the image frame. That is, the reference object and the object belonging to the battery vehicle body label in the image frame belong to the same object in different image frames.

[0084] Here, the overlapping range between the positions of the two objects can be the overlapping range between the bounding boxes corresponding to the two objects; as for how to specifically obtain the overlapping range between the bounding boxes corresponding to the two objects, for example, the overlapping range between the bounding boxes corresponding to the two objects can be determined based on the coordinate information corresponding to the bounding boxes corresponding to the two objects, which is not specifically limited in this embodiment.

[0085] The following describes how to determine whether the battery vehicle detection requirements are met based on each image frame with a mark set:

[0086] In this embodiment, there are many specific implementation methods for determining whether the electric vehicle detection requirements are met based on the image frames with marks set. For example, as an embodiment, whether the electric vehicle detection requirements are met is determined based on the number of image frames with marks set and the number of image frames within the first specified time period.

[0087] In this embodiment, as an embodiment, whether the battery vehicle detection requirements are met is determined based on the number of image frames with marks and the number of image frames within the first specified time period. In a specific implementation, for example, the ratio between the number of image frames with marks and the number of image frames within the first specified time period is first calculated, and then it is determined whether the ratio is greater than or equal to a set ratio threshold. If so, it is determined that the battery vehicle detection requirements are met. If not, it is determined that the battery vehicle detection requirements are not met. The set ratio threshold here can be flexibly set based on actual needs, for example, it can be 60%, 65%, 70%, etc., and this embodiment does not specifically limit it.

[0088] As another embodiment, whether the battery vehicle detection requirement is met is determined based on the highest confidence level corresponding to the object matching the battery vehicle body label in each marked image frame.

[0089] In this embodiment, as an embodiment, whether the battery vehicle detection requirements are met is determined based on the highest confidence level corresponding to the object that matches the battery vehicle body label in each marked image frame. In a specific implementation, for example, it can be: first, at least two image frames are selected from each marked image frame, and then, the average of the highest confidence levels corresponding to the objects that match the battery vehicle body label in each selected image frame is calculated. Finally, it is determined whether the average is greater than or equal to the set average threshold. If so, it is determined that the battery vehicle detection requirements are met. If not, it is determined that the battery vehicle detection requirements are not met. The set average threshold here can be flexibly set based on actual needs, for example, it can be 85%, 90%, 95%, etc., and this embodiment does not make specific limitations.

[0090] In this embodiment, there are many specific implementation methods for selecting at least two image frames from each marked image frame, such as, taking all marked image frames as the selected image frames; or, selecting at least two image frames before or after the acquisition start time from all marked image frames; or, selecting at least two image frames with the highest confidence level corresponding to the object matching the battery vehicle body label in the image frames from all marked image frames, and so on.

[0091] The following describes when to trigger the stop of outputting battery car alarm indication after outputting battery car alarm indication:

[0092] As an embodiment, there are many specific implementation methods for triggering the stop of outputting the battery vehicle alarm indication, for example, first obtaining each image frame captured by the camera within a second specified time period; here, the second specified time period refers to a period of time starting from the start time of outputting the battery vehicle alarm indication;

[0093] Then, each image frame acquired within the second specified time period is used as a reference image frame;

[0094] Finally, the trained target detection model is used to perform target detection on each reference image frame to obtain the target detection results corresponding to each reference image frame, and then it is determined whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each reference image frame;

[0095] Among them, if it is determined that there is no electric bicycle in the specified scene, the output of the electric bicycle alarm indication is stopped; if it is determined that there is an electric bicycle in the specified scene, the image frames captured by the camera within a period of time starting from the target moment are used as reference image frames, and the step of performing target detection on each reference image frame through the trained target detection model to obtain the target detection results corresponding to each reference image frame is returned; the target moment here refers to the latest acquisition end time among the acquisition end times of each reference image frame.

[0096] It should be noted that how to determine whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each reference image frame is similar to how to determine whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each image frame within the first specified time period. For example, as an embodiment, determining whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each reference image frame may include: first, for each reference image frame, if it is determined based on the target detection result corresponding to the reference image frame that there is an object matching the electric vehicle body label in the reference image frame, then checking whether the object and the object matching the electric vehicle body label in the previous image frame meet the object similarity requirement, and if so, setting a mark for the reference image frame; then, if it is determined based on the target detection result corresponding to the reference image frame that there is no object matching the electric vehicle body label in the reference image frame, then when it is checked that there is an object matching the electric vehicle front label or the electric vehicle rear label in the reference image frame, setting a mark for the reference image frame; finally, determining whether there is an electric vehicle in the specified scene based on each reference image frame that is marked.

[0097] The following describes the collection of training data sets for optimizing the target detection model after outputting the battery vehicle alarm indication:

[0098] As an embodiment, after outputting the battery vehicle alarm indication, each image frame captured by the camera within the third specified time period is saved in a specified storage space for subsequent use in optimizing the target detection model.

[0099] Here, the third designated time period refers to a period starting from and ending at a time when the battery vehicle alarm indication is output as a reference time.

[0100] The above-mentioned designated storage space may refer to a storage space in the local memory or a storage space on the cloud, which is not specifically limited in this embodiment.

[0101] The following describes the processing of the image frames captured by the camera before performing target detection on the image frames captured by the camera:

[0102] As an embodiment, in response to the situation where the objects in the captured image frames are small due to the installation angle and installation height of the camera, when there is an electric vehicle in the specified scene, the size of the object in the image frame that matches the electric vehicle label will be smaller than the set size threshold, and it will be considered that there is no electric vehicle in the specified scene, which will cause the problem of missing reports of electric vehicles. In order to solve this problem, in this embodiment, for the image frame captured when the camera is initially used, first, the rotation angle and size of the specified object in the image frame are obtained, and it is determined whether the difference between the rotation angle of the specified object in the image frame and the set angle is less than a first threshold, and whether the difference between the size of the specified object in the image frame and the set size is less than a second threshold; here, the first threshold and the second threshold can be flexibly set based on actual needs, and this embodiment does not make specific limitations.

[0103] Afterwards, if yes, it means that the image frame captured by the camera does not need to be adjusted; if no, it means that the image frame captured by the camera may cause the electric vehicle to be missed when used for electric vehicle detection, so the image frame captured by the camera needs to be adjusted to avoid this situation. For example, the image frame can be processed with resolution, image rotation, and image scaling. As for how to perform processing such as resolution, image rotation, and image scaling on the image frame, this embodiment does not make any specific restrictions.

[0104] This completes the description of the method provided in the embodiment of the present application. The following describes the device provided in the embodiment of the present application:

[0105] See also Figure 4 , Figure 4 This is a schematic diagram of the device structure provided in the embodiment of the present application. Figure 4 As shown, the battery vehicle detection device 400 includes:

[0106] The acquisition module 401 is used to acquire the current image frame captured by the camera installed in the specified scene; the specified scene is set to prohibit battery vehicles from entering;

[0107] A detection module 402 is configured to perform target detection on the current image frame using a trained target detection model to obtain a target detection result corresponding to the current image frame;

[0108] Determination module 403 is configured to, if it is determined based on the target detection result corresponding to the current image frame that an electric bicycle is suspected to exist in the current image frame, obtain each image frame captured by the camera within a first specified time period; the first specified time period is a period of time starting from the moment when the capture of the current image frame ends; perform target detection on each image frame within the first specified time period using a trained target detection model to obtain target detection results corresponding to each image frame within the first specified time period;

[0109] The alarm module 404 is configured to output an electric vehicle alarm indication if it is determined that an electric vehicle exists in the specified scene based on the target detection results corresponding to each image frame within the first specified time period.

[0110] In one embodiment, the target detection result indicates at least N preset category labels, an object in the image frame that belongs to at least one preset category label, and a confidence level that the object matches the at least one preset category label; N is greater than or equal to 1; and the N preset category labels include at least an electric vehicle body label;

[0111] Determining, based on the target detection result corresponding to the current image frame, that an electric battery vehicle is suspected to exist in the current image frame includes: determining, based on the target detection result corresponding to the current image frame, whether a set condition is satisfied; and if so, determining that an electric battery vehicle is suspected to exist in the current image frame; wherein the set condition at least includes: the presence of an object in the current image frame that matches a body label of the electric battery vehicle;

[0112] The existence of an object that matches the battery vehicle body label in the current image frame means: there is an object belonging to the battery vehicle body label in the current image frame, the confidence that the object matches the battery vehicle body label is greater than the confidence that the object matches other preset category labels except the battery vehicle body label, and the confidence that the object matches the battery vehicle body label is greater than or equal to the set confidence threshold.

[0113] As an embodiment, the N preset category labels further include a human body label; the target detection result further indicates the size of an object in the image frame that belongs to at least one preset category label;

[0114] The set conditions also include: the size of the object matching the battery vehicle body label in the current image frame is greater than or equal to a set size threshold, and / or, there is an object matching the human body label in the current image frame.

[0115] As an embodiment, the N preset category labels further include a battery vehicle front label and a battery vehicle rear label;

[0116] Determining the presence of an electric vehicle in a specified scene based on target detection results corresponding to each image frame within a first specified time period includes:

[0117] For each image frame within the first specified time period, if, based on the target detection result corresponding to the image frame, it is determined that an object matching the battery vehicle body label exists in the image frame, then checking whether the object meets the object similarity requirement with the object matching the battery vehicle body label in the previous image frame, and if so, setting a mark for the image frame;

[0118] If, based on the target detection result corresponding to the image frame, it is determined that there is no object matching the battery vehicle body label in the image frame, then when it is detected that there is an object matching the battery vehicle front label or an object matching the battery vehicle rear label in the image frame, a mark is set for the image frame;

[0119] When it is determined that the battery vehicle detection requirement is met based on each image frame with a mark, it is determined that an battery vehicle exists in the designated scene.

[0120] As an embodiment, the object detection result further indicates the location of an object belonging to at least one preset category label in the image frame;

[0121] When it is detected that there is no object matching the front label of the battery vehicle and no object matching the rear label of the battery vehicle in the image frame, the method further includes:

[0122] If the number of objects in the image frame that belong to the battery vehicle body label is less than the set confidence threshold, then when it is detected that there is an object matching the human body label in the image frame and the first overlapping range is larger than the second overlapping range, a mark is set for the image frame;

[0123] Among them, the first overlapping range refers to the overlapping range between the position of the object matching the human body label in the image frame and the position of the object belonging to the battery vehicle body label; the second overlapping range refers to the overlapping range between the position of the object matching the human body label in the previous image frame of the image frame and the position of the reference object; the reference object refers to the object that meets the object similarity requirements between the object matching the battery vehicle body label in the previous image frame of the image frame and the object belonging to the battery vehicle body label in the image frame.

[0124] As an embodiment, determining whether the battery vehicle detection requirements are met based on each marked image frame includes:

[0125] Determining whether the battery vehicle detection requirements are met based on the number of image frames marked and the number of image frames within the first specified time period; or,

[0126] Whether the battery vehicle detection requirements are met is determined based on the highest confidence level corresponding to the object that matches the battery vehicle body label in each marked image frame.

[0127] As an embodiment, after outputting the battery vehicle alarm indication, the device further includes:

[0128] a stop module, configured to obtain image frames captured by the camera within a second specified time period; the second specified time period is a period starting from the start time of outputting the battery vehicle alarm indication; and use the image frames acquired within the second specified time period as reference image frames;

[0129] Perform target detection on each reference image frame using the trained target detection model to obtain target detection results corresponding to each reference image frame, and determine whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each reference image frame;

[0130] If yes, stop outputting the electric vehicle alarm indication; if not, use the image frames captured by the camera within a period of time starting from the target time as reference image frames, and return to the step of performing target detection on each reference image frame through the trained target detection model to obtain the target detection results corresponding to each reference image frame; wherein, the target time refers to the latest acquisition end time among the acquisition end times of each reference image frame.

[0131] As an embodiment, determining whether an electric vehicle exists in a specified scene based on the target detection results corresponding to each reference image frame includes:

[0132] For each reference image frame, if it is determined based on the target detection result corresponding to the reference image frame that there is an object matching the battery vehicle body label in the reference image frame, then check whether the object meets the object similarity requirement with the object matching the battery vehicle body label in the previous image frame. If so, set a mark for the reference image frame;

[0133] If, based on the target detection result corresponding to the reference image frame, it is determined that there is no object matching the battery vehicle body label in the reference image frame, then when it is detected that there is an object matching the battery vehicle front label or an object matching the battery vehicle rear label in the reference image frame, a mark is set for the reference image frame;

[0134] It is determined whether there is an electric vehicle in the specified scene based on each reference image frame with a mark set.

[0135] So far, completed Figure 4Structural description of the device shown.

[0136] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0137] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present application scheme. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0138] See Figure 5 , is a schematic diagram of the hardware structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device may include a processor 501, a communication interface 502, a computer-readable storage medium 503, and a communication bus 504. The processor 501, the communication interface 502, and the computer-readable storage medium 503 communicate with each other via the communication bus 504. The computer-readable storage medium 503 stores computer program instructions; the processor 501 can execute the steps of the method described in the above embodiment by executing the computer program instructions stored on the computer-readable storage medium 503. The electronic device may also include other hardware according to the actual function of the electronic device, which will not be described in detail.

[0139] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium, on which a number of computer program instructions are stored. When the computer program instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0140] Exemplarily, the computer-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, and the like. For example, the computer-readable storage medium may be RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof. The processor and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0141] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A battery vehicle detection method, characterized in that: The method comprises: Acquire a current image frame captured by a camera installed in a specified scene; the specified scene is set to prohibit battery vehicles from entering; Performing target detection on the current image frame using a trained target detection model to obtain a target detection result corresponding to the current image frame; If it is determined that there is a suspected battery vehicle in the current image frame based on the target detection result corresponding to the current image frame, then each image frame captured by the camera within a first specified time period is obtained; the first specified time period refers to a period of time starting from the end time of the capture of the current image frame; target detection is performed on each image frame within the first specified time period using a trained target detection model to obtain target detection results corresponding to each image frame within the first specified time period; If it is determined that there is an electric battery vehicle in the designated scene according to the target detection results corresponding to each image frame within the first designated time period, an electric battery vehicle alarm indication is output.

2. The method according to claim 1, characterized in that The target detection result at least indicates N preset category labels, an object in the image frame that belongs to at least one preset category label, and a confidence level that the object matches at least one preset category label; N is greater than or equal to 1; The N preset category labels include at least a battery vehicle body label; The step of determining that a battery vehicle is suspected to exist in the current image frame based on the target detection result corresponding to the current image frame comprises: determining whether a set condition is met based on the target detection result corresponding to the current image frame, and if so, determining that a battery vehicle is suspected to exist in the current image frame; wherein the set condition at least includes: the existence of an object matching the battery vehicle body label in the current image frame; The existence of an object matching the battery vehicle body label in the current image frame means: there is an object belonging to the battery vehicle body label in the current image frame, the confidence that the object matches the battery vehicle body label is greater than the confidence that the object matches other preset category labels except the battery vehicle body label, and the confidence that the object matches the battery vehicle body label is greater than or equal to the set confidence threshold.

3. The method according to claim 2, characterized in that The N preset category labels also include a human body label; the target detection result also indicates the size of an object belonging to at least one preset category label in the image frame; The set conditions also include: the size of the object matching the battery vehicle body label in the current image frame is greater than or equal to a set size threshold, and / or there is an object matching the human body label in the current image frame.

4. The method according to claim 2, characterized in that: The N preset category labels also include a battery vehicle front label and a battery vehicle rear label; The determining that there is a battery vehicle in the specified scene according to the target detection results corresponding to each image frame within the first specified time period includes: For each image frame within the first specified time period, if it is determined based on the target detection result corresponding to the image frame that there is an object matching the battery vehicle body label in the image frame, then check whether the object and the object matching the battery vehicle body label in the previous image frame meet the object similarity requirement, and if so, set a mark for the image frame; If it is determined based on the target detection result corresponding to the image frame that there is no object matching the battery vehicle body label in the image frame, then when it is checked that there is an object matching the battery vehicle front label in the image frame, or there is an object matching the battery vehicle rear label, a mark is set for the image frame; When it is determined that the battery vehicle detection requirement is met based on each image frame set with the mark, it is determined that there is a battery vehicle in the designated scene.

5. The method according to claim 4, characterized in that The target detection result also indicates the location of an object belonging to at least one preset category label in the image frame; When it is checked that there is no object matching the front label of the battery vehicle and no object matching the rear label of the battery vehicle in the image frame, the method further includes: If the number of objects in the image frame that belong to the battery vehicle body label is less than the set confidence threshold, then when it is detected that there is an object matching the human body label in the image frame and the first overlapping range is larger than the second overlapping range, a mark is set for the image frame; Among them, the first overlapping range refers to the overlapping range between the position of the object matching the human body label in the image frame and the position of the object belonging to the battery vehicle body label; the second overlapping range refers to the overlapping range between the position of the object matching the human body label in the previous image frame of the image frame and the position of the reference object; the reference object refers to an object that meets the object similarity requirements between the object matching the battery vehicle body label in the previous image frame of the image frame and the object belonging to the battery vehicle body label in the image frame.

6. The method according to claim 4, characterized in that Determining whether the battery vehicle detection requirement is met according to each image frame on which the mark is set includes: Determining whether the battery vehicle detection requirement is met based on the number of image frames with the mark set and the number of image frames in the first specified time period; or, Whether the battery vehicle detection requirement is met is determined based on the highest confidence level corresponding to the object matching the battery vehicle body label in each image frame set with the mark.

7. The method according to claim 4, characterized in that After outputting the battery vehicle alarm indication, the method further comprises: Acquire each image frame collected by the camera within a second specified time period; the second specified time period refers to a period of time starting from the start time of outputting the battery vehicle alarm indication; and use each image frame acquired within the second specified time period as a reference image frame; Performing target detection on each reference image frame using a trained target detection model to obtain target detection results corresponding to each reference image frame, and determining whether there is an electric vehicle in the specified scene based on the target detection results corresponding to each reference image frame; If yes, stop outputting the battery vehicle alarm indication; if no, use the image frames captured by the camera within a period of time starting from the target time as reference image frames, and return to the step of performing target detection on each reference image frame through the trained target detection model to obtain the target detection results corresponding to each reference image frame; wherein the target time refers to the latest acquisition end time among the acquisition end times of each reference image frame.

8. The method according to claim 7, characterized in that The step of determining whether there is an electric vehicle in the specified scene according to the target detection results corresponding to each reference image frame includes: For each reference image frame, if it is determined that there is an object matching the battery vehicle body label in the reference image frame based on the target detection result corresponding to the reference image frame, then check whether the object and the object matching the battery vehicle body label in the previous image frame meet the object similarity requirement, and if so, set a mark for the reference image frame; If it is determined based on the target detection result corresponding to the reference image frame that there is no object matching the battery vehicle body label in the reference image frame, then when it is checked that there is an object matching the battery vehicle front label in the reference image frame, or there is an object matching the battery vehicle rear label, a mark is set for the reference image frame; It is determined whether there is a battery vehicle in the designated scene according to each reference image frame on which the mark is set.

9. A battery vehicle detection device, characterized in that: The device comprises: An acquisition module, used to acquire a current image frame captured by a camera installed in a specified scene; the specified scene is set to prohibit battery vehicles from entering; A detection module, used to perform target detection on the current image frame using a trained target detection model to obtain a target detection result corresponding to the current image frame; A determination module, for, if it is determined that a battery vehicle is suspected to exist in the current image frame based on the target detection result corresponding to the current image frame, obtaining each image frame captured by the camera within a first specified time period; the first specified time period refers to a period of time starting from the end time of the capture of the current image frame; performing target detection on each image frame within the first specified time period through a trained target detection model to obtain the target detection results corresponding to each image frame within the first specified time period; The alarm module is used to output an electric battery vehicle alarm indication if it is determined that there is an electric battery vehicle in the specified scene according to the target detection results corresponding to each image frame within the first specified time period.

10. An electronic device, characterized in that: The electronic device includes: Processor; and A computer-readable storage medium, wherein computer program instructions are stored in the computer-readable storage medium, and when the computer program instructions are executed by the processor, the processor is caused to execute the steps in any one of the methods of claims 1 to 8.

Images