Electricity consumption detection method, device and electronic equipment
By setting early warning areas in public places, using target recognition and pedestrian tracking algorithms to monitor sockets and pedestrians in real time, the problems of traditional low monitoring accuracy and high labor costs are solved, and high accuracy automatic detection of pedestrian illegal electricity use and Calling the police reduces the pressure on supervision.
Patent Information
- Application Number
- CN202210640017.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-08
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-06-08
AI Technical Summary
In public places, such as high-speed rail stations and subway stations, traditional human-made video surveillance inspection of pedestrian illegal electricity use problems with reduced accuracy and high labor costs, and there is a lack of relevant research based on deep learning algorithms, resulting in the accuracy of pedestrian illegal electricity use detection. Not high.
By obtaining the location of the surveillance camera and socket, setting the warning area, using the target recognition model to identify the socket usage status and pedestrian characteristics information, and combining the pedestrian tracking algorithm, the use of the socket is monitored in real time, and alarm information is generated, so that the pedestrian stay time reaches the preset Send alerts during the duration to automatically identify and record illegal use.
Real-time monitoring of the use of sockets in public places is realized, and the sockets and users who are used illegally are automatically discovered, reducing the burden on socket supervisors, and improving management effectiveness and accuracy.
Smart Images

Figure CN114973094B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to an electricity usage recognition method, device and electronic equipment. Background Art
[0002] In public places such as high-speed rail stations and subway stations, many sockets are set up specifically for use by specific equipment such as cleaning machines. Illegal use will cause safety hazards and need to be monitored. Traditional manual video monitoring to check pedestrians' illegal electricity use has great limitations. When the monitor observes multiple monitors at the same time, the monitoring accuracy will continue to decline, and it will be extremely energy-consuming and labor-intensive, with high labor costs.
[0003] Object detection is an important branch of computer vision and digital image processing, and is also the core of intelligent monitoring systems. Abnormal behavior recognition based on object detection and tracking is also an important part of the security monitoring field. Currently, there is little research on pedestrian illegal electricity use detection, especially the lack of relevant research based on deep learning algorithms. Moreover, it only judges whether illegal electricity use is based on the status of the person, which is not very accurate. Summary of the invention
[0004] The purpose of the present invention is to provide an electricity usage detection method that can monitor the usage of sockets in public places in real time, automatically discover sockets that are used in violation of regulations and people who use the sockets, reduce the burden on socket supervisors, and enhance the socket management effect in public places with high accuracy.
[0005] To achieve the above object, the present invention provides a method for detecting power consumption, comprising the following steps:
[0006] Obtain the locations of each surveillance camera and socket in the scene to be detected, and set the warning area according to the locations of each surveillance camera and socket;
[0007] Input the images captured by the surveillance camera into the trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area;
[0008] When the socket in the warning area is in use, the appearance feature information of the pedestrian in the warning area is extracted;
[0009] Input appearance feature information of pedestrians in the warning area into a preset pedestrian tracking algorithm, track the pedestrians in the warning area, and obtain trajectory information of each pedestrian and the stay time of each pedestrian in the warning area;
[0010] When the pedestrian stays in the warning area for a preset alarm duration, an alarm message is generated and sent.
[0011] Optionally, pedestrians in the warning area are tracked to obtain trajectory information of each pedestrian and the stay time of each pedestrian in the warning area; specifically including:
[0012] Input the appearance feature information of the pedestrian in the current frame into the pedestrian tracking algorithm;
[0013] The pedestrian tracking algorithm matches the appearance feature information of the pedestrian in the current frame with the trajectory information of multiple frames stored previously, generates the matching result and the trajectory information of the current frame, and stores the trajectory information of the current frame;
[0014] When the matching result is consistent, the pedestrian's stay time is accumulated.
[0015] Optionally, the power consumption detection method further includes:
[0016] When the pedestrian stays in the warning area for less than a preset first time period, it is determined that the pedestrian is in a normal state;
[0017] When the pedestrian's stay time in the warning area reaches a preset first duration and is less than a preset alarm duration, the pedestrian is determined to be in a stranded state;
[0018] When the pedestrian stays in the warning area for a preset alarm duration, it is determined that the pedestrian is in a wandering state.
[0019] Optionally, for a pedestrian in a stranded state, when the tracking algorithm loses the pedestrian, the length of time the pedestrian is lost is recorded;
[0020] When the lost time exceeds the preset second time, the track information and the stay time of the pedestrian are deleted; otherwise, the track information of the pedestrian continues to be stored and the stay time of the pedestrian is accumulated.
[0021] Optionally, when a socket in the warning area changes from a used state to an unused state, the unused time of the socket is recorded;
[0022] If the unused time of the socket exceeds a preset third time period, the trajectory information and stay time of all pedestrians are deleted; otherwise, the trajectory information of each pedestrian continues to be stored and the stay time of each pedestrian is accumulated.
[0023] Optionally, the power consumption detection method further includes:
[0024] Every fourth preset time period, the cosine similarity of all trajectory information is calculated;
[0025] When the cosine similarity of multiple track information is greater than a first threshold, it is determined that the multiple track information belongs to the same pedestrian;
[0026] Among multiple trajectory information belonging to the same pedestrian, the earliest one generated is saved and the others are deleted.
[0027] Optionally, the generating of the alarm information specifically includes marking the image of the used socket and the image of the pedestrian using the socket with a rectangular frame.
[0028] The present invention also provides an electricity consumption detection device, comprising:
[0029] A setting module is used to obtain the positions of each surveillance camera and socket in the scene to be detected, and set the warning area according to the positions of each surveillance camera and socket;
[0030] The recognition module is used to input the images captured by the surveillance camera into the trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area;
[0031] An extraction module, used to extract appearance feature information of pedestrians in the warning area when the socket in the warning area is in use;
[0032] A tracking module, used to input the appearance feature information of the pedestrians in the warning area into a preset pedestrian tracking algorithm, track the pedestrians in the warning area, and obtain the trajectory information of each pedestrian and the stay time of each pedestrian in the warning area;
[0033] The alarm module is used to generate and send an alarm message when the pedestrian's stay time in the warning area reaches a preset alarm duration.
[0034] The present invention also provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0035] Beneficial effects of the present invention: The present invention provides an electricity usage detection method, comprising the following steps: obtaining the positions of each monitoring camera and socket in a scene to be detected, and setting a warning area according to the positions of each monitoring camera and socket; inputting the images captured by the monitoring camera into a trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area; when the sockets in the warning area are in use, extracting the appearance feature information of the pedestrians in the warning area; inputting the appearance feature information of the pedestrians in the warning area into a preset pedestrian tracking algorithm, tracking the pedestrians in the warning area, and obtaining the trajectory information of each pedestrian and the stay time of each pedestrian in the warning area; when the stay time of the pedestrian in the warning area reaches a preset alarm duration, generating and sending an alarm message, and monitoring the illegal use of the sockets by combining the identification of the sockets and the pedestrians, it is possible to monitor the use of the sockets in public places in real time, automatically discover the sockets that are used illegally and the people who use the sockets, reduce the burden on socket supervisors, and enhance the socket management effect in public places. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings are only provided for reference and illustration and are not intended to limit the present invention.
[0037] In the accompanying drawings,
[0038] Figure 1 is a flow chart of the power consumption detection method of the present invention;
[0039] Figure 2 A flowchart of pedestrian detention determination in the power consumption detection method of the present invention;
[0040] Figure 3 It is a flow chart of the power consumption detection method of the present invention when a stranded pedestrian leaves;
[0041] Figure 4 It is a flow chart of the socket state change in the power consumption detection method of the present invention;
[0042] Figure 5 It is a flow chart of the track arrangement in the power consumption detection method of the present invention;
[0043] Figure 6 is a schematic diagram of the power consumption detection device of the present invention;
[0044] Figure 7 Schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION
[0045] To further illustrate the technical means and effects of the present invention, the following is a detailed description in conjunction with the preferred embodiments of the present invention and the accompanying drawings.
[0046] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.
[0047] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.
[0048] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0049] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0050] See also Figure 1 The present invention provides a method for detecting electricity consumption, comprising the following steps:
[0051] Step S1: Obtain the positions of each surveillance camera and socket in the scene to be detected, and set a warning area according to the positions of each surveillance camera and socket.
[0052] Specifically, according to needs, the scene to be detected described in the present invention can be a public place such as a subway station, a railway station and a shopping mall. The monitoring camera can be the original arrangement in the scene to be detected, or can be newly added for power consumption detection. The socket is a socket used for specific purposes in the scene to be detected and cannot be used by the public, such as a socket used for cleaning equipment in a subway station.
[0053] Furthermore, the warning area needs to be defined near the socket. Each warning area may include one or more sockets. The warning area should not be too large to avoid detecting too much invalid data. In some embodiments of the present invention, the warning area can be set to a range of 2m×2m around the socket.
[0054] Specifically, according to the needs, in step S1 of the present invention, specific parameter values of the alarm duration, the first duration, the second duration and the third duration to be used subsequently will be set.
[0055] Step S2: Input the images captured by the surveillance camera into the trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area.
[0056] Specifically, in step S2, a large number of socket and pedestrian samples of the scene to be detected are taken as training samples to complete the training of the target recognition model, and the specific model type of the target recognition model can be selected as needed.
[0057] Furthermore, in step S2, the trained target recognition model is used to perform target recognition on the image captured by the surveillance camera to obtain recognition results, which include socket and pedestrian recognition results. The socket recognition results should be divided into two categories: unused sockets and used sockets. The pedestrian recognition result outputs the detection frame of all pedestrians within the warning range, that is, the use status of the sockets in the warning area and the pedestrians in the warning area as described in step S2 are identified.
[0058] Optionally, when the target recognition model performs target recognition on the picture taken by the surveillance camera, it will first extract the image to be detected from the picture taken by the surveillance camera. The step of extracting the image to be detected can be performed by extracting each frame of the picture taken by the surveillance camera, or by extracting a frame of the picture taken by the surveillance camera every few frames for real-time detection.
[0059] Step S3: When the sockets in the warning area are in use, the appearance feature information of the pedestrians in the warning area is extracted.
[0060] Specifically, in step S3, if the socket in the warning area is in use, the pedestrian detection frame of the current frame is used to input the feature extraction model to obtain the appearance feature information of all detected pedestrians.
[0061] Step S4: input the appearance feature information of the pedestrians in the warning area into a preset pedestrian tracking algorithm, track the pedestrians in the warning area, and obtain the trajectory information of each pedestrian and the stay time of each pedestrian in the warning area.
[0062] Specifically, the pedestrians in the warning area are tracked to obtain the trajectory information of each pedestrian and the stay time of each pedestrian in the warning area; specifically including:
[0063] Input the appearance feature information of the pedestrian in the current frame into the pedestrian tracking algorithm;
[0064] The pedestrian tracking algorithm matches the appearance feature information of the pedestrian in the current frame with the trajectory information of multiple frames stored previously, generates the matching result and the trajectory information of the current frame, and stores the trajectory information of the current frame;
[0065] When the matching result is consistent, the pedestrian's stay time is accumulated.
[0066] Furthermore, step S4 uses a pedestrian tracking algorithm to match the appearance feature information of the pedestrians with the previously retained trajectory information. For pedestrian detection frames that can match the previous trajectory information, their stay time is accumulated and their new trajectory information of the current frame is retained again.
[0067] Step S5: When the pedestrian's stay time in the warning area reaches a preset alarm duration, an alarm message is generated and sent.
[0068] Specifically, in step S5, if the stay time of a pedestrian detected in the current frame reaches the preset alarm duration, the image information of the current frame and the coordinate information of the pedestrian who has reached the preset alarm duration are obtained, and the pedestrian who has reached the preset alarm duration is marked with a rectangular frame in the image. The above alarm result is sent to the monitoring user terminal, and a prompt alarm is triggered in the designated area, that is, the generated alarm information specifically includes marking the image of the used socket and the image of the pedestrian who uses the socket with a rectangular frame.
[0069] Specifically, see Figure 2 ,Since the scenes to be detected are usually public places with many pedestrians, even if a smaller warning area is demarcated around the socket, there are still a large number of pedestrians passing through the area and recognized by the target recognition model, if all the pedestrian trajectory information is saved, a large amount of invalid data will be generated, resulting in improper utilization of the storage device’s resources, affecting the algorithm’s operation effect.
[0070] Therefore, the present invention also limits the state of pedestrians, and sets a first duration to work together with the alarm duration to divide the state of pedestrians into three states: normal, detained and wandering.
[0071] Specifically, when the pedestrian's stay time in the warning area is less than the preset first duration, the pedestrian is determined to be in a normal state; when the pedestrian's stay time in the warning area reaches the preset first duration and is less than the preset alarm duration, the pedestrian is determined to be in a stranded state; when the pedestrian's stay time in the warning area reaches the preset alarm duration, the pedestrian is determined to be in a wandering state.
[0072] For pedestrians in normal state, when the tracking algorithm loses the pedestrian, the pedestrian’s trajectory information and stay time will be directly deleted to avoid the generation of invalid data.
[0073] For pedestrians in a stranded state, Figure 3 As shown, when the tracking algorithm loses the pedestrian, the duration of the pedestrian's loss is recorded; when the loss duration exceeds the preset second duration, the pedestrian's trajectory information and stay time are deleted, otherwise the pedestrian's trajectory information continues to be stored and the pedestrian's stay time is accumulated to avoid directly deleting the trajectory information when the pedestrian in the lingering state is blocked or leaves for a short time, resulting in the pedestrian's violation not being identified in a timely manner.
[0074] That is to say, for pedestrians who are in a state of detention, when the pedestrians leave during their stay, that is, when they are not in the warning range, all the trajectory information of the pedestrians cannot be directly deleted, because these pedestrians may return again after a short leave. At this time, it is necessary to accumulate the time when these pedestrians leave, that is, record the length of time the pedestrian is missing, and explain the two possible situations. Situation 1: When the pedestrian leaves the warning area for no more than the second time and is detected in the warning area again, all the previously stored trajectory information of the pedestrian will not be deleted, and new trajectory information will continue to be stored on this basis. It should be noted that in this case, the calculation of the stay time of the pedestrian who returns again is cumulative, including the time when the pedestrian leaves the warning area. Situation 2: When the pedestrian leaves the warning area for more than the second time, all the stored trajectory information of the pedestrian is deleted. If the pedestrian returns to the warning area again, the stay time needs to be recalculated.
[0075] Specifically, considering that the usage status of the socket may also change during the pedestrian tracking process, that is, the socket that was initially in use is no longer in use or is used again immediately after being unused for a short period of time, when the socket in use has never been used, the trajectory information of all pedestrians can be directly deleted and pedestrian tracking can be stopped. However, for the socket that is used again immediately after being unused for a short period of time, it is necessary to continue pedestrian tracking and continue to update the trajectory information and stay time of pedestrians to prevent pedestrians from escaping tracking by temporarily unplugging the plug.
[0076] At this time, if Figure 4As shown, in order to identify whether the socket has not been used for a long time or is used again immediately after being not used for a short time, the present invention further provides the following steps:
[0077] When a socket in the warning area changes from being in use to being unused, the unused time of the socket is recorded;
[0078] If the unused time of the socket exceeds a preset third time period, the trajectory information and the stay time of all the stay times are deleted; otherwise, the trajectory information of each stay time continues to be stored and the stay time of each pedestrian is accumulated.
[0079] Furthermore, since pedestrian tracking and identification is a dynamic process, when pedestrians block each other or leave the area for a short time and then return, the same pedestrian may generate multiple track information, resulting in pedestrian tracking errors. To this end, the present invention further provides a step of sorting all track information every fourth time interval to remove erroneous track information. The specific operation is as follows: Figure 5 As shown:
[0080] Every fourth preset time period, the cosine similarity of all trajectory information is calculated;
[0081] When the cosine similarity of multiple track information is greater than a first threshold, it is determined that the multiple track information belongs to the same pedestrian;
[0082] Among multiple trajectory information belonging to the same pedestrian, the earliest one generated is saved and the others are deleted.
[0083] See also Figure 6 The present invention also provides an electricity consumption detection device, comprising:
[0084] A setting module 10 is used to obtain the positions of each surveillance camera and socket in the scene to be detected, and set the warning area according to the positions of each surveillance camera and socket;
[0085] The recognition module 20 is used to input the images captured by the surveillance camera into the trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area.
[0086] An extraction module 30, configured to extract appearance feature information of pedestrians in the warning area when the sockets in the warning area are in use;
[0087] A tracking module 40, for inputting appearance feature information of pedestrians in the warning area into a preset pedestrian tracking algorithm, tracking the pedestrians in the warning area, and obtaining trajectory information of each pedestrian and the stay time of each pedestrian in the warning area;
[0088] The alarm module 50 is used to generate and send an alarm message when the pedestrian stays in the warning area for a preset alarm duration.
[0089] See also Figure 7 The present invention also provides an electronic device, comprising: a memory 100 and a processor 200, wherein the memory 100 stores a computer program, and when the computer program is executed by the processor 200, the processor 200 executes the steps of the above method.
[0090] In summary, the present invention provides an electricity usage detection method, comprising the following steps: obtaining the positions of each surveillance camera and socket in a scene to be detected, and setting a warning area according to the positions of each surveillance camera and socket; inputting the images captured by the surveillance camera into a trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area; when the sockets in the warning area are in use, extracting the appearance feature information of the pedestrians in the warning area; inputting the appearance feature information of the pedestrians in the warning area into a preset pedestrian tracking algorithm, tracking the pedestrians in the warning area, and obtaining the trajectory information of each pedestrian and the stay time of each pedestrian in the warning area; when the stay time of the pedestrian in the warning area reaches a preset alarm duration, generating and sending an alarm message, and monitoring the illegal use of the sockets by combining the identification of the sockets and the pedestrians, it is possible to monitor the use of the sockets in public places in real time, automatically discover the sockets that are used illegally and the people who use the sockets, reduce the burden on socket supervisors, and enhance the socket management effect in public places.
[0091] As described above, for ordinary technicians in this field, various other corresponding changes and modifications can be made based on the technical solutions and technical concepts of the present invention, and all these changes and modifications should fall within the protection scope of the claims of the present invention.
Claims
1. A method for detecting electricity consumption, characterized in that: The steps include: Obtain the locations of each surveillance camera and socket in the scene to be detected, and set the warning area according to the locations of each surveillance camera and socket; Input the images captured by the surveillance camera into the trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area; When the socket in the warning area is in use, the appearance feature information of the pedestrian in the warning area is extracted; Input appearance feature information of pedestrians in the warning area into a preset pedestrian tracking algorithm, track the pedestrians in the warning area, and obtain trajectory information of each pedestrian and the stay time of each pedestrian in the warning area; When the pedestrian stays in the warning area for less than a preset first time period, it is determined that the pedestrian is in a normal state; When the pedestrian's stay time in the warning area reaches a preset first duration and is less than a preset alarm duration, the pedestrian is determined to be in a stranded state; For pedestrians in a stranded state, when the tracking algorithm loses the pedestrian, the length of time the pedestrian is lost is recorded; When the lost time exceeds the preset second time, the track information and the stay time of the pedestrian are deleted, otherwise the track information of the pedestrian continues to be stored and the stay time of the pedestrian is accumulated; When a socket in the warning area changes from a used state to an unused state, the unused time of the socket is recorded; if the unused time of the socket exceeds the preset third time, the trajectory information and stay time of all pedestrians are deleted, otherwise the trajectory information of each pedestrian continues to be stored and the stay time of each pedestrian is accumulated; At every fourth preset time period, the cosine similarity of all trajectory information is calculated; when the cosine similarity of multiple trajectory information is greater than a first threshold, the multiple trajectory information is determined to belong to the same pedestrian; the earliest generated trajectory information among the multiple trajectory information belonging to the same pedestrian is retained, and the other trajectory information is deleted; When the pedestrian stays in the warning area for a preset alarm duration, the pedestrian is determined to be in a wandering state, and an alarm message is generated and sent.
2. The power consumption detection method according to claim 1, characterized in that: Tracking pedestrians in the warning area to obtain the trajectory information of each pedestrian and the time each pedestrian stayed in the warning area; specifically including: Input the appearance feature information of the pedestrian in the current frame into the pedestrian tracking algorithm; The pedestrian tracking algorithm matches the appearance feature information of the pedestrian in the current frame with the trajectory information of multiple frames stored previously, generates the matching result and the trajectory information of the current frame, and stores the trajectory information of the current frame; When the matching result is consistent, the pedestrian's stay time is accumulated.
3. The power consumption detection method according to claim 1, characterized in that: Generating the alarm information specifically includes marking the image of the used socket with a rectangular frame and the image of the pedestrian using the socket.
4. A power consumption detection device, the power consumption detection device is used to implement the power consumption detection method according to claim 1, characterized in that: include: A setting module is used to obtain the positions of each surveillance camera and socket in the scene to be detected, and set the warning area according to the positions of each surveillance camera and socket; The recognition module is used to input the images captured by the surveillance camera into the trained target recognition model to identify the usage status of the sockets in the warning area and the pedestrians in the warning area; An extraction module, used to extract appearance feature information of pedestrians in the warning area when the socket in the warning area is in use; A tracking module, used to input the appearance feature information of the pedestrians in the warning area into a preset pedestrian tracking algorithm, track the pedestrians in the warning area, and obtain the trajectory information of each pedestrian and the stay time of each pedestrian in the warning area; The alarm module is used to generate and send an alarm message when the pedestrian's stay time in the warning area reaches a preset alarm duration.
5. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Method and device for controlling socket, socket and readable storage medium
CN107976953A
Intelligent monitoring method and device
CN108694812A
Electric shock prevention system, method and device
CN110263759A
Road-occupying operation monitoring method based on target detection and pedestrian tracking
CN113887304A
Target person information detection device and method and storage medium
CN114067236A
