ELECTRONIC DEVICE FOR EXTRACTING DATA FROM VIDEOS IN REAL TIME.

Abstract

The present invention relates to an electronic device that implements a method for extracting data from videos in real time, wherein the device comprises: an image unit that captures at least one video from which the data will be extracted, which comprises: an image sensor, a lens, a motorized adjustment mechanism and a control card; a PoE card that supplies electrical power for the operation of the various components of the device and allows communication between the device and the external server; and a carrier module to which an analysis computer is attached, which allows the interconnection and communication of the analysis computer with the image unit and the PoE card, wherein the analysis computer processes and analyzes the captured video in real time, extracting the data that is sent to the external server through the PoE card, to be used by a user.The method implemented by the electronic device comprises several stages that allow the detection, identification and classification of objects of interest, their characteristics and / or events of interest in the analyzed frames, using artificial intelligence models, such as deep neural networks.

Claims

1. An electronic device for extracting data from videos in real time, characterized in that it comprises: an image unit that captures at least one video consisting of multiple frames, from which the data will be extracted, wherein the image unit comprises: an image sensor that captures the video; a lens coupled to the image sensor, which directs light rays to the image sensor, allowing it to capture the video; a motorized adjustment mechanism coupled to the lens, which allows adjusting the focus of the lens, the focal length of the lens and / or enabling / disabling an infrared filter over the lens for video capture in day and night conditions; and a control board interconnected to the motorized adjustment mechanism, which controls its operation;A PoE card interconnected with an external server, comprising: a power module that provides electrical power for the operation of the various components of the electronic device; and a communication module that allows the interconnection of the electronic device with an external server; a carrier module interconnected with the image sensor, the control card, and the PoE card, to which an analysis computer is attached, wherein said carrier module allows the interconnection and communication of the analysis computer with the image sensor, the control card, and the PoE card; wherein the analysis computer processes and analyzes in real time the video captured by the image unit, extracting the data that is sent to the external server through the PoE card, for use by a user.

2. The electronic device according to claim 1, further characterized in that the image sensor comprises multiple photodiodes and / or phototransistors which, when exposed to a lens, collect the information that makes up each frame, wherein said image sensor can capture video at a resolution of 720p at 60 frames per second or 1080p at 30 frames per second.

3. The electronic device according to claim 1, further characterized in that the image sensor is selected from the group comprising a CCD sensor, a SuperCCD sensor, a CMOS sensor and a Foveon X3 sensor.

4. The electronic device according to claim 3, further characterized in that the image sensor is a CMOS sensor.

5. The electronic device according to claim 1, further characterized in that the connection between the image sensor and the carrier module is made through a camera serial interface (CSI) by means of a fifteen-pin flexible printed circuit board (FPC) cable.

6. The electronic device according to claim 1, further characterized in that the lens has a size of 2.7 x 13.5 mm 1:1.3 (1 / 2.7”).

7. The electronic device according to claim 1, further characterized in that the motorized adjustment mechanism comprises: a first electric motor coupled to the lens and operated by the control card, which adjusts the focus of the lens; and, a second electric motor coupled to the lens and operated by the control card, which adjusts the focal length of the lens.

8. The electronic device according to claim 7, characterized in that the motorized adjustment mechanism further comprises an electromechanical device operated by the control card, which places and removes an infrared filter over the lens so that it can capture videos in daylight and nighttime lighting conditions.

9. The electronic device according to claim 1, further characterized in that the control card is interconnected with the motorized adjustment mechanism via voltage cables and with the carrier module via c / nhi η / ζζηζ / Β / γίΛΐ -27 of the l2C protocol.

10. The electronic device according to claim 1, further characterized in that the control board comprises a microcontroller and an H-bridge integrated circuit, wherein the microcontroller sends instructions to the H-bridge integrated circuit for the activation and operation of the motorized adjustment mechanism.

11. The electronic device according to claim 1, further characterized in that the power module comprises an integrated circuit consisting of a set of coil and transformer arrangements, which performs the transformation of the voltage supplied by the external server through the Ethernet connection.

12. The electronic device according to claim 1, further characterized in that the communication module comprises an isolation transformer that uses the Ethernet protocol to interconnect the analysis computer with the external server.

13. The electronic device according to claim 1, further characterized in that the carrier module comprises: a primary connection interface to which the analysis computer is coupled, allowing its interconnection with the various elements with which it interacts; a sequence controller interconnected with the analysis computer through the primary connection interface, which receives electrical power from the power module, supplying it to various components of the carrier module, wherein said sequence controller restarts the operation of the analysis computer in case of any error in its operation; a startup sequencer interconnected to the sequence controller and the analysis computer through the primary connection interface, which allows the analysis computer to start its power-up process;a debugging device interconnected with the analysis computer through the primary connection interface, which identifies and allows the debugging of errors in case of failure in the analysis computer; a first power submodule interconnected to the sequence controller, the image sensor and the analysis computer through the primary connection interface, which provides electrical power to the image sensor for its operation; and, a second power submodule interconnected to the sequence controller, the debugging device and the analysis computer through the primary connection interface, which provides electrical power to the debugging device for its operation.

14. The electronic device according to claim 13, further characterized in that the primary connection interface is an electronic socket type SoM.

15. The electronic device according to claim 13, characterized in that the carrier module further comprises a secondary interconnection interface that allows data communication and power supply between the control card and the analysis computer by means of the primary connection interface via the l2C protocol.

16. The electronic device according to claim 15, further characterized in that the secondary connection interface is a four-wire communication cable.

17. The electronic device according to claim 13, characterized in that the carrier module further comprises a tertiary connection interface that allows the interconnection of the communication module with the analysis computer by means of the primary connection interface via the Ethernet protocol.

18. The electronic device according to claim 17, further characterized in that the tertiary connection interface is an eight-wire communication cable.

19. The electronic device according to claim 13, further characterized in that the sequence controller is a microcontroller c / ntzi η / ζζηζ / Β / γίΛΐ -29 interconnected to the analysis computer through the primary connection interface using the l2C protocol.

20. The electronic device according to claim 13, characterized in that the carrier module further comprises an intelligent switch arranged between the sequence controller and the start sequencer, first power submodule and second power submodule, which is interconnected with the analysis computer through the primary connection interface by means of the l2C protocol, wherein the intelligent switch is activated while it receives a signal from the analysis computer, allowing the passage of electrical signals between said elements and is deactivated when it does not detect said signal.

21. The electronic device according to claim 20, further characterized in that the smart switch is a MOSFET type transistor.

22. The electronic device according to claim 13, characterized in that the carrier module further comprises a protective body disposed between the sequence controller and the primary connection interface, which protects the analysis computer from overcurrents and / or overloads.

23. The electronic device according to claim 22, further characterized in that the protective body is a power supply fuse.

24. The electronic device according to claim 13, further characterized in that the start sequencer is an electronic circuit comprising operational amplifiers and decoupling capacitors.

25. The electronic device according to claim 13, further characterized in that the first power supply submodule is a high-frequency synchronous buck converter optimized for small-size, high-efficiency applications.

26. The electronic device according to claim 13, further characterized in that the second power supply submodule is an electronic voltage regulator circuit.

27. The electronic device according to claim 13, further characterized in that the debugging device is an integrated circuit that communicates with the analysis computer via the l2C protocol, wherein the debugging device allows the analysis and correction of errors of the analysis computer by means of interconnection with an external computing device 28. The electronic device according to claim 27, further characterized in that the interconnection between the debugger device and the external computing equipment is made via an FTDI type cable.

29. The electronic device according to claim 13, characterized in that the carrier module further comprises a supercapacitor interconnected with the analysis computer through the primary connection interface, which provides an alternative power supply that allows the date and time of the electrical device to be maintained in case of a lack of power from the PoE card.

30. The electronic device according to claim 13, characterized in that the carrier module further comprises: an HDMI connection that allows a monitor to be connected to display, in real time, the images and video captured by the image unit; and / or a 4G interface that allows the interconnection of an integrated circuit board that supports the connection to the Internet using the 4G protocol.

31. The electronic device according to claim 13, characterized in that the carrier module further comprises one or more indicator lights that notify the user about the power status of the electronic device and the Ethernet communication.

32. The electronic device according to claim 1, further characterized in that the analysis computer comprises: a general-purpose processor that performs general search, analysis, and processing tasks, as well as communication with the external server via the PoE card; a graphics processing unit that detects, identifies, and classifies objects and / or events of interest in each frame of the video; a non-volatile memory where part of a method for extracting data from videos in real time is stored, including certain predefined values; and a volatile memory where the data extracted from the analyzed videos is temporarily stored in at least one data structure.

33. The electronic device according to claim 1, further characterized in that the analysis computer is a single-board computer, capable of processing digital videos in real time.

34. The electronic device according to claim 1, characterized in that it further comprises a housing that allows the various elements that make it up to be accommodated.

35. The electronic device according to claim 1, further characterized in that it has the ability to analyze, in real time, up to fifteen frames per second.

36. A method (2000) for extracting data from real-time videos, implemented in an electronic device for extracting real-time video data installed at a service station, wherein the electronic device comprises: an image unit that captures at least one video consisting of multiple frames, from which the data will be extracted, wherein the image unit comprises: an image sensor that captures the video; a lens coupled to the image sensor, which directs light rays to the image sensor, allowing it to capture the video; a motorized adjustment mechanism coupled to the lens, which allows adjustment of the lens focus, the lens focal length and / or enabling / disabling an infrared filter over the lens for video capture in day and night conditions; and a control board interconnected to the motorized adjustment mechanism, which controls its operation;a PoE card interconnected with an external server, comprising: a power module that provides electrical power for the operation of the various components of the electronic device; and, a communication module that allows the interconnection of the electronic device with an external server; a carrier module interconnected with the image sensor, the control card and the PoE card, to which an analysis computer is attached, wherein said carrier module allows the interconnection and communication of the analysis computer with the image sensor, the control card and the PoE card;wherein the analysis computer processes and analyzes in real time the video captured by the image unit, extracting the data that is sent to the external server, wherein said analysis computer comprises: a general-purpose processor that carries out general search, analysis and processing tasks, as well as communication with the external server through the PoE card; a graphics processing unit that detects, identifies and classifies objects and / or events of interest in each frame of the video; a non-volatile memory where part of the method (2000) is stored, including predefined values; and, a volatile memory where the data that is extracted from the analyzed videos is temporarily stored in a tracking data structure and a paused data structure, characterized in that the method comprises the steps of: capturing, with the image unit, a frame;analyze, with the graphics processing unit, said frame to identify in boxes, one or more refueling areas; store, in non-volatile memory, information related to the location of each refueling area within the service station, c / nirLn / zznz / E / YiAi -33deleting said frame (2003); capture, with the image unit, a frame (2004); analyze, with the graphics processing unit, said frame using an artificial intelligence model to identify in boxes, one or more vehicles of interest, one or more persons of interest and / or one or more license plates of interest that appear in the frame (2005); filter, with the graphics processing unit, the boxes of vehicles of interest, persons of interest and / or license plates of interest detected using the artificial intelligence model, keeping those boxes with a certainty level equal to or greater than a predefined certainty value (2006);analyze, with the general-purpose processor, the license plates of interest using optical character recognition techniques to obtain their alphanumeric representation (2007); associate, with the general-purpose processor, the alphanumeric representation of each license plate of interest with the corresponding vehicle of interest, using the intersection-on-junction standard (2008); analyze, with the graphics processing unit, each vehicle of interest using the artificial intelligence model to determine whether the hood of the analyzed vehicle of interest is open or not (2009); determine, with the general-purpose processor, the position of each detected vehicle of interest in relation to the refueling areas using the intersection-on-junction standard (2010);determine, with the general-purpose processor (410), whether each vehicle of interest is being attended to by a person of interest using the intersection-on-junction standard (2011); estimate, with the general-purpose processor (410), the speed of each vehicle of interest detected using statistical speed distribution techniques (2012); store, in volatile memory, within the tracking data structure, data for each vehicle of interest detected in the frame (2013); compare, with the general-purpose processor, the data for each vehicle of interest detected in the frame against the data contained in the tracking data structure using the intersection-on-junction standard; in case of a match, update the data for the vehicle of interest;In case of non-match, transfer the data of those vehicles of interest previously stored in the tracking data structure to the paused data structure that is stored in volatile memory (2014); verify, with the general purpose processor, the data of each vehicle of interest stored in the paused data structure; if the data of the vehicle of interest has remained for longer than a predefined time value, send said data of the vehicle of interest to the external server, deleting them from the paused data structure (2015); compare, with the general purpose processor, the data of each vehicle of interest detected in the frame against the data contained in the paused data structure using the intersection over join standard;In case of a match, transfer the data of the vehicle of interest stored in the paused data structure to the tracked data structure, updating the data of said vehicle of interest and deleting the frame; in case of no match, delete the frame (2016); and, repeat steps (2004) to (2016) successively and indefinitely while the electronic device is switched on (2017).

37. The method (2000) according to claim 36, further characterized in that the artificial intelligence model employed is a deep neural network.

38. The method (2000) according to claim 36, further characterized in that the artificial intelligence model employed is a convolutional neural network.

39. The method (2000) according to claim 35, further characterized in that the predefined certainty value is 30%.

40. The method (2000) according to claim 36, c / nfrin / zznz / E / YiAi -35 further characterized in that in step (2010), when the percentage of overlap between the boxes of each vehicle of interest and the refueling areas is equal to or greater than 10%, the vehicle of interest is considered to be ready to refuel.

41. The method (2000) according to claim 36, further characterized in that in step (2011), the box of the vehicles of interest is expanded laterally by between 25% and 100%; when one of the boxes of persons of interest completely overlaps within one of the expanded boxes of the vehicles of interest, said vehicle of interest is considered to be under attack by said person of interest.

42. The method (2000) according to claim 36, further characterized in that the data associated with each vehicle of interest comprise: position of the vehicle of interest, license plate of the vehicle of interest, whether the vehicle of interest has its hood open or not, estimated speed of the vehicle of interest, estimated position of the vehicle of interest, initial time at which the vehicle of interest was observed, time that the vehicle of interest remained in the refueling area, last time at which the vehicle of interest was observed and / or time it took a person of interest to approach the vehicle of interest.

43. The method (2000) according to claim 36, further characterized in that in step (2014), when the percentage of overlap between the boxes of the position of the vehicle of interest and the estimated position of the stored vehicle of interest is equal to or greater than 20%, both vehicles of interest are considered to be the same vehicle of interest.

44. The method (2000) according to claim 36, further characterized in that in step (2016), when the percentage of overlap between the boxes of the position of the vehicle of interest and the estimated position of the stored vehicle of interest is equal to or greater than 20%, both vehicles of interest are considered to be the same vehicle of interest.

Images