Early warning method and device based on real-time image, electronic equipment and storage medium

Abstract

The invention discloses an early warning method and device based on a real-time image, electronic equipment and a storage medium, and relates to the artificial intelligence technology, and the method comprises the steps: obtaining a real-time image of a website counter; determining a target portrait area and at least one related portrait area according to the real-time image; determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm; and performing early warning according to the distance. The peeping risk is automatically pre-warned through a machine, whether the peeping risk exists or not is judged according to the distance between the target portrait handling business and other portraits close to the target portrait according to the real-time image, then pre-warning is conducted, and the real-time performance and accuracy of peeping risk pre-warning are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a warning method, device, electronic device and storage medium based on real-time images. Background Art

[0002] With the development of society, customers pay more and more attention to personal financial sensitive information. Banks also pay more and more attention to the protection of personal financial information and sensitive data.

[0003] Banking business involves personal financial information and sensitive data, such as account balances, passwords, etc. When working at the bank counter, customers often stand within the one-meter line while others are handling business. This behavior of being too close may lead to peeping or hearing the financial information and sensitive data of the customers who are handling business, which may lead to the risk of customer privacy leakage.

[0004] Therefore, it is very necessary for staff to intervene at the first time. However, in daily work, the situation of outlets is very complicated, which may cause staff to be unable to know the situation at the first time. How to quickly warn when the above risks occur has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides a warning method, device, electronic device and storage medium based on real-time images to solve the problem that it is currently impossible to quickly warn of counter peeping risks.

[0006] According to one aspect of the present invention, there is provided an early warning method based on real-time images, comprising:

[0007] Obtain real-time images of branch counters;

[0008] determining a target portrait area and at least one related portrait area according to the real-time image;

[0009] Determine the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm;

[0010] An early warning is given according to the distance.

[0011] According to another aspect of the present invention, there is provided an early warning device based on real-time images, comprising:

[0012] An image acquisition module is used to acquire real-time images of the counter surface of the outlet;

[0013] A portrait recognition module, used to determine a target portrait area and at least one related portrait area according to the real-time image;

[0014] A distance calculation module, used for determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm;

[0015] The early warning module is used to issue an early warning according to the distance.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the early warning method based on real-time images described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the early warning method based on real-time images described in any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention obtains a real-time image of a counter surface of a branch; determines a target portrait area and at least one related portrait area based on the real-time image; determines the distance between the target portrait area and the at least one related portrait area based on a binocular vision algorithm; and issues an early warning based on the distance. Compared with the current manual inspection by staff using human eyes, the technical solution provided by the embodiment of the present invention can issue an early warning for the risk of peeping through automatic machine automation, judge whether there is a risk of peeping based on the distance between the target portrait who is handling business and other portraits close to the target portrait in the real-time image, and then issue an early warning, thereby improving the real-time and accuracy of the early warning of the risk of peeping.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1is a system structure diagram of a real-time image-based early warning method provided by an embodiment of the present invention;

[0025] Figure 2 It is a flowchart of a warning method based on real-time images provided by an embodiment of the present invention;

[0026] Figure 3 is a flow chart of another early warning method based on real-time images provided by an embodiment of the present invention;

[0027] Figure 4 is a structural schematic diagram of a warning device based on real-time images provided by an embodiment of the present invention;

[0028] Figure 5 It is a structural schematic diagram of an electronic device for implementing the early warning method based on real-time images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] The information collected by the embodiments of the present invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. Provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered to avoid relevant legal risks and public opinion risks.

[0032] With the development of society, customers pay more and more attention to personal financial sensitive information. Banks also pay more and more attention to the protection of personal financial information and sensitive data.

[0033] Banking business involves personal financial information and sensitive data, such as account balances, passwords, etc. When working at the bank counter, customers often stand within the one-meter line while others are handling business. This behavior of being too close may lead to peeping or hearing the financial information and sensitive data of the customers who are handling business, which may lead to the risk of customer privacy leakage.

[0034] Therefore, it is necessary for staff to intervene at the first time. However, in daily work, the situation of the outlets is very complicated, which may cause the staff to be unable to know the situation at the first time. Since the staff need to handle other business, they cannot always monitor the risk of peeping. Therefore, how to quickly warn when the above risks occur has become an urgent problem to be solved.

[0035] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present invention includes a binocular camera, a server and a target terminal.

[0036] Among them, the binocular camera is also called a binocular camera, which can obtain images through the left camera and the right camera respectively. The binocular camera can be installed on the top of the counter of the branch. It is used to capture the real-time image of the business handling area in real time. The real-time image is transmitted to the server for processing.

[0037] The server is used to perform face recognition and distance measurement based on real-time images to obtain the distance between different portraits. When the distance is less than a preset distance threshold, such as 1 meter, a peeping warning is issued. The peeping warning can be to send a warning message to the target terminal of the on-site staff of the outlet.

[0038] The target terminal held by the on-site staff can be a computer of the counter staff or a tablet computer of the lobby staff. When receiving the warning message transmitted by the server, they can go to the business handling area to handle the situation.

[0039] Figure 2This is a flowchart of a warning method based on real-time images provided by an embodiment of the present invention. This embodiment is applicable to the situation of warning the risk of peeping at bank counters. The method can be executed by a warning device based on real-time images. The warning device based on real-time images can be implemented in the form of hardware and / or software. The warning device based on real-time images can be configured in electronic devices such as personal computers and servers. Figure 1 As shown, the method includes:

[0040] S101. Acquire a real-time image of a branch counter.

[0041] Optionally, obtaining a real-time image of the branch counter can be implemented in the following ways:

[0042] A left real-time image captured by a left camera is obtained; and a right real-time image captured by a right camera is obtained. The left camera and the right camera are binocular vision cameras, and the binocular vision cameras are used to capture real-time images of the counter area of ​​a branch.

[0043] The binocular camera obtains real-time images through two cameras respectively. The left camera obtains the left real-time image, and the right camera obtains the right real-time image. The captured left real-time image and right real-time image can be sent to the server through the network.

[0044] S102: Determine a target portrait region and at least one related portrait region according to the real-time image.

[0045] Optionally, determining the target portrait area and at least one related portrait area according to the real-time image may be implemented in the following manner:

[0046] Inputting the left real-time image into a convolutional neural network, wherein the convolutional neural network is used to identify and mark the face area in the left real-time image to obtain a left target portrait area and at least one left related portrait area;

[0047] The right real-time image is input into a convolutional neural network, and the convolutional neural network is also used to identify and mark the face area in the right real-time image to obtain a right target portrait area and at least one right related portrait area.

[0048] Deep learning algorithms can be used to train a large number of data sets and extract feature values, compare the extracted features with the facial features obtained by training the convolutional neural network (CNN) in the database, and mark the obtained portraits. The convolutional neural network can gradually obtain the number of people contained in the real-time image by processing the real-time image.

[0049] If the number of people is greater than 1, that is, the number of people is more than 2, the distance between the portraits needs to be calculated. If it is less than or equal to 1, that is, the number of people is 1 or 0, the distance calculation between the portraits is canceled.

[0050] Furthermore, the portrait of the current server at the counter can be determined according to the attention mechanism, and the portrait is used as the target portrait, and the remaining portraits are related portraits.

[0051] By using the convolutional neural network multiple times, the left target portrait area and the left related portrait area in the left real-time image can be determined, and the right target portrait area and the right related portrait area in the right real-time image can also be determined.

[0052] The server will use portrait recognition to determine whether there is a person in the current image. If so, the target portrait of the person handling business in the calibration area will be marked as R0, and the related portraits will be marked as {R1, R2, ..., R n}.

[0053] The above implementation can accurately identify the number of portraits in real-time images (left real-time image and right real-time image) and the portrait area corresponding to each portrait through a convolutional neural network, thereby improving the efficiency of portrait recognition.

[0054] S103: Determine the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm.

[0055] Optionally, determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm includes:

[0056] The three-dimensional spatial coordinates of the target portrait are determined according to the left target portrait area and the right target portrait area. The three-dimensional spatial coordinates of the related portraits are determined according to the left related portrait area and the right related portrait area. The distance between the target portrait and the related portrait in the three-dimensional space is determined according to the target portrait spatial coordinates and the related portrait spatial coordinates.

[0057] Specifically, determining the target portrait space coordinates according to the left target portrait area and the right target portrait area can be implemented as follows:

[0058] Determine the left target projection coordinates according to the left target portrait area; determine the right target projection coordinates according to the right target portrait area;

[0059] Determine the z-axis coordinate of the target portrait according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the focal length and the camera distance;

[0060] Determine the x-axis coordinate of the target portrait based on the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance, and the x-axis coordinate of the optical center.

[0061] The y-axis coordinate of the target portrait is determined according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0062] In binocular vision technology, assume that the center coordinates of the left camera are (0,0,0), the center coordinates of R0 are (x,y,z), and the center coordinates of the right camera are (b,0,0), where b is the distance between the two cameras. x ,o y ) is the optical center coordinate, and the projection coordinate of R0 in the left real-time image is (x i ,y i ), the projection coordinates in the right real-time image are (x j ,y j ), with focal length f. Under the left camera:

[0063]

[0064] Under the right camera:

[0065]

[0066] According to the above formula, the left target projection coordinate (x i ,y i ), and the right target projection coordinates (x j ,y j ).

[0067] Based on these two coordinates, the coordinates of the center of the portrait can be calculated:

[0068]

[0069] According to the above formula, the x-axis coordinate, y-axis coordinate and z-axis coordinate of the target portrait in the three-dimensional space, ie, the spatial coordinates of the target portrait, can be calculated respectively.

[0070] The above method can accurately calculate the spatial coordinates of the target portrait, realize automatic, fast and accurate calculation, and improve the calculation efficiency of the spatial coordinates of the target portrait.

[0071] Similarly, specifically, determining the spatial coordinates of the relevant portrait according to the left relevant portrait area and the right relevant portrait area can be implemented as follows:

[0072] Determine the left-related projection coordinates according to the left-related portrait area; determine the right-related projection coordinates according to the right-related portrait area;

[0073] Determine the z-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the focal length and the camera distance;

[0074] Determine the x-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the x-axis coordinate of the optical center

[0075] The y-axis coordinate of the relevant portrait is determined according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0076] According to the above formula, the spatial coordinates of the relevant portrait can be calculated by the same formula, that is, {(x1,y1,z1),(x2,y2,z2),...,(x n ,y n ,z n )}

[0077] {R1, R2, ..., R n}The distance l from R0.

[0078]

[0079] S104: Obtain an early warning based on the distance.

[0080] Optionally, issuing an early warning based on the distance may be implemented in the following manner: if the distance is less than a preset early warning threshold, sending an early warning message to the target terminal.

[0081] The preset warning threshold may be 1 meter, and when the distance between any one of the multiple related portraits and the target portrait is less than the warning threshold, a warning is triggered. The server sends a warning message to the target terminal.

[0082] The technical solution of the embodiment of the present invention obtains a real-time image of the counter of a branch; determines a target portrait area and at least one related portrait area based on the real-time image; determines the distance between the target portrait area and the at least one related portrait area based on a binocular vision algorithm; and issues an early warning based on the distance. Compared with the current manual screening by staff using human eyes, the technical solution provided by the embodiment of the present invention can issue an early warning of the risk of peeping through automatic machine, and judges whether there is a risk of peeping based on the distance between the target portrait who is handling business and other portraits close to the target portrait based on the real-time image, and then issues an early warning, thereby improving the real-time and accuracy of the early warning of the risk of peeping.

[0083] Figure 3 A flowchart of a real-time image-based early warning method provided by an embodiment of the present invention is provided.

[0084] In one implementation, the early warning method based on real-time images provided in an embodiment of the present invention is applied to a server, including:

[0085] Step S201, obtaining a left real-time image captured by a left camera; obtaining a right real-time image captured by a right camera.

[0086] Among them, the left camera and the right camera are binocular vision cameras, and the binocular vision cameras are used to capture real-time images of the counter area of ​​the outlet.

[0087] Step S202: Input the left real-time image into the convolutional neural network to obtain a left target portrait region and at least one left related portrait region. Input the right real-time image into the convolutional neural network to obtain a right target portrait region and at least one right related portrait region.

[0088] The convolutional neural network is used to identify the face area in the real-time image (including the left real-time image and the right real-time image), and mark the target portrait area (such as the left target portrait area or the right target portrait area) and the related portrait area (such as the left related portrait area or the right related portrait area).

[0089] Step S203: determining the three-dimensional spatial coordinates of the target portrait according to the left target portrait area and the right target portrait area. Specifically, it includes:

[0090] Determine the left target projection coordinates according to the left target portrait area; determine the right target projection coordinates according to the right target portrait area;

[0091] Determine the z-axis coordinate of the target portrait according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the focal length and the camera distance;

[0092] Determine the x-axis coordinate of the target portrait based on the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance, and the x-axis coordinate of the optical center.

[0093] The y-axis coordinate of the target portrait is determined according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0094] Step S204: determining the spatial coordinates of the relevant portraits according to the left relevant portrait area and the right relevant portrait area. Specifically, it includes:

[0095] Determine the left-related projection coordinates according to the left-related portrait area; determine the right-related projection coordinates according to the right-related portrait area;

[0096] Determine the z-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the focal length and the camera distance;

[0097] Determine the x-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the x-axis coordinate of the optical center

[0098] The y-axis coordinate of the relevant portrait is determined according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0099] Step S205 : determining the distance between the target portrait and the related portrait in the three-dimensional space according to the target portrait spatial coordinates and the related portrait spatial coordinates.

[0100] Step S206: If the distance is less than a preset warning threshold, a warning message is sent to the target terminal.

[0101] The technical solution of the embodiment of the present invention obtains a real-time image of the counter of a branch; determines a target portrait area and at least one related portrait area based on the real-time image; determines the distance between the target portrait area and the at least one related portrait area based on a binocular vision algorithm; and issues an early warning based on the distance. Compared with the current manual screening by staff using human eyes, the technical solution provided by the embodiment of the present invention can issue an early warning of the risk of peeping through machine automation, and judge whether there is a risk of peeping based on the distance between the target portrait who is handling business and other portraits close to the target portrait based on the real-time image, and then issue an early warning, thereby improving the real-time and accuracy of the early warning of the risk of peeping. Through efficient and intelligent means, it protects the privacy and data security of customers, prevents and reduces potential security threats, and improves the comprehensive service quality of branches, thereby enhancing the comprehensive competitiveness of branches.

[0102] Figure 4 1 is a schematic diagram of a schematic diagram of a structural diagram of a warning device based on real-time images provided by an embodiment of the present invention, and the device is suitable for warning of the risk of peeping at bank counters. Figure 4 As shown, the device includes: an image acquisition module 31, a portrait recognition module 32, a distance calculation module 33 and an early warning module 34.

[0103] An image acquisition module 31 is used to acquire a real-time image of a counter surface of a network point;

[0104] A portrait recognition module 32, configured to determine a target portrait region and at least one related portrait region according to the real-time image;

[0105] A distance calculation module 33, used for determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm;

[0106] The early warning module 34 is used to issue an early warning according to the distance.

[0107] Based on the above embodiment, optionally, the image acquisition module 31 is used to:

[0108] Get the left real-time image taken by the left camera;

[0109] A right real-time image captured by a right camera is obtained, wherein the left camera and the right camera are binocular vision cameras, and the binocular vision camera is used to capture a real-time image of a counter area of ​​a network point.

[0110] Based on the above embodiment, optionally, the portrait recognition module 32 is used to:

[0111] Inputting the left real-time image into a convolutional neural network, wherein the convolutional neural network is used to identify and mark the face area in the left real-time image to obtain a left target portrait area and at least one left related portrait area;

[0112] The right real-time image is input into a convolutional neural network, and the convolutional neural network is also used to identify and mark the face area in the right real-time image to obtain a right target portrait area and at least one right related portrait area.

[0113] Based on the above embodiment, optionally, the distance calculation module 33 is used to:

[0114] Determine the three-dimensional spatial coordinates of the target portrait according to the left target portrait area and the right target portrait area;

[0115] Determine the three-dimensional spatial coordinates of the relevant portrait according to the left relevant portrait area and the right relevant portrait area;

[0116] The distance between the target portrait and the related portrait in the three-dimensional space is determined according to the target portrait space coordinates and the related portrait space coordinates.

[0117] Based on the above embodiment, optionally, the distance calculation module 33 is used to determine the three-dimensional spatial coordinates of the target portrait according to the left target portrait area and the right target portrait area, including:

[0118] Determine the left target projection coordinates according to the left target portrait area; determine the right target projection coordinates according to the right target portrait area;

[0119] Determine the z-axis coordinate of the target portrait according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the focal length and the camera distance;

[0120] Determine the x-axis coordinate of the target portrait based on the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance, and the x-axis coordinate of the optical center.

[0121] The y-axis coordinate of the target portrait is determined according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0122] Based on the above embodiment, optionally, the distance calculation module 33 is used to determine the spatial coordinates of the relevant portrait according to the left relevant portrait area and the right relevant portrait area, including:

[0123] Determine the left-related projection coordinates according to the left-related portrait area; determine the right-related projection coordinates according to the right-related portrait area;

[0124] Determine the z-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the focal length and the camera distance;

[0125] Determine the x-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the x-axis coordinate of the optical center

[0126] The y-axis coordinate of the relevant portrait is determined according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0127] On the basis of the above embodiment, optionally, the warning module 34 is used to: if the distance is less than a preset warning threshold, send a warning message to the target terminal.

[0128] The early warning device based on real-time images provided by the embodiment of the present invention comprises an image acquisition module 31 for acquiring a real-time image of the counter of a branch; a portrait recognition module 32 for determining a target portrait area and at least one related portrait area according to the real-time image; a distance calculation module 33 for determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm; and an early warning module 34 for issuing an early warning according to the distance. Compared with the current manual screening by the staff through human eyes, the early warning device based on real-time images provided by the embodiment of the present invention can issue an early warning for the risk of peeping through machine automation, and judge whether there is a risk of peeping based on the distance between the target portrait who is handling the business and other portraits close to the target portrait according to the real-time image, and then issue an early warning, thereby improving the real-time and accuracy of the early warning of the risk of peeping. Through efficient and intelligent means, the privacy and data security of customers are protected, potential security threats are prevented and reduced, and the comprehensive service quality of the branch is improved, and the comprehensive competitiveness of the branch is strengthened.

[0129] The early warning device based on real-time images provided in the embodiment of the present invention can execute the early warning method based on real-time images provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0130] Figure 51 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0131] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0132] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0133] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as an early warning method based on real-time images.

[0134] In some embodiments, the early warning method based on real-time images can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the early warning method based on real-time images described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the early warning method based on real-time images in any other appropriate manner (for example, by means of firmware).

[0135] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] The computer program for implementing the real-time image-based early warning method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or completely on a remote machine or server.

[0137] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute an early warning method based on real-time images, the method comprising:

[0138] Obtain real-time images of branch counters;

[0139] determining a target portrait area and at least one related portrait area according to the real-time image;

[0140] Determine the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm;

[0141] An early warning is given according to the distance.

[0142] Based on the above embodiment, optionally, obtaining a real-time image of a counter surface of a branch includes:

[0143] Get the left real-time image taken by the left camera;

[0144] A right real-time image captured by a right camera is obtained, wherein the left camera and the right camera are binocular vision cameras, and the binocular vision camera is used to capture a real-time image of a counter area of ​​a network point.

[0145] Based on the above embodiment, optionally, determining the target portrait area and at least one related portrait area according to the real-time image includes:

[0146] Inputting the left real-time image into a convolutional neural network, wherein the convolutional neural network is used to identify and mark the face area in the left real-time image to obtain a left target portrait area and at least one left related portrait area;

[0147] The right real-time image is input into a convolutional neural network, and the convolutional neural network is also used to identify and mark the face area in the right real-time image to obtain a right target portrait area and at least one right related portrait area.

[0148] Based on the above embodiment, optionally, determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm includes:

[0149] Determine the three-dimensional spatial coordinates of the target portrait according to the left target portrait area and the right target portrait area;

[0150] Determine the three-dimensional spatial coordinates of the relevant portrait according to the left relevant portrait area and the right relevant portrait area;

[0151] The distance between the target portrait and the related portrait in the three-dimensional space is determined according to the target portrait space coordinates and the related portrait space coordinates.

[0152] Based on the above embodiment, optionally, determining the target portrait space coordinates according to the left target portrait area and the right target portrait area includes:

[0153] Determine the left target projection coordinates according to the left target portrait area; determine the right target projection coordinates according to the right target portrait area;

[0154] Determine the z-axis coordinate of the target portrait according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the focal length and the camera distance;

[0155] Determine the x-axis coordinate of the target portrait based on the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance, and the x-axis coordinate of the optical center.

[0156] The y-axis coordinate of the target portrait is determined according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0157] On the basis of the above embodiment, optionally, determining the spatial coordinates of the relevant portrait according to the left relevant portrait area and the right relevant portrait area includes:

[0158] Determine the left-related projection coordinates according to the left-related portrait area; determine the right-related projection coordinates according to the right-related portrait area;

[0159] Determine the z-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the focal length and the camera distance;

[0160] Determine the x-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the x-axis coordinate of the optical center

[0161] The y-axis coordinate of the relevant portrait is determined according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the y-axis coordinate of the optical center.

[0162] Based on the above embodiment, optionally, giving an early warning according to the distance includes:

[0163] If the distance is less than a preset warning threshold, a warning message is sent to the target terminal.

[0164] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0165] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0166] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0167] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0168] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0169] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A warning method based on real-time images, characterized in that: include: Obtain real-time images of branch counters; determining a target portrait area and at least one related portrait area according to the real-time image; Determine the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm; An early warning is given according to the distance.

2. The method according to claim 1, characterized in that: Get real-time images of branch counters, including: Get the left real-time image captured by the left camera; A right real-time image captured by a right camera is obtained, wherein the left camera and the right camera are binocular vision cameras, and the binocular vision camera is used to capture a real-time image of a counter area of ​​a network point.

3. The method according to claim 2, characterized in that Determining a target portrait area and at least one related portrait area according to the real-time image includes: Inputting the left real-time image into a convolutional neural network, wherein the convolutional neural network is used to identify and mark the face area in the left real-time image to obtain a left target portrait area and at least one left related portrait area; The right real-time image is input into a convolutional neural network, and the convolutional neural network is also used to identify and mark the face area in the right real-time image to obtain a right target portrait area and at least one right related portrait area.

4. The method according to claim 3, characterized in that Determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm includes: Determine the three-dimensional spatial coordinates of the target portrait according to the left target portrait area and the right target portrait area; Determine the three-dimensional spatial coordinates of the relevant portrait according to the left relevant portrait area and the right relevant portrait area; The distance between the target portrait and the related portrait in the three-dimensional space is determined according to the target portrait space coordinates and the related portrait space coordinates.

5. The method according to claim 4, characterized in that Determining the target portrait space coordinates according to the left target portrait region and the right target portrait region includes: Determine the left target projection coordinates according to the left target portrait area; determine the right target projection coordinates according to the right target portrait area; Determine the z-axis coordinate of the target portrait according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the focal length and the camera distance; Determine the x-axis coordinate of the target portrait based on the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance, and the x-axis coordinate of the optical center. The y-axis coordinate of the target portrait is determined according to the x-axis coordinate of the left target projection coordinate, the x-axis coordinate of the right target projection coordinate, the camera distance and the y-axis coordinate of the optical center.

6. The method according to claim 4, characterized in that Determining the spatial coordinates of the relevant portrait according to the left relevant portrait region and the right relevant portrait region includes: Determine the left-related projection coordinates according to the left-related portrait area; determine the right-related projection coordinates according to the right-related portrait area; Determine the z-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the focal length and the camera distance; Determine the x-axis coordinate of the relevant portrait according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the x-axis coordinate of the optical center The y-axis coordinate of the relevant portrait is determined according to the x-axis coordinate of the left-related projection coordinate, the x-axis coordinate of the right-related projection coordinate, the camera distance and the y-axis coordinate of the optical center.

7. The method according to claim 1, characterized in that An early warning is given according to the distance, including: If the distance is less than a preset warning threshold, a warning message is sent to the target terminal.

8. A warning device based on real-time images, characterized in that: include: An image acquisition module is used to acquire real-time images of the counter surface of the outlet; A portrait recognition module, used to determine a target portrait area and at least one related portrait area according to the real-time image; A distance calculation module, used for determining the distance between the target portrait area and the at least one related portrait area according to a binocular vision algorithm; The early warning module is used to issue an early warning according to the distance.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the early warning method based on real-time images as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the early warning method based on real-time images as described in any one of claims 1 to 7 when executed.

Images