Boundary box determination method and related equipment

Abstract

The invention discloses a bounding box determination method and related equipment. According to the method, at least two candidate bounding boxes corresponding to a to-be-scanned document in a to-be-scanned image are determined; based on the four angles of the candidate bounding box, the four edges of the candidate bounding box and the included angle between the extension lines of the opposite edges of the candidate bounding box, determining candidate box confidence corresponding to the candidate bounding box; and determining the candidate bounding box with the highest candidate box confidence as a target bounding box, wherein the target bounding box is used for assisting in scanning the to-be-scanned document. The bounding box corresponding to the to-be-scanned document is automatically identified, and the identified bounding box is more accurate.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a bounding box determination method and related devices. Background Art

[0002] With the rapid development of mobile phone technology, users no longer need to use scanners and other devices to convert paper documents into electronic versions. Users can scan the required documents into electronic versions through their mobile phones.

[0003] When scanning a document, it is necessary to scan based on the document's bounding box to ensure that the content in the document is scanned completely and that no content outside the document is scanned. In actual applications, there may be multiple documents in the environment, or there may be many lines in the background, causing the phone to be unable to determine the bounding box of the document to be recognized. The user can only manually select the bounding box of the document to be recognized so that the phone can recognize the document. Summary of the invention

[0004] The present application provides a bounding box determination method and related equipment, which can realize automatic identification of a bounding box corresponding to a document to be scanned, and make the identified bounding box more accurate.

[0005] In a first aspect, some embodiments of the present application provide a method for determining a bounding box. The method for determining a bounding box may include: determining at least two candidate bounding boxes corresponding to a document to be scanned in an image to be scanned; determining the candidate box confidence corresponding to the candidate bounding box based on four angles of the candidate bounding box, four sides of the candidate bounding box, and the angle between the extension lines of the opposite sides of the candidate bounding box; determining the candidate bounding box with the highest candidate box confidence as a target bounding box, and the target bounding box is used to assist in scanning the document to be scanned.

[0006] Through the above method, the candidate bounding box confidence is jointly determined by the four angles of the candidate bounding box, the four sides of the candidate bounding box and the angle between the extension lines of the opposite sides of the candidate bounding box, so that the determined candidate bounding box confidence is more accurate, and thus the target bounding box determined by the candidate bounding box confidence can realize automatic identification of the bounding box corresponding to the document to be scanned, and the identified bounding box is more accurate.

[0007] In a possible implementation, based on the four angles of the candidate bounding box, the four sides of the candidate bounding box, and the angle of the extension line of the opposite side of the candidate bounding box, the candidate box confidence corresponding to the candidate bounding box is determined, including: based on the four angles of the candidate bounding box, the confidence of the four corners of the candidate bounding box is determined; based on the four sides of the candidate bounding box, the confidence of the screen ratio of the candidate bounding box, the confidence of the opposite side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, and the confidence of the center position of the candidate bounding box are determined; based on the angle of the extension line of the opposite side of the candidate bounding box, the confidence of the opposite side angle of the candidate bounding box is determined; based on the confidence of the four corners of the candidate bounding box, the confidence of the screen ratio of the candidate bounding box, the confidence of the opposite side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, the confidence of the center position of the candidate bounding box, and the confidence of the opposite side angle of the candidate bounding box, the candidate box confidence corresponding to the candidate bounding box is determined.

[0008] Through the above method, six confidences related to the candidate bounding box are determined by the four angles of the candidate bounding box, the four sides of the candidate bounding box and the angles of the extension lines of the opposite sides of the candidate bounding box. The confidence of the candidate box can be accurately determined by the six confidences related to the candidate bounding box.

[0009] In one possible implementation, the candidate box confidence corresponding to the candidate bounding box is the sum of the confidences of the four corners of the candidate bounding box, the confidence of the screen ratio of the candidate bounding box, the confidence of the side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, the confidence of the center position of the candidate bounding box, and the confidence of the opposite side angle of the candidate bounding box.

[0010] Through the above method, the candidate box confidence corresponding to the candidate bounding box is the sum of the six confidences, and the candidate box confidence can be determined more efficiently and quickly through the six confidences.

[0011] In a possible implementation, determining at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned includes: determining a plurality of first straight lines from the image to be scanned; taking any first straight line located in a first orientation as an initial straight line, and sequentially traversing the first straight lines located in remaining orientations except the first orientation in accordance with a preset traversal order, so as to determine at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned; wherein the first orientation is any one of the four orientations of up, down, left, and right.

[0012] By using the above method, it is ensured that the first straight line will not be missed and all the first straight lines are traversed, and the candidate boundary boxes corresponding to the documents to be scanned in all the images to be scanned can be determined more accurately.

[0013] In a possible implementation, the orientation of the first straight line is the orientation of the first straight line relative to the transverse center axis or the longitudinal center axis of the image to be scanned.

[0014] By using the above method, the size of the candidate bounding box corresponding to the document to be scanned in the image to be scanned is closer to the image to be scanned, and the candidate bounding box corresponding to the document to be scanned in the image to be scanned is more accurate.

[0015] In a possible implementation, the position coordinates of any corner of the candidate bounding box are within a preset coordinate range, and the angle of any corner of the candidate bounding box is within a preset angle range.

[0016] Through the above method, the candidate bounding box is ensured to be located within the screen through the preset position coordinates, and the candidate bounding box is determined to be more consistent with the shape of the document to be scanned through the preset angle range.

[0017] In one possible implementation, a plurality of first straight lines are determined from an image to be scanned, including: inputting the image to be scanned into an edge scanning model to obtain a plurality of second straight lines; inputting the image to be scanned into a straight line detection model to obtain a plurality of third straight lines; and determining a plurality of first straight lines based on the plurality of second straight lines and the plurality of third straight lines.

[0018] Through the above method, the image to be scanned is input into the two models respectively, and the straight lines obtained from the two models are used together to determine the first straight line, so that the determined first straight line is more accurate and more complete.

[0019] In a possible implementation, the image to be scanned is input into an edge scanning model to obtain a plurality of second straight lines, including: inputting the image to be scanned into an edge detection model to obtain a fourth straight line output by the edge detection model; merging the fourth straight lines that meet the merging condition to obtain a plurality of second straight lines, and the merging condition is that the angle between any two straight lines is less than a first threshold, or the distance between the midpoints of any two straight lines is less than a second threshold; inputting the image to be scanned into a straight line detection model to obtain a plurality of third straight lines, including: inputting the image to be scanned into the straight line detection model to obtain a fifth straight line output by the straight line detection model; merging the fifth straight lines that meet the merging condition to obtain a plurality of third straight lines.

[0020] Through the above method, the straight lines obtained by the two models are cross-screened to filter out the straight lines irrelevant to the candidate bounding box, thereby accurately determining the first straight line.

[0021] In a possible implementation, the image to be scanned is respectively input into an edge detection model and a line detection model to obtain multiple second straight lines and multiple third straight lines, including: inputting the image to be scanned into the edge detection model to obtain a fourth straight line output by the edge detection model; merging the fourth straight lines that meet the merging condition to obtain multiple second straight lines; inputting the image to be scanned into the line detection model to obtain a fifth straight line output by the line detection model; merging the fifth straight lines that meet the merging condition to obtain multiple third straight lines; the merging condition is that the angle between any two straight lines is less than a first threshold, or the distance between the midpoints of any two straight lines is less than a second threshold.

[0022] By using the above method, straight lines with similar distances are merged, the number of obtained second straight lines and third straight lines is reduced, and the efficiency of subsequent processing is improved.

[0023] In a second aspect, the present application provides an electronic device. The electronic device includes: one or more processors, and one or more memories; the one or more memories are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the method described in the first aspect or any one of the implementations of the first aspect.

[0024] In a third aspect, the present application provides a computer storage medium, which includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0025] In a fourth aspect, an embodiment of the present application provides a chip. The chip can be applied to an electronic device, and the chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method described in the first aspect or any implementation of the first aspect.

[0026] In a fifth aspect, the present application provides a computer program or a computer program product, including codes or instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0028] Figure 1B A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of a flow chart of a method for determining a bounding box provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the angle between the extended lines of opposite sides of a candidate bounding box provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of a flow chart of another method for determining a bounding box provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of a first straight line corresponding to a document to be scanned provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of multiple candidate boundaries corresponding to a document to be scanned provided in an embodiment of the present application;

[0034] Figure 7 A flowchart of another method for determining a bounding box provided in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of a second straight line and a third straight line provided in an embodiment of the present application;

[0036] Fig.9A A schematic diagram of a cross-screening provided in an embodiment of the present application;

[0037] Fig. 9B A schematic diagram of another cross-screening provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0039] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0040] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0041] The hardware structure and software structure of the electronic device involved in the embodiment of the present application are introduced below:

[0042] Figure 1A A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device may include: a processor, an external memory interface, an internal memory, a Universal Serial Bus (USB) interface, a charging management module, a power management module, a battery, an antenna 1, an antenna 2, a mobile communication module, a wireless communication module, a sensor module, a button, a motor, an indicator, a camera, a display screen, and a SIM card slot, etc. The audio module may include a speaker, a receiver, a microphone, an earphone interface, etc., and the sensor module may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0043] The structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In some embodiments of the present application, the electronic device may include more components than shown in the figure. In some embodiments of the present application, the electronic device may include fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a limitation on the structure of the electronic device.

[0044] The processor may include one or more processing units, for example, the processor may include an application processor (AP), a modem (also called a baseband processor), a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The processor is the nerve center and command center of the electronic device. The controller may generate an operation control signal according to the instruction opcode and timing signal to complete the control of fetching and executing instructions.

[0045] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker, a receiver, etc.), or displays an image or video through a display screen. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor and be set in the same device as the mobile communication module or other functional modules.

[0046] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module, wireless communication module, and modem. In some embodiments, the antenna 1 of the electronic device is coupled with the mobile communication module, and the antenna 2 is coupled with the wireless communication module, so that the electronic device can communicate with the network side device and other electronic devices through wireless communication technology.

[0047] The wireless communication function of the electronic device can be realized through antenna 1, antenna 2, a mobile communication module, a wireless communication module, a modulation and demodulation processor, and a baseband processor.

[0048] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0049] The mobile communication module can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module can receive electromagnetic waves from antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module may be arranged in the processor. In some embodiments, at least some of the functional modules of the mobile communication module may be arranged in the same device as at least some of the modules of the processor.

[0050] The wireless communication module can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module can be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor. The wireless communication module can also receive the signal to be sent from the processor, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through antenna 2.

[0051] In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module, and the antenna 2 is coupled to the wireless communication module, so that the electronic device can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite based augmentation system (SBAS).

[0052] The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor through the external memory interface to implement data storage functions. For example, music, video and other files can be saved in the external memory card.

[0053] The internal memory can be used to store computer executable program codes, which include instructions. The processor executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image or video playback function, etc.). The data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.). In addition, the internal memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0054] Electronic devices implement display functions through GPU, display screen, and application processor. GPU is a microprocessor for image processing, connecting the display screen and application processor. GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs, which execute program instructions to generate or change display information.

[0055] The display screen is used to display images, videos, etc. The display screen includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens, where N is a positive integer greater than 1.

[0056] The sensor may include, but is not limited to, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, an ambient light sensor, a fingerprint sensor, or a temperature sensor.

[0057] Figure 1BA schematic diagram of the software structure of an electronic device provided in an embodiment of the present application. The software structure adopts a layered architecture, which divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. Taking the Android system, the Android system running on the AP as an example, in some embodiments, the Android system is divided into five layers, from top to bottom, respectively, the application layer, the application framework layer (Framework), the Android runtime (Android runtime) and the system library, the hardware abstraction layer (HAL) and the system kernel layer (Kernel).

[0058] Among them, the application layer may include a series of application packages. Application packages may include camera, gallery, calendar, call, map, WLAN, Bluetooth, music, video, short message and other APPs. The application layer may also include system UI (system UI). System UI is used to display the interface of the electronic device, such as displaying the signal icon corresponding to the SIM card, displaying the call interface, etc. The application framework layer provides an application programming interface (Application Programming Interface, API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. The phone manager is used to provide call functions for electronic devices, such as management of call status (including connecting, hanging up, etc.). The phone manager is in Figure 1B The application framework layer may also include a Radio Interface Layer (RIL), and a modem processor (Modem) may exchange information with the telephony through the RIL.

[0059] Modem may include NAS (Non-Access Stratum, non-access layer) layer, RRC (radio resource control, radio resource control) layer, Packet Data Convergence Protocol (Packet Data Convergence Protocol, PDCP) layer, Radio Link Control (Radio Link Control, RLC) layer, Medium Access Control Layer (Medium Access Control Layer, MAC) layer and Physical (Physical, PHY) layer. The aforementioned layers may be software modules. Modem may interact with base stations through antennas.

[0060] In addition, some embodiments of the present application provide an electronic device, the electronic device comprising: one or more processors and a memory; the memory is used to store computer program code, the computer program code comprises computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the above-mentioned paging message processing method.

[0061] Some embodiments of the present application provide a chip system, which is applied to an electronic device. The chip system includes at least one processor and an interface, and the interface is used to receive instructions and transmit them to at least one processor; at least one processor executes the instructions so that the electronic device executes the above-mentioned paging message processing method. Among them, the chip system can be a modem, or a system on chip (System on Chip, Soc) including a modem, and the above-mentioned method can be implemented by a modem.

[0062] The following is a further introduction to the bounding box determination method provided in the embodiment of the present application:

[0063] See also Figure 2 , Figure 2 A schematic diagram of a flow chart of a method for determining a bounding box provided in an embodiment of the present application, wherein:

[0064] 201. The electronic device determines at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned.

[0065] The image to be scanned may be an image captured by a camera of an electronic device, an image saved in an album of an electronic device, or an image downloaded from the Internet by the electronic device. The present application does not limit the source of the image to be scanned. The document to be scanned may be a card, a contract, a paper, or other document. The candidate bounding box is a bounding box whose shape and size are close to the boundary of the document to be scanned, and the candidate bounding box is a quadrilateral.

[0066] 202. The electronic device determines a candidate box confidence corresponding to the candidate bounding box based on four angles of the candidate bounding box, four sides of the candidate bounding box, and an angle between extension lines of opposite sides of the candidate bounding box.

[0067] The candidate bounding box is a quadrilateral, the four angles of the candidate bounding box are the angles of the four corners of the quadrilateral, the four sides of the candidate bounding box are the four sides of the quadrilateral, and the angle between the extension lines of the opposite sides of the candidate bounding box is the angle between the extension lines of the two opposite sides of the quadrilateral. If the opposite sides of the quadrilateral are parallel, the angle between the extension lines of the opposite sides of the candidate bounding box is 0. To better understand the angle between the extension lines of the opposite sides of the candidate bounding box, please refer to Figure 3 , as shown in the figure, Figure 3 The portion marked with 301 is the candidate bounding box 301. Figure 3The two dotted lines in are: the extension line of the upper side of the candidate bounding box 301 and the extension line of the lower side of the candidate bounding box 301 . Figure 3 The angle marked with 302 is the angle between the extension lines of the opposite sides of the candidate bounding box 301. One candidate bounding box corresponds to one candidate box confidence, and multiple candidate bounding boxes correspond to multiple candidate box confidences, that is, the candidate bounding boxes correspond to the candidate box confidences one by one.

[0068] In a possible embodiment, the electronic device determines the candidate box confidence corresponding to the candidate bounding box based on the four angles of the candidate bounding box, the four sides of the candidate bounding box, and the angle of the extension lines of the opposite sides of the candidate bounding box, including: the electronic device determines the confidence of the four corners of the candidate bounding box based on the four angles of the candidate bounding box; the electronic device determines the confidence of the screen ratio of the candidate bounding box, the confidence of the opposite side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, and the confidence of the center position of the candidate bounding box based on the four sides of the candidate bounding box; the electronic device determines the confidence of the opposite side angle of the candidate bounding box based on the angle of the extension lines of the opposite sides of the candidate bounding box; the electronic device determines the candidate box confidence corresponding to the candidate bounding box based on the confidence of the four corners of the candidate bounding box, the confidence of the screen ratio of the candidate bounding box, the confidence of the opposite side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, the confidence of the center position of the candidate bounding box, and the confidence of the opposite side angle of the candidate bounding box.

[0069] Six confidences related to the candidate bounding box are determined by the four angles of the candidate bounding box, the four sides of the candidate bounding box, and the angle between the extension lines of the opposite sides of the candidate bounding box. Then, the candidate box confidence corresponding to the candidate bounding box is determined by the six confidences related to the candidate bounding box.

[0070] The following describes six ways to determine the confidence associated with the candidate bounding box:

[0071] The confidence of the four corners of the candidate bounding box is determined as follows:

[0072] Based on the largest angle among the four corners of the selected bounding box, the smallest angle among the four corners of the selected bounding box, and the preset first weight value, the confidence of the four corners of the candidate bounding box is determined. The specific calculation of the confidence of the four corners of the candidate bounding box can refer to the following formula 1:

[0073]

[0074] Among them, confidence 1 is the confidence of the four corners of the candidate bounding box, and angle 1, angle 2, angle 3, and angle 4 are the angles of the four corners of the candidate bounding box. max(angle 1, angle 2, angle 3, angle 4) represents the largest angle among the four angles, and conversely, min(angle 1, angle 2, angle 3, angle 4) represents the smallest angle among the four angles. Weight1 is the first weight value, and the first weight value is a pre-set value. The larger the first weight value, the greater the influence of the confidence of the four corners of the candidate bounding box on the confidence of the candidate box corresponding to the candidate bounding box.

[0075] For example, the angles of the four corners of the candidate bounding box are: 80° (angle 1), 90° (angle 2), 90° (angle 3), and 100° (angle 4). The maximum angle value of the four corners of the candidate bounding box is 100°, and the minimum angle value of the four corners of the candidate bounding box is 80°. Weight1 is 0.3. Confidence 1 is: 0.2958.

[0076] For example, the angles of the four corners of the candidate bounding box are: 70° (angle 1), 90° (angle 2), 90° (angle 3), and 110° (angle 4). The maximum angle value of the four corners of the candidate bounding box is 110°, and the minimum angle value of the four corners of the candidate bounding box is 70°. Weight1 is 0.3. Confidence 1 is: 0.2475.

[0077] The difference between the largest and smallest corners of the four corners of the candidate bounding box is inversely proportional to the confidence of the four corners of the candidate bounding box. The smaller the difference between the largest and smallest corners of the four corners of the candidate bounding box, the greater the confidence of the four corners of the candidate bounding box. When the four corners of the candidate bounding box are all 90°, the candidate bounding box is a rectangle, the difference between the largest and smallest corners of the four corners of the candidate bounding box is 0, and the confidence of the four corners of the candidate bounding box is the highest.

[0078] The confidence of the candidate bounding box screen ratio is determined as follows:

[0079] The confidence of the screen ratio of the candidate bounding box is based on the ratio of the candidate bounding box to the entire screen. The ratio can be the ratio of the area of ​​the candidate bounding box to the screen area, and the area of ​​the candidate bounding box can be determined based on the four sides of the candidate bounding box. This application does not limit the method for determining the ratio. The specific calculation of the confidence of the screen ratio of the candidate bounding box can refer to the following formula 2:

[0080]

[0081] Among them, confidence 2 is the confidence of the candidate bounding box's screen occupation ratio, and areaRatio is the ratio of the candidate bounding box to the entire screen. When 0.1 < areaRatio ≤ 0.9, this confidence 2 is 1; when areaRatio < 0.1, this confidence 2 is 1.0 - (1.0 - areaRatio) * Weight2, where Weight2 is a preset second weight; when areaRatio > 0.9, this confidence 2 is 1.0 - areaRatio.

[0082] Exemplarily, if the ratio areaRatio of the candidate bounding box to the entire screen is 0.8, then this confidence 2 is 1; if the ratio areaRatio of the candidate bounding box to the entire screen is 0.08 and Weight2 is 0.2, then this confidence 2 is 0.816; if the ratio areaRatio of the candidate bounding box to the entire screen is 0.92, then this confidence 2 is 0.08.

[0083] Determination method for the confidence of the aspect ratio of the candidate bounding box:

[0084] Based on the four sides of the selected bounding box and a preset third weight value, determine the confidence of the aspect ratio of the candidate bounding box. The specific calculation of the confidence of the aspect ratio of the candidate bounding box can refer to the following formula three:

[0085]

[0086] Among them, confidence 3 is the confidence of the aspect ratio of the candidate bounding box, side length 1 is the side length of the upper side of the candidate bounding box, side length 2 is the side length of the lower side of the candidate bounding box, side length 3 is the side length of the left side of the candidate bounding box, and side length 4 is the side length of the right side of the candidate bounding box. min(side length 1, side length 2) represents the shorter side length of the two side lengths, max(side length 1, side length 2) represents the longer side length of the two side lengths, and min(side length 3, side length 4) and max(side length 3, side length 4) are the same. Weight3 is the third weight value, and this third weight value is a preset value. The larger this third weight value is, the greater the impact of the confidence of the aspect ratio of the candidate bounding box on the confidence of the candidate box corresponding to the candidate bounding box.

[0087] Exemplarily, the side lengths of the four sides of the selected bounding box are: 60 cm (side length 1), 57 cm (side length 2), 100 cm (side length 3), 110 cm (side length 4). Weight3 is 1. Confidence 3 is: 0.86.

[0088] For example, the lengths of the four sides of the selected bounding box are: 60 cm (side length 1), 60 cm (side length 2), 100 cm (side length 3), 101 cm (side length 4). Weight 3 is 1. Confidence 3 is: 0.99.

[0089] The difference between the lengths of two opposite sides of the candidate bounding box (such as the side of the candidate bounding box located on the upper side and the side of the candidate bounding box located on the lower side) is inversely proportional to the confidence of the candidate bounding box's side ratio. The closer the lengths of the two opposite sides of the candidate bounding box are, the greater the confidence of the candidate bounding box's side ratio. When the two opposite sides of the candidate bounding box are the same, the confidence of the candidate bounding box's side ratio is the highest.

[0090] The confidence of the neighboring ratio of the candidate bounding box is determined as follows:

[0091] Based on the lengths of the four sides of the candidate bounding box, the preset aspect ratio value and the fourth weight value, the confidence of the adjacent side ratio of the candidate bounding box is determined. The specific calculation of the confidence of the adjacent side ratio of the candidate bounding box can refer to the following formula 4:

[0092]

[0093] ratio1 = max(side length 1 + side length 2, side length 3 + side length 4)

[0094] ratio2=min(side length 1+side length 2, side length 3+side length 4)

[0095]

[0096] Among them, confidence 4 is the confidence of the adjacent edge ratio of the candidate bounding box, side length 1 is the side length of the side of the candidate bounding box located on the upper side, side length 2 is the side length of the side of the candidate bounding box located on the lower side, side length 3 is the side length of the side of the candidate bounding box located on the left side, and side length 4 is the side length of the side of the candidate bounding box located on the right side. ratio1 is the sum of the two side lengths with the largest side length, and ratio2 is the sum of the two side lengths with the smallest side length. aspectRatio is the ratio of the length of a preset document to the width of a preset document, or the ratio of the width of a preset document to the length of a preset document. The preset document is a preset document of the same type as the document to be scanned. For example, if the scanned document is a user's ID card, the preset document length and preset document width are the length and width commonly used for ID cards. Weight4 is a fourth weight value, and the fourth weight value is a preset value. The larger the fourth weight value, the greater the influence of the confidence of the adjacent edge ratio of the candidate bounding box on the confidence of the candidate box corresponding to the candidate bounding box.

[0097] For example, the lengths of the four sides of the selected bounding box are 60 cm (side length 1), 57 cm (side length 2), 100 cm (side length 3), and 110 cm (side length 4), aspectRatio is 0.6, and Weight4 is 1. Then ratio1 is 117 cm (side length 1 + side length 2), and ratio2 is 210 cm (side length 3 + side length 4). Confidence 4 is 0.928.

[0098] For example, the lengths of the four sides of the selected bounding box are 60 cm (side length 1), 61 cm (side length 2), 105 cm (side length 3), and 110 cm (side length 4), aspectRatio is 0.6, and Weight4 is 1. Then ratio1 is 121 cm (side length 1 + side length 2), and ratio2 is 215 cm (side length 3 + side length 4). Confidence 4 is 0.933.

[0099] The closer the ratio of ratio1 to ratio2 is to aspectRatio, the greater the confidence of the adjacent edge ratio of the candidate bounding box. When the ratio of ratio1 to ratio2 is the same as aspectRatio, the confidence of the adjacent edge ratio of the candidate bounding box is the highest.

[0100] Optionally, if aspectRatio cannot be determined, that is, the type of the document to be scanned cannot be determined, the value corresponding to Weight4 is smaller than the value corresponding to Weight4 when the type of the document to be scanned can be determined. For example, when the type of the document to be scanned cannot be determined, Weight4 is 0.1; when the type of the document to be scanned can be determined, Weight4 is 1.

[0101] The type of the document to be scanned may be determined by a recognition result obtained by recognizing the image to be scanned by an image recognition model. The specific method for determining the type of the document to be scanned is not described in detail in this application.

[0102] The confidence of the center position of the candidate bounding box is determined as follows:

[0103] The confidence of the center position of the candidate bounding box is determined based on the center position of the candidate bounding box. The center position of the candidate bounding box can be determined based on the four sides of the candidate bounding box. The specific calculation of the confidence of the center position of the candidate bounding box can refer to the following formula 5:

[0104]

[0105] Among them, confidence 5 is the confidence of the center position of the candidate bounding box, the distance between the center of the quadrilateral and the center of the picture is the distance between the center position of the candidate bounding box and the center position of the image to be scanned, imageLengthWeight is a value determined based on the size of the image to be scanned, and the imageLengthWeight is proportional to the size of the image to be scanned. The larger the size of the image to be scanned, the larger the imageLengthWeight, and vice versa. The smaller the size of the image to be scanned, the smaller the imageLengthWeight. Weight5 is the fifth weight value, and the fifth weight value is a pre-set value. The larger the fifth weight value, the greater the influence of the confidence of the center position of the candidate bounding box on the confidence of the candidate box corresponding to the candidate bounding box.

[0106] The confidence of the angle of the candidate bounding box is determined as follows:

[0107] The confidence of the angle of the opposite side of the candidate bounding box is determined based on the angle between the extension lines of the opposite sides of the candidate bounding box. The specific calculation of the confidence of the angle of the opposite side of the candidate bounding box can refer to the following formula 6:

[0108]

[0109] Among them, confidence 6 is the confidence of the angle of the opposite side of the candidate bounding box, angleUpDown is the angle between the extension line of the upper side of the candidate bounding box and the extension line of the lower side, and angleLeftRight is the angle between the extension line of the left side of the candidate bounding box and the extension line of the right side. The angle of the extension line of the opposite side of the candidate bounding box can be referred to the above introduction, and this application will not repeat it here. When the opposite sides are parallel, the angle of the extension line of the opposite side of the candidate bounding box is 0.

[0110] For example, angleUpDown is 15°, angleLeftRight is 10°, Weight6 is 0.5, and Confidence 6 is: 0.865.

[0111] For example, angleUpDown is 5°, angleLeftRight is 0°, Weight6 is 0.5, and Confidence 6 is: 0.975.

[0112] The confidence of the angle of the opposite side of the candidate bounding box is inversely proportional to the angle of the extension line of the opposite side of the candidate bounding box, that is, the smaller the angle of the extension line of the opposite side of the candidate bounding box, the greater the confidence of the angle of the opposite side of the candidate bounding box, and vice versa. The larger the angle of the extension line of the opposite side of the candidate bounding box, the smaller the confidence of the angle of the opposite side of the candidate bounding box. When angleUpDown and angleLeftRight are both 0, that is, the upper side of the candidate bounding box is parallel to the lower side of the candidate bounding box, and the left side of the candidate bounding box is parallel to the right side of the candidate bounding box, the confidence of the angle of the opposite side of the candidate bounding box is the largest.

[0113] Through the above method, six confidences related to the candidate bounding box are determined by the four angles of the candidate bounding box, the four sides of the candidate bounding box and the angles of the extension lines of the opposite sides of the candidate bounding box. The confidence of the candidate box can be accurately determined by the six confidences related to the candidate bounding box.

[0114] The above describes the method for determining six confidences associated with the candidate bounding boxes. The following describes the method for determining the confidence of the candidate box corresponding to the candidate bounding box based on the six confidences associated with the candidate bounding boxes:

[0115] In a possible embodiment, the candidate box confidence corresponding to the candidate bounding box is the sum of the confidences of the four corners of the candidate bounding box, the confidence of the screen ratio of the candidate bounding box, the confidence of the side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, the confidence of the center position of the candidate bounding box, and the confidence of the opposite side angle of the candidate bounding box.

[0116] For example, if the confidence of the four corners of the candidate bounding box is 0.296, the confidence of the screen ratio of the candidate bounding box is 0.2, the confidence of the opposite side ratio of the candidate bounding box is 0.86, the confidence of the adjacent side ratio of the candidate bounding box is 0.928, the confidence of the center position of the candidate bounding box is 0.6, and the confidence of the opposite side angle of the candidate bounding box is 0.865, then the candidate box confidence corresponding to the candidate bounding box is: 0.296+0.2+0.86+0.928+0.6+0.865=3.749.

[0117] Through the above method, the candidate box confidence corresponding to the candidate bounding box is the sum of the six confidences, and the candidate box confidence can be determined more efficiently and quickly through the six confidences.

[0118] 203. The electronic device determines the candidate bounding box with the highest confidence as the target bounding box, and the target bounding box is used to assist in scanning the document to be scanned.

[0119] Among them, the candidate box confidence corresponding to the target bounding box is greater than the candidate box confidence corresponding to other candidate bounding boxes in at least two candidate bounding boxes. For example, the document to be scanned in the image to be scanned corresponds to three candidate bounding boxes: candidate bounding box 1, candidate bounding box 2, and candidate bounding box 3. The candidate box confidences corresponding to the three candidate bounding boxes are 3.749, 2.14, and 5.63, respectively. Among them, the candidate bounding box with the highest candidate box confidence is candidate bounding box 3 (5.63). Candidate bounding box 3 is determined as the target bounding box.

[0120] Through the above method, the candidate bounding box confidence is jointly determined by the four angles of the candidate bounding box, the four sides of the candidate bounding box and the angle between the extension lines of the opposite sides of the candidate bounding box, so that the determined candidate bounding box confidence is more accurate, and thus the target bounding box determined by the candidate bounding box confidence can realize automatic identification of the bounding box corresponding to the document to be scanned, and the identified bounding box is more accurate.

[0121] See also Figure 4 , Figure 4 A flowchart of another method for determining a bounding box provided in an embodiment of the present application.

[0122] 401. The electronic device determines a plurality of first straight lines from an image to be scanned.

[0123] The first straight line is a straight line related to the bounding box in the image to be scanned and recognized by the model, such as Figure 5 As shown, Figure 5 The image marked with 501 is the image to be scanned. Figure 5 The image marked with 502 is the document to be scanned. Figure 5 The multiple straight lines in the right image are multiple first straight lines, and the first straight lines are straight lines related to the boundary box of the to-be-scanned document 502. The specific identification method of the first straight lines is described below.

[0124] 402. The electronic device takes any first straight line located at a first orientation as an initial straight line, and sequentially traverses the first straight lines located at other orientations except the first orientation in accordance with a preset traversal order, so as to determine at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned; wherein the first orientation is any one of the four orientations of up, down, left, and right.

[0125] The preset traversal order can be clockwise or counterclockwise. If the first direction is up and the traversal order is counterclockwise, the first straight line with the direction of left, the first straight line with the direction of down, and the first straight line with the direction of right are traversed in sequence (up->left->down->right); if the first direction is right and the traversal order is clockwise, the first straight line with the direction of down, the first straight line with the direction of left, and the first straight line with the direction of up are traversed in sequence (right->down->left->up).

[0126] like Figure 6 As shown, Figure 6 If the first straight line 601 is taken as the initial straight line, the first straight line 604, the first straight line 603 and the first straight line 602 are traversed in a clockwise manner, and a candidate bounding box 610 is obtained after the traversal. If the first straight line 605 is taken as the initial straight line, the first straight line 604, the first straight line 603 and the first straight line 602 are traversed in a clockwise manner, and a candidate bounding box 620 is obtained after the traversal. Since the first straight line is a straight line, the lines that do not intersect during the traversal can be delayed so that the two lines that do not intersect have an intersection. For example, Figure 6 The first straight line 601 in the diagram does not intersect with the first straight line 602 , and the first straight line 602 may be extended so that the first straight line 602 intersects with the first straight line 601 .

[0127] In a possible embodiment, the position of the first straight line is the position of the first straight line relative to the transverse center axis or the longitudinal center axis of the image to be scanned.

[0128] The horizontal center axis and the vertical center axis of the image to be scanned are compared to the midpoint of the image to be scanned. If the first straight line A is a horizontal straight line and the first straight line A is located above the horizontal center axis of the image to be scanned, the orientation of the first straight line A is up; if the first straight line B is a horizontal straight line and the first straight line B is located below the horizontal center axis of the image to be scanned, the orientation of the first straight line B is down; if the first straight line C is a vertical straight line and the first straight line C is located to the left of the vertical center axis of the image to be scanned, the orientation of the first straight line C is left; if the first straight line D is a vertical straight line and the first straight line D is located to the right of the vertical center axis of the image to be scanned, the orientation of the first straight line C is right.

[0129] For example, Figure 6 As shown, Figure 6 The straight line marked with 606 is the horizontal center axis 606 of the image to be scanned. Figure 6The straight line marked with 607 is the longitudinal center axis 607 of the image to be scanned. Since the first straight line 601 and the first straight line 605 are above the transverse center axis 606, the orientation of the first straight line 601 and the first straight line 605 is upward; similarly, since the first straight line 603 is below the transverse center axis 606, the orientation of the first straight line 602 is downward, and the longitudinal first straight lines 602 and 604 are the same, and this application will not be repeated here.

[0130] By using the above method, it is ensured that the first straight line will not be missed and all the first straight lines are traversed, and the candidate boundary boxes corresponding to the documents to be scanned in all the images to be scanned can be determined more accurately.

[0131] In a possible implementation, the position coordinates of any corner of the candidate bounding box are within a preset coordinate range, and the angle of any corner of the candidate bounding box is within a preset angle range.

[0132] The preset coordinate range is the range of the image to be scanned. For example, if the length of the image to be scanned is 17 cm and the width is 10 cm, and a coordinate system is established with the lower left corner of the image to be scanned as the origin, the preset coordinate range may be (0-17, 0-10). That is, the horizontal coordinate of any corner of the candidate bounding box is in the range of 0-17, and the vertical coordinate is in the range of 0-10. In order to ensure that the candidate bounding box is close to a rectangle (to ensure that the angle of any corner of the candidate bounding box is close to 90°), the preset angle range may be (70°-100°).

[0133] 403. The electronic device determines a candidate box confidence corresponding to the candidate bounding box based on four angles of the candidate bounding box, four sides of the candidate bounding box, and an angle between extension lines of opposite sides of the candidate bounding box.

[0134] This step can refer to the description in the above step 202, and this application will not elaborate on it here.

[0135] 404. The electronic device determines the candidate bounding box with the highest confidence as the target bounding box, and the target bounding box is used to assist in scanning the document to be scanned.

[0136] This step can refer to the description in the above step 203, and this application will not elaborate on it here.

[0137] See also Figure 7 , Figure 7 A flowchart of another method for determining a bounding box provided in an embodiment of the present application is shown below.

[0138] 701. The electronic device inputs the image to be scanned into an edge scanning model to obtain a plurality of second straight lines.

[0139] The edge scanning model may be a deep edge detection model or a lightweight deep edge detection model, and the lightweight deep edge detection model may be specifically Unet, mobilenetV2, mobilenetV3, RepVGG, etc. The edge scanning model may identify the edge of the document to be scanned in the image to be scanned. Figure 8 As shown, Figure 8 The straight line marked with 801 is a second straight line.

[0140] In a possible embodiment, the electronic device inputs the image to be scanned into an edge detection model to obtain a fourth straight line output by the edge detection model; merges the fourth straight lines that meet the merging conditions to obtain multiple second straight lines, and the merging conditions are that the angle between any two straight lines is less than a first threshold, or the distance between the midpoints of any two straight lines is less than a second threshold.

[0141] The first threshold is a preset value, and the second threshold is also a preset value. The first threshold and the second threshold are used to merge two similar straight lines into one straight line.

[0142] Optionally, if the output of the edge detection model is a first image (such as the output of a Unet model), the first image is binarized to obtain a second image, and straight line segment detection is performed on the second image to obtain a fourth straight line.

[0143] The line segment detection may be a Hough line segment detection. The image binarization is to set the grayscale value of the pixel points on the image to 0 or 255, so that the entire image appears in black and white. Binarization can reduce the amount of data in the image, thereby highlighting the contour.

[0144] 702. The electronic device inputs the image to be scanned into a straight line detection model to obtain a plurality of third straight lines.

[0145] The line detection model can be a traditional edge detection model, specifically, it can be: LSD edge detection method, Hough line detection method and EDline method. The line detection model can detect all the straight lines in the image to be scanned. That is to say, compared with the above-mentioned edge detection model, the line detection model will detect the straight lines outside the contour, and the straight lines detected by the line detection model are more accurate. For example Figure 8 As shown, Figure 8 The straight line marked with 802 is a third straight line.

[0146] In a possible embodiment, the electronic device inputs the image to be scanned into a line detection model to obtain a fifth line output by the line detection model; the electronic device merges the fifth lines that meet the merging condition to obtain multiple third lines.

[0147] The merging conditions may refer to the description in the above step 701, and this application will not elaborate on them here.

[0148] 703. The electronic device determines a plurality of first straight lines based on the plurality of second straight lines and the plurality of third straight lines.

[0149] In a possible embodiment, multiple first straight lines are determined based on multiple second straight lines and multiple third straight lines, including: using multiple third straight lines to filter multiple second straight lines to obtain multiple filtered second straight lines; using multiple second straight lines to filter multiple third straight lines to obtain multiple filtered third straight lines; merging the filtered multiple second straight lines and the filtered multiple third straight lines to obtain multiple first straight lines.

[0150] Among them, using multiple third straight lines to screen multiple second straight lines is to screen out the second straight lines that are not related to any third straight line. Specifically, the second straight lines whose distance from any third straight line is greater than a preset third threshold value among the second straight lines can be screened out. For example, five second straight lines: second straight line 1, second straight line 2, second straight line 3, second straight line 4, and second straight line 5. Among them, if the distance between second straight line 5 and any third straight line is greater than the preset third threshold value, second straight line 5 is screened out, and the multiple second straight lines after screening are: second straight line 1, second straight line 2, second straight line 3, and second straight line 4.

[0151] For example, Fig.9A As shown, Fig.9A The number 901 in the figure indicates multiple second straight lines. Fig.9A The third straight lines 902 are multiple third straight lines. The multiple second straight lines are screened using the multiple third straight lines to obtain the screened second straight lines 903.

[0152] Similarly, using multiple second straight lines to filter multiple third straight lines is to filter out the third straight lines that are not related to any second straight line. Specifically, the third straight lines whose distance from any second straight line is greater than the preset third threshold value can be filtered out. For example, seven third straight lines: third straight line 1, third straight line 2, third straight line 3, third straight line 4, third straight line 5, third straight line 6 and third straight line 7. Among them, the distances between third straight line 1 and third straight line 7 and any third straight line are greater than the preset third threshold value, then third straight line 1 and third straight line 7 are filtered out, and the multiple third straight lines after filtering are obtained as follows: third straight line 2, third straight line 3, third straight line 4, third straight line 5 and third straight line 6.

[0153] For example, Fig. 9B As shown, Fig. 9BThe 903 mark is a number of third straight lines. Fig. 9B The plurality of second straight lines are marked with 904. The plurality of third straight lines are screened using the plurality of second straight lines to obtain the screened third straight line 905.

[0154] Among them, the electronic device merges the multiple second straight lines after screening and the multiple third straight lines after screening to obtain multiple first straight lines, specifically: any two straight lines with similar distances are merged into the same straight line, and the any two straight lines can be the first straight line and the second straight line, and the similar distance can be that the distance between the midpoints is less than a preset threshold, or that the angle between the two straight lines is less than a preset threshold.

[0155] 704. The electronic device takes any first straight line located at the first orientation as the initial straight line, and traverses the first straight lines located at the remaining orientations except the first orientation in sequence according to a preset traversal order to determine at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned; wherein the first orientation is any one of the four orientations of up, down, left, and right.

[0156] This step can refer to the description in the above step 402, and this application will not elaborate on it here.

[0157] 705. The electronic device determines a candidate box confidence corresponding to the candidate bounding box based on four angles of the candidate bounding box, four sides of the candidate bounding box, and an angle between extension lines of opposite sides of the candidate bounding box.

[0158] This step can refer to the description in the above step 202, and this application will not elaborate on it here.

[0159] 706. The electronic device determines the candidate bounding box with the highest confidence as the target bounding box, and the target bounding box is used to assist in scanning the document to be scanned.

[0160] This step can refer to the description in the above step 203, and this application will not elaborate on it here.

[0161] Through the above method, the straight lines obtained by the two models are cross-screened to filter out the straight lines irrelevant to the candidate bounding box, thereby accurately determining the first straight line.

[0162] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining a bounding box, characterized in that: The method comprises: Determine at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned; Determine a candidate box confidence corresponding to the candidate bounding box based on four angles of the candidate bounding box, four sides of the candidate bounding box, and an angle between extensions of opposite sides of the candidate bounding box; A candidate bounding box with the highest confidence level among the candidate boxes is determined as a target bounding box, and the target bounding box is used to assist in scanning the document to be scanned.

2. The method according to claim 1, characterized in that The determining, based on four angles of the candidate bounding box, four sides of the candidate bounding box, and angles between extensions of opposite sides of the candidate bounding box, a candidate box confidence corresponding to the candidate bounding box includes: Determining the confidence of the four corners of the candidate bounding box based on the four angles of the candidate bounding box; Based on the four sides of the candidate bounding box, determine the confidence of the screen ratio of the candidate bounding box, the confidence of the opposite side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, and the confidence of the center position of the candidate bounding box; Determining the confidence of the angle of the opposite side of the candidate bounding box based on the angle of the extension line of the opposite side of the candidate bounding box; The candidate box confidence corresponding to the candidate bounding box is determined based on the confidence of the four corners of the candidate bounding box, the confidence of the screen ratio of the candidate bounding box, the confidence of the side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, the confidence of the center position of the candidate bounding box, and the confidence of the side angle of the candidate bounding box.

3. The method according to claim 2, characterized in that The candidate box confidence corresponding to the candidate bounding box is the sum of the confidences of the four corners of the candidate bounding box, the confidence of the screen ratio of the candidate bounding box, the confidence of the opposite side ratio of the candidate bounding box, the confidence of the adjacent side ratio of the candidate bounding box, the confidence of the center position of the candidate bounding box, and the confidence of the opposite side angle of the candidate bounding box.

4. The method according to any one of claims 1 to 3, characterized in that: The step of determining at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned includes: Determining a plurality of first straight lines from the image to be scanned; Taking any one of the first straight lines located at the first orientation as an initial straight line, and sequentially traversing the first straight lines located at the remaining orientations except the first orientation according to a preset traversal order, so as to determine at least two candidate bounding boxes corresponding to the document to be scanned in the image to be scanned; The first direction is any one of the four directions of up, down, left and right.

5. The method according to claim 4, characterized in that The orientation of the first straight line is the orientation of the first straight line relative to the transverse center axis or the longitudinal center axis of the image to be scanned.

6. The method according to any one of claims 1 to 5, characterized in that: The position coordinates of any corner of the candidate bounding box are within a preset coordinate range, and the angle of any corner of the candidate bounding box is within a preset angle range.

7. The method according to any one of claims 4 to 6, characterized in that: The step of determining a plurality of first straight lines from the image to be scanned comprises: Inputting the image to be scanned into an edge scanning model to obtain a plurality of second straight lines; Inputting the image to be scanned into a straight line detection model to obtain a plurality of third straight lines; Based on the plurality of second straight lines and the plurality of third straight lines, a plurality of first straight lines are determined.

8. The method according to claim 7, characterized in that The determining of a plurality of first straight lines based on the plurality of second straight lines and the plurality of third straight lines comprises: Using the plurality of third straight lines, screening the plurality of second straight lines to obtain a plurality of screened second straight lines; Using the plurality of second straight lines, screening the plurality of third straight lines to obtain a plurality of screened third straight lines; The filtered multiple second straight lines and the filtered multiple third straight lines are combined to obtain multiple first straight lines.

9. The method according to claim 7 or 8, characterized in that: The step of inputting the image to be scanned into an edge scanning model to obtain a plurality of second straight lines comprises: Inputting the image to be scanned into an edge detection model to obtain a fourth straight line output by the edge detection model; Merging the fourth straight lines that meet a merging condition to obtain a plurality of second straight lines, wherein the merging condition is that the angle between any two straight lines is less than a first threshold, or the distance between the midpoints of any two straight lines is less than a second threshold; The step of inputting the image to be scanned into a straight line detection model to obtain a plurality of third straight lines comprises: Inputting the image to be scanned into a straight line detection model to obtain a fifth straight line output by the straight line detection model; The fifth straight lines satisfying the merging condition are merged to obtain a plurality of third straight lines.

10. An electronic device, comprising one or more memories and one or more processors, characterized in that: The memory is used to store a computer program; the processor is used to call the computer program so that the electronic device executes the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The computer storage medium stores computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 9.

Images