Order payment processing method and its device, equipment, medium and product
By using pre-trained models on the e-commerce platform to distinguish and classify payment bill images, the complex payment links of cross-border e-commerce platforms are solved, the automation efficiency and accuracy of order payment are improved, and the business closed loop is realized.
Patent Information
- Application Number
- CN202210217223.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-07
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-03-07
AI Technical Summary
In areas where infrastructure development is insufficient, cross-border e-commerce platforms cannot directly call bank payment interfaces in user payment links, resulting in complex and inefficient payment processes, and lack of adaptive technical conditions to achieve fully automated business logic.
Using the bill judgment model, text detection and recognition model and text classification model pre-trained to converge state, the payment bill image identification, text recognition and payment information are classified on the images to be identified submitted by the audience user in the live broadcast room, and the payment information is pushed to the anchor user terminal. , and respond to the payment confirmation instruction to make order payment reversal.
It improves the efficiency and accuracy of image recognition processing of payment bills, reduces manual verification and order operations, improves the automation efficiency of order placing e-commerce orders, and realizes a closed-loop business.
Smart Images

Figure CN114612917B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of e-commerce information technology, and in particular to an order payment processing method and its corresponding device, computer equipment, computer-readable storage medium, and computer program product. Background Art
[0002] On the one hand, with the continuous transformation of online media forms, the business scenarios of e-commerce platforms are constantly changing, from early page shopping to live broadcast shopping, which has led to changes in the backend business logic; on the other hand, in cross-border e-commerce platforms that operate with multiple independent sites, different independent sites are located in different physical spaces, and need to adapt to the legal, cultural, and infrastructure conditions of different regions to implement different business logics, which also makes the business logic more and more complex. In view of this, the fully automated business logic from online ordering to payment to logistics in the existing shopping platforms in advanced regions may not be applicable in some areas with insufficient infrastructure development.
[0003] In a typical real-world application scenario, after a user places an order on an e-commerce platform, the payment process cannot directly call the bank payment interface to make payment. Therefore, the user needs to enter the bank's application or make another payment offline, and finally communicate with the seller to confirm the payment fact before the online shopping process can be completed. This phenomenon is caused by the market environment and legal environment of the region concerned on the one hand, and the lack of adaptive supporting technical conditions to systematize the services required for this situation on the other hand. Therefore, it is of practical significance to implement relevant business logic in the e-commerce platform and improve the execution efficiency of e-commerce orders to meet the needs of the corresponding regions. Summary of the invention
[0004] The primary purpose of the present application is to solve at least one of the above problems and to provide an order payment processing method and its corresponding device, computer equipment, computer-readable storage medium, and computer program product.
[0005] In order to meet the various objectives of this application, this application adopts the following technical solutions:
[0006] An order payment processing method provided for one of the purposes of this application comprises the following steps:
[0007] Obtain the image to be identified submitted by the audience in the live broadcast room;
[0008] Using a bill decision model that has been pre-trained to a convergent state to classify and decide the image to be identified, so as to determine whether the image to be identified is a payment bill image;
[0009] Using a text detection and recognition model that has been pre-trained to a convergent state to perform text recognition on the payment note image to obtain multiple pieces of text information therein;
[0010] Using a text classification model that has been pre-trained to a convergent state to classify the text information of each item, so as to classify and identify payment information from the text information, wherein the payment information includes a payment amount and a remark order number, and the payment information is pushed to a terminal device of the anchor user in the live broadcast room for display;
[0011] In response to the payment confirmation instruction submitted by the anchor user, a payment offset event is initiated, and the payment amount in the payment information is used to offset the amount to be paid of the order corresponding to the remark order number.
[0012] In an extended embodiment, the training process of the bill decision model includes the following steps:
[0013] Calling a training sample in the first training data set, the training sample includes a positive sample and a negative sample, the positive sample is a payment bill image, and the negative sample is a non-payment bill image;
[0014] The image feature extraction network in the bill judgment model is used to extract deep semantic information from the training sample to obtain image feature information;
[0015] Using the binary classifier in the bill judgment model to classify and map the image feature information to obtain corresponding classification labels;
[0016] The loss value of the classification label is calculated according to whether the training sample is a positive sample or a negative sample. When the loss value has not converged, the weight parameters of the bill decision model are corrected according to the loss value and the iterative training is continued. The training is terminated when the loss value reaches a convergence state.
[0017] In a further embodiment, a text detection and recognition model pre-trained to a convergent state is used to perform text recognition on the payment note image to obtain multiple text information therein, including the following steps:
[0018] The text detection and recognition model performs text region recognition on the payment note image to identify one or more text regions in the payment note image;
[0019] The text detection and recognition model cuts out text region images corresponding to the multiple text regions from the payment note image;
[0020] The text detection and recognition model performs text recognition on each of the text area images to obtain corresponding text information.
[0021] In a further embodiment, a text classification model pre-trained to a convergent state is used to classify each item of text information, including the following steps:
[0022] The text classification model performs word vector encoding on all text information to obtain a corresponding word vector sequence;
[0023] The text classification model extracts deep semantic information from the word vector sequence to obtain corresponding text feature information;
[0024] The text classification model performs classification mapping on the text feature information to obtain classification labels corresponding to each piece of text information, wherein the classification labels include classification labels corresponding to the payment amount and the remark order number.
[0025] In a further embodiment, in response to the payment confirmation instruction submitted by the anchor user, a payment reversal event is initiated, including the following steps:
[0026] Receive the payment confirmation instruction submitted by the anchor user, and obtain the remark order number in the payment information corresponding to the payment confirmation instruction;
[0027] Call the order corresponding to the remarked order number, and offset the amount to be paid in the order with the payment amount included in the payment information;
[0028] Determine whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the judgment is true, trigger the order delivery business process, otherwise, send a notification message to the anchor user and the audience user.
[0029] In an extended embodiment, it is determined whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the determination is established, the following steps are included:
[0030] Pushing a notification message to the live broadcast room indicating that the user corresponding to the order has completed payment, wherein the notification message includes a link to the ordering business process of the product corresponding to the order;
[0031] In response to any user in the live broadcast room triggering an access instruction of the link, an order page for the product corresponding to the order is pushed to the user, and a corresponding notification message is sent to the live broadcast user.
[0032] An order payment processing device provided to meet one of the purposes of the present application includes: an image submission module, an image classification module, a text recognition module, a text classification module, and a payment reversal module. The image submission module is used to obtain the image to be recognized submitted by the audience user in the live broadcast room; the image classification module is used to classify and judge the image to be recognized using a bill judgment model pre-trained to a convergent state to determine whether the image to be recognized is a payment bill image; the text recognition module is used to perform text recognition on the payment bill image using a text detection and recognition model pre-trained to a convergent state to obtain multiple text information therein; the text classification module is used to classify each of the text information using a text classification model pre-trained to a convergent state to classify and recognize payment information from the text information, the payment information including the payment amount and the remark order number, and push the payment information to the terminal device of the anchor user in the live broadcast room for display; the payment reversal module is used to respond to the payment confirmation instruction submitted by the anchor user, initiate a payment reversal event, and use the payment amount in the payment information to reverse the amount to be paid of the order corresponding to the remark order number.
[0033] In an extended embodiment, the order payment processing device of the present application includes a training unit for executing a training process for the bill judgment model, and the training unit includes: a sample calling module for calling a training sample in a first training data set, wherein the training sample includes a positive sample and a negative sample, wherein the positive sample is a payment bill image, and the negative sample is a non-payment bill image; a semantic extraction module, configured to use the image feature extraction network in the bill judgment model to extract deep semantic information from the training sample to obtain image feature information; a classification mapping module, configured to use the binary classifier in the bill judgment model to perform classification mapping on the image feature information to obtain a corresponding classification label; an iterative decision module, configured to calculate the loss value of the classification label according to whether the training sample belongs to a positive sample or a negative sample, and when the loss value has not converged, to correct the weight parameters of the bill judgment model according to the loss value and continue iterative training, and to terminate the training when the loss value reaches a convergence state.
[0034] In a further embodiment, the text recognition module includes: a region recognition submodule, configured to perform text region recognition on the payment bill image by the text detection and recognition model to identify one or more text regions in the payment bill image; a region cropping submodule, configured to crop text region images corresponding to the multiple text regions from the payment bill image by the text detection and recognition model; and a text recognition submodule, configured to perform text recognition on each of the text region images by the text detection and recognition model to obtain corresponding text information.
[0035] In a further embodiment, the text classification module includes: a vector encoding submodule, which is configured to use the text classification model to perform word vector encoding on all the text information to obtain a corresponding word vector sequence; a feature extraction submodule, which is configured to use the text classification model to extract deep semantic information from the word vector sequence to obtain corresponding text feature information; a label determination submodule, which is configured to use the text classification model to classify and map the text feature information to obtain a classification label corresponding to each piece of text information, wherein the classification label includes a classification label corresponding to the payment amount and the remark order number.
[0036] In a further embodiment, the payment offset module includes: an instruction response submodule, used to receive the payment confirmation instruction submitted by the anchor user, and obtain the remark order number in the payment information corresponding to the payment confirmation instruction; an order calling submodule, used to call the order corresponding to the remark order number, and offset the amount to be paid in the order with the payment amount contained in the payment information; a flow control submodule, used to determine whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the judgment is established, the order delivery business process is triggered; otherwise, a notification message is sent to the anchor user and the audience user.
[0037] In an extended embodiment, the flow control submodule determines whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the judgment is established, it also includes running the following functions: pushing a notification message to the live broadcast room indicating that the user corresponding to the order has completed the payment, and the notification message includes a link to the order business process of the product corresponding to the order; in response to any user in the live broadcast room triggering an access instruction of the link, pushing the order page of the product corresponding to the order to the user, and sending a corresponding notification message to the live broadcast user.
[0038] A computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the order payment processing method described in the present application.
[0039] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the order payment processing method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the method are executed.
[0040] A computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of the present application.
[0041] Compared with the prior art, the advantages of the present application are as follows: the present application first identifies the image to be identified as a payment bill image for the image to be identified submitted by the audience in the live broadcast room, and after determining that it is a payment bill image, performs text recognition on it to obtain multiple text information, and then identifies the payment information according to the preset business classification, including the remark order number and the payment amount, and finally uses the payment amount to offset the e-commerce order corresponding to the remark order number to realize the business closed loop of the order process. The entire process is implemented based on artificial intelligence technology, and in the middle, various neural network models are used to realize the functions of payment bill image identification, text recognition, and payment information recognition respectively, which can ensure the recognition and processing efficiency and accuracy of the payment bill image, thereby effectively avoiding manual verification of the payment bill image and excessive manual operations on the order, improving the automation efficiency of e-commerce orders, and thus completing the business closed loop of e-commerce orders. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0043] Figure 1 A flowchart of a typical embodiment of the order payment processing method of the present application;
[0044] Figure 2 A flowchart of the training process of the bill judgment model of this application;
[0045] Figure 3 A schematic diagram of the process of performing text recognition by a text detection and recognition model in an embodiment of the present application;
[0046] Figure 4 A schematic diagram of a flow chart of text information classification and recognition performed by a text classification model in an embodiment of the present application;
[0047] Figure 5 This is a flowchart of the order cancellation process in the embodiment of the present application;
[0048] Figure 6 This is a principle block diagram of the order payment processing device of the present application;
[0049] Figure 7 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION
[0050] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
[0051] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.
[0052] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.
[0053] It will be understood by those skilled in the art that the terms "client", "terminal" and "terminal device" as used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capability, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (aviation, shipping and / or land), or adapted and / or configured to run locally, and / or in a distributed form, running at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, such as a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or may be a smart TV, a set-top box and other devices.
[0054] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.
[0055] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.
[0056] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.
[0057] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.
[0058] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.
[0059] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.
[0060] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.
[0061] The order payment processing method of the present application can be programmed as a computer program product and deployed in the cluster server of the e-commerce platform to run and implement. Figure 1 In a typical embodiment, the order payment processing method of the present application includes the following steps:
[0062] Step S1100: Obtain the image to be identified submitted by the audience user in the live broadcast room:
[0063] In the exemplary application scenario of the present application, an independent site of a cross-border e-commerce platform launches a live broadcast on the basis of its online store, and the host user promotes goods to the audience users in the live broadcast room. The audience users place orders for goods in the online store of the independent site. The independent site creates an order for the product for this order event in the background, generates a corresponding order number and amount to be paid, and sends a payment notification message to the corresponding audience users. The payment notification message includes the order number, amount to be paid, payment account information, etc., so as to instruct the audience users to complete the payment of the order through other payment channels on their own, thereby realizing a closed loop of the order business.
[0064] The audience user can perform the corresponding remittance based on the payment notification message in other applications suitable for executing online payments on his mobile terminal, so as to obtain a screenshot of the payment note information interface returned after the payment. The audience user can also handle the remittance business corresponding to the payment notification message through methods such as bank ATMs or manual remittances, and finally obtain the corresponding paper payment notes. The paper payment notes can obtain the corresponding electronic images by scanning or photographing. Usually, when handling payment services, audience users are generally required to provide the order number as the remark order number. Therefore, the payment information contained in the obtained payment note usually includes the remark order number, payment amount, payment time, etc. Among them, the remark order number and payment amount play a relatively more critical role in realizing the closed loop of order business.
[0065] As mentioned above, audience users can convert payment receipts obtained by completing payments outside the live broadcast room into image information by means of screenshots, photography, scanning, etc. The image information can be further submitted by the audience user to the background system of the live broadcast room so that this application can realize the recognition of payment information based on it.
[0066] In order to realize the recognition of payment information, the audience user is allowed to submit image files through a preset access portal in the graphical user interface of his live broadcast room. The image text will be regarded as an image to be recognized by the server of the live broadcast room, that is, the cluster server of the e-commerce platform, to determine whether it is a payment note image containing the payment information.
[0067] To this end, in one embodiment, an image publishing control is provided in the input bar of the public screen flow speech area of the live broadcast room. After the audience user touches the image publishing control, he can further determine an image to be identified by any method such as screenshot, shooting, searching for local pictures, etc., and then determine and send it to the public screen flow speech area. Correspondingly, this image to be identified is also submitted to the server of the e-commerce platform through the background. This implementation method allows audience users to publish their payment information to the public screen flow speech area to realize the public release of user payment information in the live broadcast room, which helps to stimulate transaction traffic in the live broadcast room.
[0068] Step S1200: using a bill decision model that has been pre-trained to a convergent state to classify and decide the image to be identified, so as to determine whether the image to be identified is a payment bill image:
[0069] Taking the above-mentioned release of the image to be identified from the public screen flow area as an example, due to errors or the need to send different types of pictures, the images to be identified uploaded to the e-commerce platform server may not all be payment bill images containing the payment information. In order to improve the efficiency of extracting payment information in the background, this application provides a bill judgment model for image type identification of the image to be identified, so as to quickly determine whether it belongs to a payment bill image. When the image to be identified is not the payment bill image, it is ignored so as to be submitted to other business logic for processing. For example, when the image to be identified is an expression image, it is directly pushed to the public screen flow area; when the image to be identified is a payment bill image, it is submitted to the subsequent steps of this application for further payment information identification processing. As a result, the subsequent steps of this application can centrally process payment bill images, thereby improving the centralized processing efficiency of subsequent steps.
[0070] The bill judgment model is implemented using a convolutional neural network model. For example, the basic model of the Resnet series version is used as an image feature extraction network to extract deep semantic information from the image to be identified, obtain the corresponding image feature information, and then classify and discriminate the image feature information with the help of a classifier to determine whether it is the payment bill image. It is not difficult to understand that the bill judgment model is pre-trained to a convergence state so that it can learn the representation of the image to be identified and judge whether it belongs to the payment bill image based on it, so that it can be put into use. The subsequent embodiments of this application will further reveal the training process of the bill judgment model, which will not be discussed here for the time being.
[0071] Step S1300: Use a text detection and recognition model that has been pre-trained to a convergent state to perform text recognition on the payment note image to obtain multiple pieces of text information:
[0072] In order to realize text recognition of the payment bill image, the present application is implemented using a text detection and recognition model, which is composed of a line text detection network and a line text recognition network, wherein the line text detection network is used to detect the line text boxes in the payment bill image, and the line text recognition network is used to detect the text information of the local image corresponding to each line text box.
[0073] In one embodiment, the line text detection network is first used to detect the payment bill image, and the detection method is text detection based on candidate boxes. Specifically, the pixel features of the payment bill image are extracted, and the pixel features are used to set a number of default boxes (also called anchor boxes) to generate some corresponding candidate text boxes, and a series of adjustments and screening are performed on these candidate text boxes. Further, the NMS (non-maximum suppression) algorithm is called to obtain the final text boundary box, that is, the line text box and its corresponding coordinate position in the payment bill image. The default box can be flexibly set by those skilled in the art based on prior knowledge or experimental experience. The line text detection network can be R2CNN, TextBoxes++, SegLink, RFCN, CTPN, EAST, etc., which are preferably called by those skilled in the art as needed. Thus, the local image is intercepted according to the coordinate position of the line text box corresponding to the payment bill image, and the corresponding line text image and its line text image coordinates are obtained, and the line text image contains one or more texts.
[0074] Further, a line text recognition network is called to recognize each of the line text images. The recognition is text line recognition. Specifically, the visual features of the line text image are extracted in the convolutional backbone network to obtain a character feature sequence, which is input into a sequential encoder to aggregate part or all of the features of the character feature sequence to obtain a coded character feature sequence, which is then input into a decoder for transcription to obtain the corresponding text, i.e., the line text data. The line text recognition network can be CNN+RNN+CTC, CNN+RNN based on Attention ion, etc., which can be called by technicians in this field as needed.
[0075] It is not difficult to understand that the text detection and recognition model is implemented using a convolutional neural network and is trained to a convergence state in advance so that it can learn the ability to obtain one or more text information from the payment note image. The training process can be flexibly implemented by technical personnel in this field according to various training ideas without affecting the creative embodiment of the present application.
[0076] The various text information identified from the payment note image varies depending on the information items included in the corresponding payment note. However, the payment information required to complete the order loop, mainly including the remark order number and payment amount, is usually indispensable. Therefore, the text information will contain these indispensable key information, and of course can also flexibly include other richer information.
[0077] Step S1400: Classify the text information of each item using a text classification model that has been pre-trained to a convergent state, so as to classify and identify payment information from the text information, wherein the payment information includes a payment amount and a remark order number, and push the payment information to the terminal device of the anchor user in the live broadcast room for display:
[0078] In order to realize the utilization of various text information recognized from the payment bill image, a text classification model is further used to extract deep semantic information from the word vector sequence formed by the encoding of each text, and classify and map it to a preset classification space, so that each text information can be correspondingly matched to each preset classification label of the classification space. These classification labels include but are not limited to indicating the payment amount, remark order number, payment time, etc. in the payment information.
[0079] Similarly, the text classification model is suitable for implementation using a convolutional neural network model, which is trained to a convergence state in advance so that it can learn to perform classification mapping based on the encoding results corresponding to multiple text information to determine the classification labels corresponding to each piece of text information and put it into use. The training process of the text classification model can be flexibly determined by those skilled in the art based on the specific basic model adopted, without affecting the creative embodiment of the present application. In an embodiment measured by the applicant, LSTM was used as the basic model of the text classification model, and good test results were obtained. It can achieve the correct classification of the text information and meet the application requirements of the e-commerce platform. Of course, other similar basic models can also be used to replace the implementation, and those skilled in the art can flexibly choose.
[0080] After the payment amount and the remark order number in the payment information are determined, a unicast message can be generated, which contains the payment information, especially the payment amount and the remark order number, and the summary information of the order corresponding to the remark order number is called and included in the unicast message. The unicast message is pushed to the terminal device of the anchor user in the live broadcast room, and displayed in the graphical user interface of the anchor user after being parsed by the terminal device, and a corresponding confirmation button is provided so that the anchor user can verify and confirm it; if necessary, other correction buttons can also be provided so that the anchor user can temporarily archive the corresponding transaction for subsequent processing when the confirmation fails, or the anchor user can directly perform error correction processing to correct the key payment information such as the payment amount or the remark order number. In any case, once the anchor user confirms that the payment information is correct, the confirmation button can be operated to trigger the corresponding payment confirmation instruction.
[0081] Step S1500: In response to the payment confirmation instruction submitted by the anchor user, a payment offset event is initiated, and the amount to be paid of the order corresponding to the remark order number is offset with the payment amount in the payment information:
[0082] After the anchor user triggers the payment information, the corresponding payment confirmation instruction is triggered and sent to the cluster server of the e-commerce platform. The server responds to the payment confirmation instruction and initiates the payment reversal event to implement the reversal process of the pending payment amount of the order corresponding to the remark order number specified in the payment information.
[0083] The offset process is mainly to offset the amount to be paid of the order corresponding to the order number noted in the payment information with the payment amount in the payment information, that is, to obtain the amount to be paid minus the payment amount in the payment information to obtain the difference after offset. Generally speaking, a valid transaction can only be reached when the payment amount is equal to the amount to be paid and the difference is zero, thereby completing the payment business process of the order and realizing a closed business loop. In some special cases, the payment amount in the payment information is also allowed to be greater than the amount to be paid, so that the difference after offset is greater than zero, such as when the audience user needs to pay additional taxes on the basis of the amount to be paid; in another special case, it can even be allowed that the payment amount in the payment information is less than the amount to be paid, so that the difference after offset is less than zero, which is applicable to discounted goods. In any case, due to the existence of the link of confirmation by the anchor user, it is allowed to exist selectively. However, if the convenience of automatically driving the order delivery link is to achieve an automatic closed loop of business, it can generally be set that when the payment amount in the payment information is greater than or equal to the amount to be paid, that is, when the difference after offset is less than or equal to zero, the triggering conditions of the automatic closed loop of business can be met and enter the subsequent business link.
[0084] It is not difficult to understand that after completing the above-mentioned cancellation process, the payment business link of the order is theoretically completed. In this process, the anchor user is prevented from manually verifying the relevant images to be identified to the greatest extent, and only the payment information provided by the server is needed to confirm the payment situation. Since this application makes extensive use of artificial intelligence technology, its intelligence level is relatively high, and the image recognition efficiency, text recognition accuracy, text information classification efficiency, and operation efficiency are all good, thus greatly improving the execution efficiency of e-commerce platform orders.
[0085] According to the above typical embodiments of the present application and its alternative embodiments, it can be known that the present application first identifies the image to be identified submitted by the audience in the live broadcast room as a payment bill image. After determining that it is a payment bill image, it performs text recognition on it to obtain multiple text information, and then identifies the payment information according to the preset business classification, including the remark order number and the payment amount. Finally, the payment amount is used to offset the e-commerce order corresponding to the remark order number to realize the business closed loop of the order placement process. The entire process is implemented based on artificial intelligence technology, and in the middle, various neural network models are used to realize the functions of payment bill image identification, text recognition, and payment information recognition respectively, which can ensure the recognition and processing efficiency and accuracy of the payment bill image, thereby effectively avoiding manual verification of the payment bill image and excessive manual operations on the order, improving the automation efficiency of e-commerce orders, and thus improving the business closed loop of e-commerce orders.
[0086] See also Figure 2 In an extended embodiment, the training process of the bill decision model includes the following steps:
[0087] Step S2100: call a training sample in the first training data set, wherein the training sample includes a positive sample and a negative sample, wherein the positive sample is a payment bill image, and the negative sample is a non-payment bill image:
[0088] A training data set is prepared, in which images as training samples are stored, and the training samples in the training data set are marked as positive samples and negative samples respectively according to whether the images are payment bill images, so as to call each training sample in the training data set one by one to perform iterative training on the bill judgment model of the present application. In each iterative training process, a training sample in the training data set is called and input into the bill judgment model.
[0089] Step S2200: Use the image feature extraction network in the bill judgment model to extract deep semantic information from the training sample to obtain image feature information:
[0090] The image feature extraction network in the bill judgment model is responsible for representation learning of the training samples so as to extract their deep semantic information and obtain corresponding image feature information. The image feature extraction network can use Resnet, CNN, etc. as the basic network.
[0091] Step S2300: Use the binary classifier in the bill judgment model to classify and map the image feature information to obtain corresponding classification labels:
[0092] After obtaining the image feature information, it is mapped to a classifier through a fully connected layer. The classifier is preferably a binary classifier in this embodiment. Therefore, its mapping result is a binarization result to obtain whether the image feature information is mapped to a classification label corresponding to the payment bill image.
[0093] Step S2400, calculating the loss value of the classification label according to whether the training sample is a positive sample or a negative sample, and when the loss value has not converged, correcting the weight parameter of the bill decision model according to the loss value and continuing iterative training, and terminating the training when the loss value reaches a convergence state:
[0094] As mentioned above, the sample attributes corresponding to each training sample have been marked in the training data set, that is, positive samples or negative samples. With this as the supervision label, the cross entropy function can be applied to calculate the loss value of the binarization result obtained by the binary classifier, and then determine whether the bill judgment model has been trained to a convergence state based on whether the loss value reaches the preset value. When the bill judgment model has reached a convergence state, the model training can be terminated; when the bill judgment model has not reached a convergence state, it is necessary to implement a gradient update on the model based on the loss value, and correct the weight parameters in each link of the model through back propagation, so that the loss function of the model is closer to convergence, and then jump to step S2100 to call the next training sample to continue iterative training of the model, and so on, until the model is trained to a convergence state.
[0095] This embodiment is adapted to the requirements of this application. A training data set is used in advance to train the bill judgment model so that the model can be put into use in this application after being trained to convergence. Since the model uses a binary classifier for classification judgment, it is relatively simpler and more efficient and can be trained to a convergence state more quickly. When it is put into the inference stage, it can quickly and efficiently determine whether the image to be identified belongs to a payment bill image, thereby avoiding a large amount of invalid calculations in the subsequent text recognition steps, thereby improving the operating efficiency of the technical solution of this application.
[0096] See also Figure 3In a further embodiment, the step S1300, using a text detection and recognition model pre-trained to a convergent state to perform text recognition on the payment note image to obtain multiple text information therein, includes the following steps:
[0097] Step S1310: The text detection and recognition model performs text region recognition on the payment note image to identify one or more text regions in the payment note image.
[0098] As mentioned above, the text detection and recognition model includes a line text detection network. In a preferred embodiment, the line text detection model is a TextFuseNet model, which is responsible for detecting text regions on payment bill images. The payment bill image is pre-processed and input into a TextFuseNet model that is pre-trained to convergence. The global level features are extracted through the semantic segmentation branch, and the character level and word level features are extracted through the detection branch and the mask branch, respectively. After obtaining the features at the character level, word level, and global level, the multi-path feature fusion architecture is further called to fuse the three features to generate a more representative feature representation to detect the text region in the image to be recognized, obtain a multi-point line text box surrounding the text region, and convert the multi-point line text box into a four-point line text box. The four points of the four-point line text box are the four vertices corresponding to the rectangle. Thus, further, a rectangular line text box corresponding to the four-point box is constructed to represent the corresponding text region.
[0099] Step S1320: The text detection and recognition model cuts out text region images corresponding to the multiple text regions from the payment note image.
[0100] According to the rectangular line text box, the corresponding text area is selected from the image to be identified to capture the rectangular long strip of line text image as the text area image, and the position of the text area image in the payment note image represented by coordinates is obtained as the coordinates of the text area image.
[0101] It can be seen that the TextFuseNet model is called to detect the text area in the image to be identified. In this process, the character-level, word-level, and global-level features are integrated into the detection results, so that text detection with higher accuracy can be achieved and the robustness and reliability of text detection can be improved.
[0102] Step S1330: The text detection and recognition model performs text recognition on each of the text region images to obtain corresponding text information:
[0103] As mentioned above, the text detection and recognition model also includes a line text recognition network, which is responsible for text recognition in each text area image.
[0104] Before performing text recognition, the text area image can be preprocessed. The image preprocessing is to generate a new text area image by transforming the original text area image through data enhancement so as to expand the data source for inputting the model. The data enhancement method can include horizontal or vertical flipping, multi-angle rotation, scaling, cropping, translation, interpolation, Gaussian noise, contrast transformation, Fancy PCA and other operations, which can be flexibly selected by those skilled in the art as needed.
[0105] The text recognition network can preferably use a CRNN+CTC model, input the pre-processed text area image into a CRNN+CTC model that is pre-trained to convergence, use CNN in the convolution layer of the CRNN+CTC model to extract the image feature sequence from the text area image, input it into the circulation layer of the CRNN+CTC model, and use RNN to predict the image feature sequence to obtain its corresponding label (true value) distribution. Further, the image feature sequence is fully connected through the fully connected layer in the transcription layer of the CRNN+CTC model, and the label distribution corresponding to the image feature sequence is de-reintegrated using CTC, and the image features in the image feature sequence are correspondingly mapped to the preset classification space, and the corresponding text information is obtained according to the determined label corresponding to the image feature sequence. Accordingly, each text area can obtain its corresponding text information.
[0106] It can be seen that the CRNN+CTC model is called to recognize the text area image to obtain the text, that is, text information. In this process, robust features are extracted and the extremely difficult character segmentation in traditional algorithms is avoided through sequence recognition. At the same time, serialization recognition also embeds time dependency, so that sequences of any length can be processed, greatly improving the robustness of text recognition.
[0107] This embodiment uses a text detection and recognition model to identify the text information contained in the text area in the payment note image in steps, and obtains character data that is easier for the computer to understand, which facilitates the subsequent further identification of the category of the text information, thereby improving the recognition accuracy of the payment information contained in the payment note image. In addition, since the model can select a high-quality basic model, the robustness of the model operation can be enhanced, thereby ensuring the stable operation of the online services built on this basis.
[0108] See also Figure 4 In a further embodiment, the step S1400, using a text classification model pre-trained to a convergent state to classify each item of text information, comprises the following steps:
[0109] Step S1410: The text classification model performs word vector encoding on all the text information to obtain a corresponding word vector sequence:
[0110] In order to realize the type recognition of each text information identified from the payment bill information, the encoding information of each text information is first represented and learned using the text classification model. The text classification model can be used after being trained to a convergence state using basic models such as LSTM and Bert that are suitable for extracting deep semantic information from text.
[0111] Before learning the representation of each piece of text information, the each piece of text information is first converted into feature sequence information according to a preset dictionary, and then a fully connected linear layer is used to encode the feature sequence information into word vectors to obtain a word vector sequence. In one embodiment, the linear layer output dimension is set to 512 dimensions.
[0112] Step S1420: The text classification model extracts deep semantic information from the word vector sequence to obtain corresponding text feature information:
[0113] Taking the text classification model using LSTM as an example, the word vector sequence is input into LSTM for representation learning. Similarly, the number of hidden state features of LSTM is also set to 512 dimensions. After memory learning by LSTM, the text feature information corresponding to the word vector sequence is obtained.
[0114] Step S1430: The text classification model performs classification mapping on the text feature information to obtain classification labels corresponding to each piece of text information, wherein the classification labels include classification labels corresponding to the payment amount and the remark order number:
[0115] In the text classification model, the text feature information obtained by LSTM is input into a classifier through a fully connected linear layer for classification. The classification space of the classifier is set to multiple categories corresponding to the specific type of the payment information, that is, the fully connected linear layer is set to corresponding multiple outputs, and different categories correspond to different specific payment information, such as: amount name, payment amount, remark order number, payment time, other information, etc. Accordingly, after the text classification model classifies and maps the text feature information, the classification labels corresponding to each piece of text information can be obtained, thereby determining the payment information, especially the payment amount and remark order number therein, which can be used to execute subsequent business links.
[0116] In this embodiment, the text classification model classifies the serialized text information and distinguishes the information category corresponding to each text information, so as to determine the specific payment information for completing the business closed loop of the e-commerce order. Since the text information can be processed in batches, the category recognition efficiency can be improved, and the remark order number and payment amount can be obtained more quickly, thereby realizing the rapid processing of e-commerce orders.
[0117] See also Figure 5 In a further embodiment, the step S1500, responding to the payment confirmation instruction submitted by the anchor user, and initiating a payment reversal event, comprises the following steps:
[0118] Step S1510: Receive the payment confirmation instruction submitted by the anchor user, and obtain the remark order number in the payment information corresponding to the payment confirmation instruction:
[0119] When the anchor user triggers the payment confirmation instruction and sends it to the server of the e-commerce platform, the server can determine the remark order number in the corresponding payment information according to the payment confirmation instruction, so as to hit the order corresponding to the remark order number.
[0120] Step S1520: call the order corresponding to the remarked order number, and use the payment amount included in the payment information to offset the amount to be paid in the order:
[0121] According to the remark order number, the server calls out the corresponding order from the order queue. The order is described by the user characteristic information of the audience who placed the order, the product characteristic information, the amount to be paid for the product and other information. Based on this, the payment amount in the payment information identified from the payment note image can be used to offset the amount to be paid in the order, that is, the amount to be paid is deducted from the payment amount in the payment information to achieve the offset and obtain the difference after the offset. The difference is used to indicate whether the payment of the order has been completed.
[0122] Step S1530: determine whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the determination is true, trigger the order delivery business process. Otherwise, send a notification message to the anchor user and the audience user:
[0123] In order to control the flow of orders, the difference is further compared to see if it is less than or equal to 0. If this judgment fails, according to the default business logic of this embodiment, the order is deemed not to have been fully paid. At this time, a corresponding notification message can be sent to the anchor user and the audience user at the same time so that both parties can coordinate and improve the order payment process. If this judgment is established, it means that the order has been successfully paid, which can trigger the business process corresponding to the order shipment, and submit the order to the service interface corresponding to the logistics link to start the shipment process.
[0124] In an embodiment extended on this basis, when determining whether the difference after the amount to be paid of the order is offset is less than or equal to 0, if the determination is established, the following steps may be further performed:
[0125] Step S1540: Push a notification message to the live broadcast room indicating that the user corresponding to the order has completed payment, wherein the notification message includes a link to the ordering business process of the product corresponding to the order:
[0126] Now that it is confirmed that the order has been paid in full, a notification message can be pushed to the live broadcast room where the anchor user and the audience user are located to indicate the fact that the order has been paid in full, so that all users in the live broadcast room can receive this notification message. Usually, this notification message can be pushed directly to the public screen flow area of the live broadcast room, or other notification areas displayed in the graphical user interface of the live broadcast room. In order to facilitate user access, the notification message contains a link to the ordering business process of the product corresponding to the order. When the notification message is displayed on the graphical user interface, the text or image information displayed on the interface is associated with the link, so as to facilitate access to the link through the text or image information.
[0127] Step S1550: In response to any user in the live broadcast room triggering an access instruction of the link, an order page for the product corresponding to the order is pushed to the user, and a corresponding notification message is sent to the live broadcast user:
[0128] After the notification message is displayed in the graphical user interface of the live broadcast room, when any user touches the notification message in the graphical user interface of the live broadcast room, he will jump to the order page pointed to by the link associated with the notification message through the link. The order page displays the shopping business process of the product pointed to by the order, so that the user can quickly place an order on the shopping page of the product, realize further quick recommendation of the product, and help improve the execution efficiency of users in the live broadcast room placing orders for the product.
[0129] In this embodiment, by providing more abundant follow-up steps after the order is completed, the functionality of the present application is expanded so that the order delivery business process and the recommendation events of the products corresponding to the order can be seamlessly connected, thereby helping to further improve the execution efficiency of the e-commerce platform's orders and achieve a complete business closed loop.
[0130] See also Figure 6An order payment processing device provided to meet one of the purposes of the present application is a functional embodiment of the order payment processing method of the present application. The device includes: an image submission module 1100, an image classification module 1200, a text recognition module 1300, a text classification module 1400, and a payment offset module 1500. Among them, the image submission module 1100 is used to obtain the image to be identified submitted by the audience user in the live broadcast room; the image classification module 1200 is used to classify and judge the image to be identified by using a bill judgment model pre-trained to a convergent state, so as to determine whether the image to be identified is a payment bill image; the text recognition module 1300 is used to use a text detection and recognition model pre-trained to a convergent state to perform text recognition on the payment bill image to obtain multiple text information therein; the text classification module 1400 is used to use a text classification model pre-trained to a convergent state to classify each item of text information, so as to classify and identify payment information from the text information, the payment information includes the payment amount and the remark order number, and push the payment information to the terminal device of the anchor user in the live broadcast room for display; the payment offset module 1500 is used to respond to the payment confirmation instruction submitted by the anchor user, initiate a payment offset event, and offset the amount to be paid of the order corresponding to the remark order number with the payment amount in the payment information.
[0131] In an extended embodiment, the order payment processing device of the present application includes a training unit for executing a training process for the bill judgment model, and the training unit includes: a sample calling module for calling a training sample in a first training data set, wherein the training sample includes a positive sample and a negative sample, wherein the positive sample is a payment bill image, and the negative sample is a non-payment bill image; a semantic extraction module, configured to use the image feature extraction network in the bill judgment model to extract deep semantic information from the training sample to obtain image feature information; a classification mapping module, configured to use the binary classifier in the bill judgment model to perform classification mapping on the image feature information to obtain a corresponding classification label; an iterative decision module, configured to calculate the loss value of the classification label according to whether the training sample belongs to a positive sample or a negative sample, and when the loss value has not converged, to correct the weight parameters of the bill judgment model according to the loss value and continue iterative training, and to terminate the training when the loss value reaches a convergence state.
[0132] In a further embodiment, the text recognition module 1300 includes: a region recognition submodule, configured to perform text region recognition on the payment bill image by the text detection and recognition model to identify one or more text regions in the payment bill image; a region cropping submodule, configured to crop text region images corresponding to the multiple text regions from the payment bill image by the text detection and recognition model; and a text recognition submodule, configured to perform text recognition on each of the text region images by the text detection and recognition model to obtain corresponding text information.
[0133] In a further embodiment, the text classification module 1400 includes: a vector encoding submodule, configured to use the text classification model to perform word vector encoding on all items of text information to obtain a corresponding word vector sequence; a feature extraction submodule, configured to use the text classification model to extract deep semantic information from the word vector sequence to obtain corresponding text feature information; a label determination submodule, configured to use the text classification model to perform classification mapping on the text feature information to obtain classification labels corresponding to each item of text information, wherein the classification labels include classification labels corresponding to the payment amount and the remark order number.
[0134] In a further embodiment, the payment offset module 1500 includes: an instruction response submodule, used to receive the payment confirmation instruction submitted by the anchor user, and obtain the remark order number in the payment information corresponding to the payment confirmation instruction; an order calling submodule, used to call the order corresponding to the remark order number, and offset the amount to be paid in the order with the payment amount contained in the payment information; a flow control submodule, used to determine whether the difference after the amount to be paid of the order is offset is less than or equal to 0, if the judgment is established, the order delivery business process is triggered, otherwise, a notification message is sent to the anchor user and the audience user.
[0135] In an extended embodiment, the flow control submodule determines whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the judgment is established, it also includes running the following functions: pushing a notification message to the live broadcast room indicating that the user corresponding to the order has completed the payment, and the notification message includes a link to the order business process of the product corresponding to the order; in response to any user in the live broadcast room triggering an access instruction of the link, pushing the order page of the product corresponding to the order to the user, and sending a corresponding notification message to the live broadcast user.
[0136] In order to solve the above technical problems, the present application also provides a computer device. Figure 7As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement an order payment processing method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the order payment processing method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0137] In this embodiment, the processor is used to execute Figure 6 The memory stores the program codes and various data required to execute the above-mentioned units or sub-units. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program codes and data required to execute all units / sub-units in the order payment processing device of this application, and the server can call the program codes and data of the server to execute the functions of all sub-units.
[0138] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the order payment processing method of any embodiment of the present application.
[0139] The present application also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any embodiment of the present application when executed by one or more processors.
[0140] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0141] To sum up, the present application can ensure the recognition and processing of payment bill images uploaded by audience users in the e-commerce live broadcast scenario, ensure recognition efficiency and accuracy, thereby effectively avoiding manual verification of payment bill images and excessive manual operations on orders, improving the automation efficiency of e-commerce orders, and ultimately realizing the business closed loop of e-commerce orders.
[0142] Those skilled in the art will appreciate that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, altered, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted.
[0143] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. An order payment processing method, characterized in that: The steps include: Obtain the image to be identified submitted by the audience in the live broadcast room; Using a bill decision model that has been pre-trained to a convergent state to classify and decide the image to be identified, so as to determine whether the image to be identified is a payment bill image; Using a text detection and recognition model that has been pre-trained to a convergent state to perform text recognition on the payment note image to obtain multiple pieces of text information therein; Using a text classification model that has been pre-trained to a convergent state to classify the text information of each item, so as to classify and identify payment information from the text information, wherein the payment information includes a payment amount and a remark order number, and the payment information is pushed to a terminal device of the anchor user in the live broadcast room for display; In response to the payment confirmation instruction submitted by the anchor user, a payment offset event is initiated, and the payment amount in the payment information is used to offset the amount to be paid of the order corresponding to the remark order number.
2. The order payment processing method according to claim 1, characterized in that: The training process of the bill judgment model includes the following steps: Calling a training sample in the first training data set, the training sample includes a positive sample and a negative sample, the positive sample is a payment bill image, and the negative sample is a non-payment bill image; The image feature extraction network in the bill judgment model is used to extract deep semantic information from the training sample to obtain image feature information; Using the binary classifier in the bill judgment model to classify and map the image feature information to obtain corresponding classification labels; The loss value of the classification label is calculated according to whether the training sample is a positive sample or a negative sample. When the loss value has not converged, the weight parameters of the bill decision model are corrected according to the loss value and the iterative training is continued. The training is terminated when the loss value reaches a convergence state.
3. The order payment processing method according to claim 1, characterized in that: Using a text detection and recognition model that has been pre-trained to a convergent state to perform text recognition on the payment note image to obtain multiple text information therein, the method includes the following steps: The text detection and recognition model performs text region recognition on the payment note image to identify one or more text regions in the payment note image; The text detection and recognition model cuts out text area images corresponding to the multiple text areas from the payment note image; The text detection and recognition model performs text recognition on each of the text area images to obtain corresponding text information.
4. The order payment processing method according to claim 1, characterized in that: Using a text classification model that has been pre-trained to a convergent state to classify each item of text information, the steps include: The text classification model performs word vector encoding on all text information to obtain a corresponding word vector sequence; The text classification model extracts deep semantic information from the word vector sequence to obtain corresponding text feature information; The text classification model performs classification mapping on the text feature information to obtain classification labels corresponding to each piece of text information, wherein the classification labels include classification labels corresponding to the payment amount and the remark order number.
5. The order payment processing method according to any one of claims 1 to 4, characterized in that: In response to the payment confirmation instruction submitted by the anchor user, a payment reversal event is initiated, including the following steps: Receive the payment confirmation instruction submitted by the anchor user, and obtain the remark order number in the payment information corresponding to the payment confirmation instruction; Call the order corresponding to the remarked order number, and offset the amount to be paid in the order with the payment amount included in the payment information; Determine whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the judgment is true, trigger the order delivery business process, otherwise, send a notification message to the anchor user and the audience user.
6. The order payment processing method according to claim 5, characterized in that: Determine whether the difference after the amount to be paid of the order is offset is less than or equal to 0. If the determination is established, the following steps are included: Pushing a notification message to the live broadcast room indicating that the user corresponding to the order has completed payment, wherein the notification message includes a link to the ordering business process of the product corresponding to the order; In response to any user in the live broadcast room triggering an access instruction of the link, an order page for the product corresponding to the order is pushed to the user, and a corresponding notification message is sent to the live broadcast user.
7. An order payment processing device, characterized in that: include: An image submission module is used to obtain images to be identified submitted by viewers in the live broadcast room; An image classification module, used for classifying and judging the image to be identified by using a bill judgment model pre-trained to a convergent state, so as to determine whether the image to be identified is a payment bill image; A text recognition module, used to perform text recognition on the payment note image using a text detection and recognition model that has been pre-trained to a convergence state, so as to obtain multiple pieces of text information therein; A text classification module, used to classify the text information of each item using a text classification model pre-trained to a convergence state, so as to classify and identify payment information from the text information, wherein the payment information includes a payment amount and a remark order number, and push the payment information to a terminal device of the anchor user in the live broadcast room for display; The payment reversal module is used to respond to the payment confirmation instruction submitted by the anchor user, initiate a payment reversal event, and use the payment amount in the payment information to offset the amount to be paid of the order corresponding to the remark order number.
8. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 6 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.
