Information processing method and device, equipment, storage medium and product

Abstract

The invention discloses an information processing method and device, equipment, a storage medium and a product, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a target image containing an information code and a background image of the information code; the background image is an image of an environment where the information code is located; performing information extraction on a background image in the target image from the N information extraction dimensions to obtain image information of the background image under each information extraction dimension in the N information extraction dimensions; performing information fusion processing on the image information of the background image under each information extraction dimension to generate fused image information of the background image; and performing anomaly detection on the environment where the information code is located based on the fused image information to obtain a detection result used for indicating whether the environment where the information code is located is abnormal or normal. By adopting the method and the device, the environment where the information code is located can be subjected to anomaly detection, so that the safety of the environment where the information code is located is ensured, and the anomaly detection accuracy of the information code is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an information processing method, device, equipment, storage medium and product. Background Art

[0002] With the widespread application of information codes in various scenarios, many abnormal objects use information codes to engage in abnormal business, increasing the risk of using information codes.

[0003] At present, when determining whether an information code is abnormal, the business characteristics of the information code are generally used to determine whether the information code is abnormal. Therefore, at present, the information code is generally determined based on business characteristics such as whether the information code provider has been complained, whether the information code provider has joined an abnormal organization, and whether the business execution time is an abnormal time period. However, this method of determining the abnormality of the information code is relatively simple and cannot accurately determine whether the information code is abnormal. Therefore, how to improve the accuracy of abnormality detection of information codes is an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide an information processing method, apparatus, device and storage medium, which can perform anomaly detection on the environment in which the information code is located to ensure the security of the environment in which the information code is located, thereby improving the accuracy of anomaly detection on the information code.

[0005] In a first aspect, the present application provides an information processing method, comprising:

[0006] Acquire a target image to be processed; the target image includes an information code and a background image of the information code, and the background image is an image of the environment in which the information code is located;

[0007] Extract information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions; N is a positive integer;

[0008] Performing information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image;

[0009] Based on the fused image information, an abnormality detection is performed on the environment where the information code is located to obtain a detection result of the environment where the information code is located; the detection result is used to indicate whether the environment where the information code is located is abnormal or normal.

[0010] In a second aspect, the present application provides an information processing device, including:

[0011] An image acquisition unit is used to acquire a target image to be processed; the target image includes an information code and a background image of the information code, and the background image is an image of the environment where the information code is located;

[0012] An information extraction unit, configured to extract information of the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions; N is a positive integer;

[0013] An information fusion unit, used for performing information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image;

[0014] The abnormality detection unit is used to perform abnormality detection on the environment where the information code is located based on the fused image information to obtain a detection result of the environment where the information code is located; the detection result is used to indicate whether the environment where the information code is located is abnormal or normal.

[0015] Optionally, the N information extraction dimensions include an object extraction dimension and a text extraction dimension; the information extraction unit is specifically used to:

[0016] The image at the position where the information code is located in the target image is removed to obtain the background image;

[0017] Performing object detection on the background image from the object extraction dimension to obtain a detection frame of the object contained in the background image; the detection frame belongs to the image information of the background image at the object extraction dimension; and,

[0018] Text extraction is performed on the background image from the text extraction dimension to obtain text information contained in the background image; the text information belongs to the image information of the background image under the text extraction dimension.

[0019] Optionally, the information fusion unit is specifically used to:

[0020] Extracting the object image in the detection frame from the background image, and calling the detection model to embed the object image to generate an object embedding feature of the object image;

[0021] Calling the detection model to perform embedding processing on the text information to generate text embedding features of the text information;

[0022] Performing feature fusion processing on the object embedding feature and the text embedding feature to generate the fused image information;

[0023] The fusion processing of the object embedding feature and the text embedding feature is any one of the following: splicing processing and addition processing.

[0024] Optionally, the anomaly detection unit is specifically used to:

[0025] The detection model is called to perform abnormality detection on the environment where the information code is located based on the fused image information, and a detection probability of the environment where the information code is located is obtained; the detection probability is the probability that the environment where the information code is located is abnormal;

[0026] If the detection probability is greater than or equal to the probability threshold, determining that the detection result is a result indicating that the environment in which the information code is located is abnormal;

[0027] If the detection probability is less than the probability threshold, it is determined that the detection result is a result indicating that the environment in which the information code is located is normal.

[0028] Optionally, the information processing device further includes: a model training unit, the model training unit being configured to:

[0029] Acquire a detection model to be trained and a sample image; the sample image includes a sample information code and a sample background image of the sample information code, the sample background image is an image of the environment where the sample information code is located, and the sample image has a sample label, and the sample label is used to indicate whether the environment where the sample information code is located is actually normal or abnormal;

[0030] Extract information from the sample background image in the sample image from the N information extraction dimensions to obtain sample image information of the sample background image in each information extraction dimension in the N information extraction dimensions;

[0031] Calling the detection model to be trained to perform information fusion processing on the sample image information of the sample background image under the various information extraction dimensions to generate sample fused image information of the sample background image;

[0032] Calling the detection model to be trained to perform abnormality detection on the environment where the sample information code is located based on the sample fusion image information, and obtaining a sample detection result of the environment where the sample information code is located; the sample detection result is used to indicate whether the detected environment where the sample information code is located is abnormal or normal;

[0033] Based on the difference between the sample detection result and the sample label, the model parameters of the detection model to be trained are corrected to obtain the detection model.

[0034] Optionally, there are multiple sample images, and the multiple sample images include a positive sample image and a negative sample image. The sample label of the positive sample image is used to indicate that the environment in which the sample information code included in the positive sample image is actually normal, and the sample label of the negative sample image is used to indicate that the environment in which the sample information code included in the negative sample image is actually abnormal.

[0035] The ratio between the number of the negative sample images and the number of the positive sample images satisfies a target ratio, and the target ratio is used to make the number of the negative sample images and the number of the positive sample images in a balanced state.

[0036] Optionally, the information code includes information for executing a target service; the information processing device further includes: a service processing unit, the service processing unit being configured to terminate the execution of the target service if the detection result indicates that the environment in which the information code is located is abnormal;

[0037] The target business is any one of the following: resource transfer business, relationship building business, and information query business.

[0038] Optionally, the target object is used to initiate the target service based on the information code; the service processing unit is further used to:

[0039] If the detection result indicates that the environment in which the information code is located is abnormal, a prompt message indicating that the execution environment of the target service is abnormal is sent to the target terminal of the target object, so that the target terminal outputs the prompt message and the execution inquiry information for the target service in association;

[0040] When receiving the confirmation information sent by the target terminal, executing the target service; the confirmation information is generated by the target terminal after detecting that the output execution inquiry information is confirmed;

[0041] When the termination information sent by the object terminal is received, the execution of the target service is terminated; the termination information is generated by the object terminal after detecting that the output execution inquiry information is terminated.

[0042] In a third aspect, the present application provides a computer device, comprising a processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer program executes the above-mentioned information processing method.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the above-mentioned information processing method.

[0044] In a fifth aspect, the present application provides a computer program product, which includes a computer program / instructions, and the computer program / instructions can implement the above-mentioned information processing method when executed by a processor.

[0045] In the embodiment of the present application, by obtaining a target image containing an information code and a background image of the information code; the background image is an image of the environment in which the information code is located, the information code can be judged in combination with the image of the environment in which the information code is located to determine whether the environment in which the information code is located is safe. By extracting information from the background image in the target image from N information extraction dimensions, the image information of the background image in each information extraction dimension in the N information extraction dimensions is obtained, and information extraction from the background image can be realized from multiple dimensions, and image information of more dimensions is obtained, thereby increasing the accuracy of subsequent abnormality detection. Since the image information extracted from multiple information extraction dimensions is subjected to information fusion processing to obtain fused image information, the fused image information may include more image information, and then when the environment in which the information code is located is detected for abnormality based on the fused image information, that is, the environment in which the information code is located is detected for abnormality from multiple dimensions, the accuracy of abnormality detection for the information code can be improved. Furthermore, since the environment in which the abnormal information code is located usually has specific environmental characteristics, the present application can realize abnormality detection of the information code from the environment in which the information code is located, so as to ensure the security of the environment in which the information code is located, thereby improving the accuracy of abnormality detection of the information code. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a schematic diagram of a network architecture of an information processing system provided in an embodiment of the present application;

[0048] Figure 2 It is a schematic diagram of an application scenario of an information processing method provided in an embodiment of the present application;

[0049] Figure 3 It is a flowchart of an information processing method provided by an embodiment of the present application;

[0050] Figure 4 is a schematic diagram of a scenario for performing anomaly detection based on a detection model provided in an embodiment of the present application;

[0051] Figure 5 It is a flowchart of another information processing method provided by an embodiment of the present application;

[0052] Figure 6 It is a schematic diagram of the structure of an information processing device provided in an embodiment of the present application;

[0053] Figure 7 It is a schematic diagram of the composition structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0056] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The solution provided in the embodiment of this application belongs to computer vision technology and machine learning technology under the field of artificial intelligence.

[0057] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, follow and measure targets, and further process images so that the computer processes them into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0058] For example, in the present application, image processing technology in computer vision technology can be used to perform object detection on the background image from the object extraction dimension to obtain the detection frame of the object contained in the background image, and OCR technology in computer vision technology can be used to perform text extraction on the background image from the text extraction dimension to obtain the text information contained in the background image, and so on.

[0059] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0060] For example, in the present application, machine learning technology can be used to extract information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions; perform information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image; and perform anomaly detection on the environment in which the information code is located based on the fused image information to obtain detection results of the environment in which the information code is located, and so on.

[0061] Intelligent Vehicle Infrastructure Cooperative Systems (IVICS), referred to as IVICS, is a development direction of Intelligent Transportation Systems (ITS). The IVICS adopts advanced wireless communications and new generation Internet technologies to implement all-round dynamic real-time information interaction between vehicles and roads, and carries out active vehicle safety control and road cooperative management based on the collection and integration of dynamic traffic information in all time and space, fully realizing the effective coordination of people, vehicles and roads, ensuring traffic safety, and improving traffic efficiency, thus forming a safe, efficient and environmentally friendly road traffic system. For example, the information code in this application may refer to the information code displayed in the intelligent vehicle-road cooperative system, or the information code pasted or hung on the vehicle-mounted equipment, and so on.

[0062] The technical solution of the present application can be applied to the scenario of detecting anomalies in the environment where the information code is located and determining whether the environment where the information code is located is abnormal. Optionally, the technical solution of the present application can also be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0063] See also Figure 1 , Figure 1 is a schematic diagram of a network architecture of an information processing system provided in an embodiment of the present application, such as Figure 1 As shown, the computer device can exchange data with the terminal device, and the number of the terminal devices can be one or at least two. For example, when the number of the terminal devices is multiple, the terminal device may include Figure 1 The terminal device 101a, the terminal device 101b and the terminal device 101c in the computer device 102 are as follows. Taking the terminal device 101a as an example, the computer device 102 can obtain the target image to be processed. Further, the background image in the target image is extracted from multiple information extraction dimensions to obtain the image information of the background image in each of the multiple information extraction dimensions. Further, the computer device 102 can also perform information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image, perform abnormality detection on the environment where the information code is located based on the fused image information, and obtain a detection result indicating whether the environment where the information code is located is abnormal or normal. Optionally, the computer device 102 can send the detection result of the environment where the information code is located to the terminal device 101a, so that the detection result of the environment where the information code is located can be displayed on the terminal device 101a, thereby prompting the target object that initiates the target business based on the information code, and so on.

[0064] It is understandable that the computer devices mentioned in the embodiments of the present application include but are not limited to terminal devices or servers. In other words, the computer device can be a server or a terminal device, or it can be a system composed of a server and a terminal device. Among them, the terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablet computers, desktop computers, laptop computers, PDAs, vehicle-mounted devices, intelligent voice interaction devices, augmented reality / virtual reality (AR / VR) devices, helmet displays, wearable devices, smart speakers, smart home appliances, aircraft, digital cameras, cameras and other mobile Internet devices (mobile internet device, MID) with network access capabilities. Among them, the server mentioned above can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road collaboration, content distribution networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0065] It should be noted that the collection and processing of relevant data (such as target images, sample images, etc.) in the embodiments of this application should be strictly in accordance with the requirements of relevant local laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and the subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and personal information subjects. For example, the object can refer to the user of a terminal device or a computer device.

[0066] For further information, see Figure 2 , Figure 2 is a schematic diagram of an application scenario of an information processing method provided in an embodiment of the present application; Figure 2As shown, the terminal device 21 can scan the background where the information code is located to obtain the target image 22, and upload the target image 22 to the computer device 23. The target image 22 includes the information code 221 and the background image 222 of the information code. The background image 222 is an image of the environment where the information code 221 is located. Further, the computer device 23 can extract information from the background image 222 in the target image 22 from multiple information dimensions to obtain image information of the background image 222 under each information extraction dimension. For example, the multiple information dimensions may include two information dimensions, namely, the object extraction dimension and the text extraction dimension, and then the image information 24 under the object extraction dimension and the image information 25 under the text extraction dimension can be obtained. Furthermore, the computer device 23 can perform information fusion processing on the image information 24 under the object extraction dimension and the image information 25 under the text extraction dimension, generate fused image information 26 of the background image 222, call the detection model 27 to perform abnormality detection on the environment where the information code 221 is located based on the fused image information 26, and obtain the detection result 28 of the environment where the information code 221 is located; for example, the detection result 28 is used to indicate that the environment where the information code 221 is located is normal.

[0067] For further information, see Figure 3 , Figure 3 is a flowchart of an information processing method provided by an embodiment of the present application; Figure 3 As shown, the information processing method can be applied to a computer device, and the computer device can be, for example, a server. The information processing method includes but is not limited to the following steps:

[0068] S101, obtaining a target image to be processed.

[0069] At present, the security issues of information codes are generally determined by obtaining business characteristics of information codes, such as whether the information code provider has been complained, whether the information code provider has joined an illegal organization, and whether the execution time of the business executed based on the information code is an abnormal time period, so as to determine whether the information code is abnormal, thereby determining the security issues of the information code. For example, to determine whether the information code provider is an abnormal object, the information code provider can refer to the object that provides the information code, for example, it can refer to a normal merchant that uses the information code to execute a business, or it can refer to an abnormal object that uses the information code to execute an abnormal business.

[0070] However, when judging the security of information codes, the business characteristics of the information codes are generally used, and the environmental information of the payment codes is not judged. Therefore, it is impossible to accurately determine whether the information codes are used to perform abnormal business. In addition, the current information codes have a problem of insufficient timeliness when performing abnormal identification. It is necessary to perform a certain number of businesses based on the information codes before judging whether the information codes are abnormal.

[0071] Therefore, the technical solution of the present application provides a method for obtaining the environmental information of the information code to perform anomaly detection, determine whether the environment in which the information code is located is abnormal, and thus determine whether the object providing the information code is an abnormal object, and then determine whether there is a security problem with the information code. The technical solution of the present application can be executed in the intermediate process of executing a business based on the information code, so that it can be determined whether to continue to execute the business based on the detection result of whether the environment in which the information code is located is abnormal. For example, if the detection result indicates that the environment in which the information code is located is abnormal, the business executed based on the information code can be intercepted to reduce property losses. Or if the detection result indicates that the environment in which the information code is located is normal, the business executed based on the information code can continue to be executed without affecting the execution of normal business. Since the technical solution of the present application can perform anomaly detection on the environment in which the information code is located, and determine whether the environment in which the information code is located is risky based on the detection result, it does not need to rely on business characteristics, that is, it does not need to rely on obtaining business characteristics after executing the business, so it can improve coverage and has high timeliness.

[0072] For example, when the business is a transaction business, the target object, such as a consumer, initiates a transaction by scanning an information code. By integrating the anomaly detection process in the technical solution of the present application after scanning the information code and before completing the transaction, for example, when it is determined that the environment in which the information code is located is abnormal, it means that the object providing the information code is an abnormal object. Then, the transaction initiated by the target object can be intercepted to reduce transaction losses.

[0073] In an embodiment of the present application, a computer device may obtain a target image to be processed. The target image may include an information code and a background image of the information code. The information code may include, for example, but is not limited to, a payment code, a reward code, a QR code, a barcode, a mini-program code, and the like. The background image of the information code is an image of the environment in which the information code is located. The environment in which the information code is located may refer to the environment of the actual place where the information code is located, or it may be the environment of the interface content on the application interface. For example, the environment of the actual place where the information code is located may refer to the environment in which a paper information code is pasted in an offline business location. The environment of the interface content on the application interface may refer to the environment in which the information code is located as presented on the display screen of the terminal device where the application is installed.

[0074] Specifically, the background image of the information code may refer to an image composed of all elements in the environment where the information code is located, such as an image composed of objects, text, and people in the environment where the information code is located. For example, in an offline consumption payment scenario, the background image of the information code may refer to an image composed of goods at the location where the information code is placed, such as a background image that refers to an image composed of goods behind the location where the information code is posted. Goods may include but are not limited to tables, chairs, cups, counters, doors, etc. Or in an online payment scenario, the background image of the information code may also refer to an image composed of other information contained in the display screen that displays the information code, such as an image composed of abnormal images and abnormal text. Abnormal images may refer to images that do not comply with regulations or are guiding, and abnormal text may refer to guiding text or misleading text, etc.

[0075] After acquiring the target image to be processed, the computer device can further perform image recognition on the background image in the target image to determine the objects contained in the background image, and thus determine the usage scenario of the information code in the target image in combination with the objects contained in the background image, for example, the usage scenario of the information code includes the information code provided by the object in the normal business place, or the information code provided by the object of the abnormal object to perform abnormal business. When the normal object performs business based on the information code, the environment in which the information code is located generally includes objects such as tables, chairs, cups, sofas, counters, doors, etc., and when the abnormal object performs abnormal business based on the information code, the environment in which the information code is located generally includes abnormal images, terminal devices, display screens, and guiding or misleading images, guiding texts or misleading texts on the display screen, etc. Therefore, it is possible to detect whether the environment in which the information code is located is abnormal in combination with the environmental information in the environment in which the information code is located.

[0076] In an optional implementation, the terminal device may scan the environment where the information code is located to obtain a target image. Alternatively, the terminal device may also take a photo of the environment where the information code is located to obtain a target image, or may obtain the target image in other ways. When the terminal device obtains the target image, the target image may be uploaded to the computer device, so that the computer device may perform subsequent abnormality detection based on the target image.

[0077] In the embodiment of the present application, the application scenario of the present application is illustrated by taking the information code as an example of a payment code. For example, when the target object, such as the consumer object, uses a terminal device to scan the payment code (the payment code is provided by the payment code provider) to scan the code for payment, the terminal device can scan the payment code and the environment where the payment code is located in the link of scanning the payment code for payment, so as to obtain a target image containing the payment code and the background image of the payment code, and the terminal device can further upload the target image to the computer device for processing. For example, the computer device can perform image recognition on the target image to determine the items contained in the background image in the target image. And perform OCR recognition on the background image to determine the text information contained in the background image. Then, anomaly detection can be performed based on the items and text information to determine whether the environment where the payment code is located is abnormal. If the environment where the payment code is located is abnormal, the operation of scanning the payment code based on the payment code by the consumer object can be intercepted, and the transaction of the payment code provider object, that is, the abnormal object, can be intervened to ensure the property safety of the consumer object and reduce the frequency of illegal payment using the payment code by the abnormal object.

[0078] In the embodiment of the present application, since the target image of the image to be processed contains the information code and the background image of the information code, by performing abnormality detection on the background image of the information code, it can be determined whether the environment in which the information code is located is abnormal or normal, thereby determining whether the object providing the information code is abnormal, and then determining the management method for the information code or the object providing the information code.

[0079] S102, extracting information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions.

[0080] In the embodiment of the present application, since the target image is acquired, information extraction can be performed on the target image to obtain image information, and then abnormality detection can be performed on the extracted image information to determine whether the environment in which the information code is located is abnormal. When performing information extraction, information extraction can be performed on the background image in the target image from multiple information extraction dimensions, so as to perform abnormality detection based on the image information under multiple information extraction dimensions, thereby improving the accuracy of abnormality detection.

[0081] In one implementation, the computer device may extract information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions, where N is a positive integer. The N information extraction dimensions may be, for example, multiple different dimensions. By extracting information from the background image in the target image from multiple different dimensions, since different dimensions have different concerns about the same background image, anomaly detection may be performed in combination with image information extracted from multiple dimensions, thereby improving the accuracy of anomaly detection.

[0082] In one embodiment, the N information extraction dimensions include an object extraction dimension and a text extraction dimension. The object extraction dimension can be used to extract object information in the background image, and the text extraction dimension can be used to extract text information in the background image. By extracting information from the background image from the object extraction dimension and the text extraction dimension, when performing anomaly detection later, anomaly detection can be performed on the background image from different dimensions, thereby improving the accuracy of anomaly detection.

[0083] For example, the image information in each of the N information extraction dimensions may be obtained in the following manner:

[0084] The image at the position where the information code is located in the target image is removed to obtain a background image; object detection is performed on the background image from the object extraction dimension to obtain a detection frame of the object contained in the background image; the detection frame belongs to the image information of the background image under the object extraction dimension; and text extraction is performed on the background image from the text extraction dimension to obtain text information contained in the background image; the text information belongs to the image information of the background image under the text extraction dimension.

[0085] Among them, the removal process can be used to remove the image at the location of the information code in the target image, that is, to remove the information code from the target image. Removal can, for example, refer to removing the image at the location of the information code in the target image by image cropping or the like. By performing the removal process on the information code in the target image, a background image is obtained. When information is subsequently extracted from the background image, only the background image needs to be extracted without extracting information from the information code, which can improve the efficiency of information extraction.

[0086] Furthermore, by performing object detection on the background image from the object extraction dimension, the method of obtaining the detection frame of the object contained in the background image can be specifically: performing object detection on the background image to obtain the detection frame of the object contained in the background image, the detection frame is used to frame the object in the background image, and the detection frame may include but is not limited to the position of the object in the background image and the category of the object. Among them, the category of the object includes but is not limited to tables, chairs, water cups, sofas, counters, doors, etc. Alternatively, the category of the object may also include abnormal images, terminal devices or display screens, etc. By performing object detection on the objects in the background image, the detection frame of the object contained in the background image can be obtained, so that the detection frame can be identified and the category of the object in the detection frame can be determined. For example, when identifying the object in the detection frame, an image recognition model can be used for identification to determine the category of the object in the detection frame, thereby obtaining the category of the object contained in the background image.

[0087] Optionally, the image recognition model may include but is not limited to a convolutional neural network model (CNN), a recurrent neural network model (RNN), a deep neural network model (DNN) or other types of neural network models, etc.

[0088] In the embodiment of the present application, by performing object detection on the background image from the object extraction dimension, the detection frame of the object contained in the background image can be detected, and then combined with the detection frame of the object contained in the background image to determine whether the environment where the information code is located is abnormal. Since the abnormal object generally uses the information code in an abnormal application, when the information code is displayed in the abnormal application, the environment where the information code is located will also include abnormal images. Therefore, by extracting the image information in the environment where the information code is located, the environment where the information code is located can be combined with the object extraction dimension to perform abnormal detection on the environment where the information code is located, thereby improving the accuracy of abnormal detection.

[0089] Furthermore, the text information may refer to the text in the background image, for example. In an optional implementation, the text information in the background image may be identified by OCR recognition. When the text information in the background image is identified, the background image may be subjected to text extraction from the text extraction dimension to obtain the text information contained in the background image. Subsequently, the text information and the object information in the detection frame may be combined to determine whether the environment in which the information code is located is abnormal, thereby improving the accuracy of abnormality detection. Among them, OCR recognition refers to the process of analyzing and identifying images containing text content to obtain text and layout information in the image.

[0090] Further optionally, the N information extraction dimensions may also include a color extraction dimension, and the background image may be color extracted from the color extraction dimension to obtain color information contained in the background image, and the color information belongs to the image information of the background image under the color extraction dimension. The color information may include, for example, the color of the object image in the background image, the color other than the object image in the background image, or the color of the information code, etc., and the embodiments of the present application do not limit this.

[0091] In the embodiment of the present application, by extracting the colors in the background image, the environment in which the information code is located can be further combined with the types of colors in the background image to perform anomaly detection on the environment in which the information code is located, to determine whether the environment in which the information code is located is abnormal, and to extract richer image information for anomaly detection. Since information codes are generally used on counters in business premises, the background image of the information code may include colors corresponding to a variety of commodities. Abnormal objects generally use information codes in abnormal applications, so the colors in the background image in which the information code is located are relatively single. Therefore, by combining the color extraction dimension in the background image for anomaly detection, the accuracy of anomaly detection can be improved.

[0092] In an optional implementation, the N information extraction dimensions may also include other dimensions besides the object extraction dimension, the text extraction dimension, and the color extraction dimension, such as the object quantity extraction dimension, which is not limited in the embodiment of the present application.

[0093] S103, performing information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image.

[0094] In an embodiment of the present application, since the computer device extracts information from the background image in the target image from N information extraction dimensions and obtains image information of the background image in each of the N information extraction dimensions, anomaly detection can be performed in combination with the image information of the background image in each information extraction dimension to determine whether the environment in which the information code is located is abnormal.

[0095] For example, the computer device may perform abnormality detection on the image information of the background image in each information extraction dimension, determine whether the image information in each information extraction dimension is abnormal, and then determine whether the environment where the information code is located is abnormal based on the results of whether the image information in each information extraction dimension is abnormal. Alternatively, the computer device may also perform information fusion processing on one or more image information in the image information of the background image in each information extraction dimension, generate fused image information of the background image, and then perform abnormality detection on the one or more image information after fusion, obtain abnormality detection results corresponding to one or more image information, and then determine whether the environment where the information code is located is abnormal based on the abnormality detection results corresponding to one or more image information.

[0096] In one embodiment, the computer device may perform information fusion processing on the image information of the background image under each information extraction dimension to generate fused image information of the background image. By performing information fusion processing on the image information under each information extraction dimension to generate fused image information of the background image, anomaly detection may be performed on the fused image information subsequently to determine whether the environment in which the information code is located is abnormal. Since the image information under each information extraction dimension is fused when performing anomaly detection, the accuracy of the anomaly detection result may be improved.

[0097] In an optional implementation, for example, the method of generating fused image information of a background image and performing anomaly detection may be as follows: extracting an object image in a detection frame from the background image, and calling a detection model to perform embedding processing on the object image to generate an object embedding feature of the object image; calling a detection model to perform embedding processing on text information to generate a text embedding feature of the text information; performing feature fusion processing on the object embedding feature and the text embedding feature to generate fused image information. The fusion processing of the object embedding feature and the text embedding feature is any of the following: splicing processing, summing processing.

[0098] The object image may refer to a local image of the object in the background image. Since the detection frame of the object contained in the background image is determined by object detection on the background image, and the object image is contained in the detection frame, when generating the fused image information of the background image, the object image in the detection frame may be embedded to generate the object embedding feature of the object image. For example, the object image in the detection frame may be cut out from the background image, and the cut out object image may be embedded. Since only the cut out object image needs to be processed, there is no need to embed the image other than the object image in the background image, which can improve the image processing efficiency.

[0099] Alternatively, the computer device may also directly embed the item image in the detection frame contained in the background image. Since the detection frame only contains the item image, the item image is embedded to obtain the item embedding feature, which is convenient for subsequent fusion processing with the text embedding feature to improve the accuracy of anomaly detection. It is understandable that in the embodiment of the present application, the item image in the background image may be processed in other ways, as long as the item embedding feature can be obtained, and the embodiment of the present application does not limit this.

[0100] Furthermore, embedding an object image may refer to encoding the object image into a feature vector, and an object embedding feature refers to a feature vector obtained by feature encoding the object image. Embedding text information may refer to encoding text information such as text into a feature vector, and a text embedding feature refers to a feature vector obtained by feature encoding the text information.

[0101] Specifically, embedding the text information may refer to mapping the text information to a numerical vector space to obtain text embedding features (i.e., feature vectors). For example, a certain text in the text space may be mapped or embedded into another numerical vector space to obtain a feature vector corresponding to the text. By converting the object image and text information into embedding features (i.e., feature vectors), it is convenient for the two to perform feature fusion processing later. Among them, the splicing process may refer to splicing the object embedding features and the text embedding features together, for example, splicing the object embedding features in front and the text embedding features in the back. Alternatively, the text embedding features may be spliced ​​in front and the object embedding features in the back, and so on. The embodiments of the present application do not limit the specific splicing process. The summation process may refer to adding the object embedding features and the text embedding features bit by bit.

[0102] In an optional implementation, if the fusion process refers to a splicing process, the object embedding feature and the text embedding feature may also be fused in combination with their weights. For example, a first weight corresponding to the object embedding feature and a second weight corresponding to the text embedding feature may be obtained respectively; the object embedding feature and the text embedding feature may be fused based on the first weight and the second weight to generate fused image information.

[0103] Optionally, for example, in some scenarios, such as scenarios where transactions are conducted based on information codes, if the background image includes an item image and text information, the image in the background image can better reflect the abnormality in the background image than the text, so the first weight of the image embedding feature can be greater than the second weight of the text embedding feature. Or in other scenarios, the first weight may also be less than the second weight, which is not limited in the embodiments of the present application. Alternatively, if one of the item embedding feature or the text embedding feature is empty, the weight corresponding to the embedding feature can be set to 0 or a value close to 0.

[0104] In a possible implementation, if the N information extraction dimensions also include color extraction dimensions and other extraction dimensions, the weights corresponding to the respective information extraction dimensions can be obtained respectively, so as to fuse the embedded features extracted from the respective information extraction dimensions based on the weights corresponding to the respective information extraction dimensions to generate fused image information. By splicing the embedded features extracted from the respective information extraction dimensions in combination with the weights corresponding to the respective information extraction dimensions, the detection model can pay more attention to the embedded features with higher weights when performing anomaly detection on the environment where the information code is located based on the fused image information, thereby improving the accuracy of anomaly detection.

[0105] Optionally, the detection model may refer to, for example, a gradient boosting model (eXtreme Gradient Boosting, XGBoost), a deep neural network model (Deep Neural Networks, DNN), a residual neural network model (RESNET), a convolutional neural network model (Convolutional Neural Network, CNN), a single-shot multi-box detection model (SingleShot MultiBox Detector, SSD), and the like.

[0106] In one possible implementation, if the background image does not include one of the object image or text information, for example, the background image does not include text information, then the text information obtained by performing text extraction on the background image from the text extraction dimension is empty. During the subsequent feature fusion processing, if the text embedding feature is empty, the object embedding feature and the text embedding feature are subjected to feature fusion processing, and the generated fused image information is the object embedding feature. Alternatively, when it is determined that the background image does not include text information, the object image in the background image may be directly embedded to generate the object embedding feature of the object image, and the object embedding feature may be determined as the fused image information. Alternatively, when it is determined that the background image does not include text information, the object embedding feature may be directly subjected to anomaly detection after the object embedding feature of the object image is generated, thereby improving the efficiency of anomaly detection.

[0107] S104, performing abnormality detection on the environment where the information code is located based on the fused image information to obtain a detection result of the environment where the information code is located.

[0108] In the embodiment of the present application, since information extraction is performed on the background image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions, and fused image information is generated by performing information fusion processing on the image information in the N information extraction dimensions, anomaly detection can be performed on the environment in which the information code is located based on the fused image information to obtain a detection result of the environment in which the information code is located.

[0109] The detection result is used to indicate whether the environment in which the information code is located is abnormal or normal. The abnormal environment in which the information code is located may mean that the information code is provided to an abnormal object, and the abnormal object uses the information code in a scenario of performing abnormal business to obtain benefits. The normal environment in which the information code is located may mean that the information code is provided to a normal object, for example, it may refer to a scenario in which a merchant in a normal business place uses the information code to perform business.

[0110] In the embodiment of the present application, performing anomaly detection on the environment where the information code is located based on the fused image information may refer to performing anomaly detection on the fused image information, and determining whether the environment where the information code is located is abnormal based on the detection result. For example, a detection model may be called to perform anomaly detection on the environment where the information code is located based on the fused image information. The main content of the anomaly detection performed by the detection model is to perform risk assessment on the objects and texts in the current background image of the information code, and obtain a classification result for indicating whether the environment where the information code is located is abnormal.

[0111] Specifically, the detection model can be called to perform anomaly detection on the environment where the information code is located based on the fused image information, and obtain the detection probability of the environment where the information code is located; the detection probability is the probability that the environment where the information code is located is abnormal; if the detection probability is greater than or equal to the probability threshold, then the detection result is determined to be a result indicating that the environment where the information code is located is abnormal; if the detection probability is less than the probability threshold, then the detection result is determined to be a result indicating that the environment where the information code is located is normal.

[0112] Among them, the detection probability of the environment where the information code is located can be determined by the detection model, so that whether the environment where the information code is located is abnormal can be determined based on the detection probability. For example, if the detection probability is greater than or equal to the probability threshold, it means that the environment where the information code is located is abnormal, and if the detection probability is less than the probability threshold, it means that the environment where the information code is located is normal.

[0113] In the embodiment of the present application, by calling the detection model to perform anomaly detection on the fused image information, the detection probability of the environment in which the information code is located can be obtained, thereby determining whether the environment in which the information code is located is abnormal. Since the image information extracted from multiple information extraction dimensions is combined, and the image information extracted from multiple information extraction dimensions is fused to obtain the fused image information, by performing anomaly detection on the fused image information, the background image can be detected from multiple dimensions, which can improve the accuracy of detecting whether the environment in which the information code is located is abnormal.

[0114] See also Figure 4 , Figure 4It is a schematic diagram of a scene for abnormality detection based on a detection model provided by an embodiment of the present application, wherein, by identifying the detection frame of the object contained in the background image, the object 41 contained in the background image can be determined, for example, the object 41 contained in the background image can include one or more of a table, a water cup, a chair, a terminal device, and a display screen. By embedding the object images corresponding to the various objects contained in the background image (i.e., feature encoding), the object embedding features (i.e., feature vectors) of the object images are generated. Furthermore, by embedding the text information 42, the text embedding features of the text information are generated (i.e., the text information is feature encoded to obtain feature vectors), and by performing feature fusion processing such as vector splicing on the object embedding features and the text embedding features, the fused image information 43 is obtained, and then the detection model 44 can be called to perform abnormality detection on the fused image information 43, so as to determine whether there is a risk in the information code, that is, whether the environment in which the information code is located is abnormal.

[0115] In one implementation, if the N information extraction dimensions include an object extraction dimension, a text extraction dimension, and a color extraction dimension, object detection can be performed on the background image from the object extraction dimension to obtain a detection frame of the object contained in the background image. Text extraction can be performed on the background image from the text extraction dimension to obtain text information contained in the background image. Color extraction can be performed on the background image from the color extraction dimension to obtain color information contained in the background image, and the color information belongs to the image information of the background image under the color extraction dimension.

[0116] Further optionally, the detection model may be called to embed the object image to generate the object embedding feature of the object image. The detection model may be called to embed the text information to generate the text embedding feature of the text information. The detection model may be called to embed the color information to generate the color embedding feature of the color information. The object embedding feature, the text embedding feature and the color embedding feature may be fused to generate fused image information, and then the detection model may be called to perform abnormality detection on the environment where the information code is located based on the fused image information to obtain the detection probability of the environment where the information code is located.

[0117] In the embodiment of the present application, since the color information in the background image is further obtained, the anomaly detection of the environment in which the information code is located is performed by combining multiple dimensions of color information, object image and text information, which can improve the accuracy of anomaly detection.

[0118] In the embodiment of the present application, before calling the detection model to perform anomaly detection on the environment where the information code is located based on the fused image information and obtaining the detection probability of the environment where the information code is located, the detection model and training samples to be trained can be obtained in advance, and the training samples can be used to train the detection model to be trained, so that the detection model obtained after training has the ability to perform anomaly detection on the environment where the information code is located based on the fused image information and obtain the detection probability of the environment where the information code is located, thereby realizing anomaly detection. The specific method of training the detection model to be trained can be referred to Figure 5 The method of step S201 is not described in detail here.

[0119] The detection model in the embodiment of the present application is mainly used to perform binary classification on the environment where the information code is located, and determine whether the environment where the information code is located is abnormal or normal. In the optional implementation mode, the environment where the information code is located can also be multi-classified to determine whether the environment where the information code is located is abnormal or normal, and when it is determined that the environment where the information code is located is abnormal, determine which specific abnormality the environment where the information code is located is. For example, the abnormality type may include but is not limited to the first type of abnormality and the second type of abnormality, and the first type of abnormality and the second type of abnormality are two different types of abnormalities. If it is necessary to perform multi-classification on the environment where the information code is located, sample images corresponding to multiple abnormal categories can be selected when preparing training samples, so that the detection model obtained by training has the ability to perform multi-classification on the background image. By adopting a multi-classification method, it can be determined whether the environment where the information code is located is abnormal and what specific abnormality it is, and the abnormality type of the information code providing object can be determined, so that the information code providing object can be quickly targeted.

[0120] In one embodiment, the information code includes information for executing the target business, and the target business can be any of the following: resource transfer business, relationship establishment business, and information query business. Resource transfer business can refer to a transaction business, such as a business of scanning a code to pay for electronic resources. Relationship establishment business can refer to a business of establishing a relationship between two objects by scanning an information code, such as a business of scanning a code to add a friend. Information query business can refer to a business of scanning a code to view information details, and so on. Further optionally, if the detection result is used to indicate that the environment in which the information code is located is abnormal, the execution of the target business can be terminated.

[0121] Since the environment in which the information code is located is abnormal, by terminating the execution of the target business, property losses can be reduced and information security can be improved. For example, if the target business is a resource transfer business and the environment in which the information code is located is abnormal, by terminating the execution of the resource transfer business, transaction risks can be reduced and property losses can be avoided. Or if the target business is a relationship establishment business and the environment in which the information code is located is abnormal, by terminating the execution of the relationship establishment business, relationships with abnormal objects can be avoided and losses can be reduced. For example, if the target business is an information query business and the environment in which the information code is located is abnormal, by terminating the execution of the information query business, the disclosure of one's own identity information can be avoided and information security can be improved.

[0122] The technical solution of the present application is applied in the detection scenario of whether the environment in which the information code is located is abnormal. Through experiments, the accuracy of the offline evaluation results of the detection model is obtained to be more than 99%, and the information code can be identified in time when it is used for the first time to perform abnormal business, so that the object providing the information code can be quickly punished. In an optional implementation, in the feature extraction process, the accuracy of anomaly detection can be improved by further extracting color information in the background image and performing subsequent anomaly detection. The detection model in the embodiment of the present application can select any one neural network model, or select multiple neural network models to perform multi-model fusion to obtain a detection model, so as to use the model obtained by multi-model fusion to perform anomaly detection and obtain anomaly detection results.

[0123] In the embodiment of the present application, by obtaining a target image containing an information code and a background image of the information code; the background image is an image of the environment in which the information code is located, the information code can be judged in combination with the image of the environment in which the information code is located to determine whether the environment in which the information code is located is safe. By extracting information from the background image in the target image from N information extraction dimensions, the image information of the background image in each information extraction dimension in the N information extraction dimensions is obtained, and information extraction from the background image can be realized from multiple dimensions, and image information of more dimensions is obtained, thereby increasing the accuracy of subsequent abnormality detection. Since the image information extracted from multiple information extraction dimensions is subjected to information fusion processing to obtain fused image information, the fused image information may include more image information, and then when the environment in which the information code is located is detected for abnormality based on the fused image information, that is, the environment in which the information code is located is detected for abnormality from multiple dimensions, the accuracy of abnormality detection for the information code can be improved. Furthermore, since the environment in which the abnormal information code is located usually has specific environmental characteristics, the present application can realize abnormality detection of the information code from the environment in which the information code is located, so as to ensure the security of the environment in which the information code is located, thereby improving the accuracy of abnormality detection of the information code.

[0124] For further information, see Figure 5 , Figure 5 is a flow chart of another information processing method provided by an embodiment of the present application. The information processing method can be applied to an information processing system and executed by a server and a client in the information processing system; Figure 5 As shown, the information processing method includes but is not limited to the following steps:

[0125] S201, the server obtains a detection model to be trained and a sample image, and trains the detection model to be trained based on the sample image to obtain a detection model.

[0126] In the embodiment of the present application, since the server needs to use a detection model to perform abnormal detection on the environment where the information code is located to obtain the detection result of the environment where the information code is located, a training sample and a detection model to be trained can be obtained first, and the detection model to be trained can be trained using the training sample, so that the detection model can be trained. The training sample can include multiple sample images, and the processing method for each sample image can be the same.

[0127] The sample image includes a sample information code and a sample background image of the sample information code. The sample background image is an image of the environment where the sample information code is located. The sample image has a sample label, and the sample label is used to indicate whether the environment where the sample information code is located is actually normal or abnormal. The sample label is used to indicate that the environment where the sample information code is located is actually abnormal, which can indicate that the sample image corresponding to the sample label is an image containing the information code provided by the abnormal object, and the abnormal object can obtain benefits by providing the sample image.

[0128] In an embodiment of the present application, the sample background image of the sample information code is an image of the environment in which the sample information code is located, for example, it may refer to an image composed of all elements in the environment in which the sample information code is located, for example, it may include an image composed of objects, text, and people in the environment in which the sample information code is located. The sample background image of the sample information code may, for example, refer to an image composed of goods at the location where the sample information code is placed, for example, the sample background image includes an image composed of goods after the location where the information code is posted, and the goods may include, but are not limited to, tables, chairs, cups, counters, doors, etc. Or the sample background image of the sample information code may also refer to an image composed of other information contained in the display screen that displays the sample information code, for example, the sample image may include, but is not limited to, an image composed of abnormal images or abnormal text, etc.

[0129] In an optional implementation, there are multiple sample images, and the multiple sample images include positive sample images and negative sample images. The sample label of the positive sample image is used to indicate that the environment in which the sample information code included in the positive sample image is actually normal. For example, the positive sample image may refer to an information code image provided in a normal business place, such as an image containing an information code pasted in a normal business place. The sample label of the negative sample image is used to indicate that the environment in which the sample information code included in the negative sample image is actually abnormal. For example, the negative sample image may refer to an image containing an information code provided by an abnormal object for performing abnormal business.

[0130] Furthermore, the ratio between the number of negative sample images and the number of positive sample images can meet the target ratio, and the target ratio is used to make the number of negative sample images and the number of positive sample images in a balanced state. For example, the target ratio can be 1:1, 2:1, 3:1, etc., which can be set according to specific needs. For example, when it is expected that the detection model can detect more abnormal results of the environment in which the information code is located, the number of negative sample images can be set to be greater than the number of positive sample images, so that the detection model can learn more information about the negative sample images during model training, thereby improving the detection accuracy of the abnormal environment in which the information code is located.

[0131] When training the detection model to be trained, the sample images can be extracted by random sampling without replacement, so that the detection model can learn richer information. In practical applications, the difficulty of obtaining white sample images is much lower than that of obtaining black sample images, so the number of white sample images is generally much larger than the number of black sample images. Therefore, in order to improve the robustness of the model, the white sample images can be resampled each time the training is performed. That is, each time the detection model is trained, a part of the white sample images and black sample images are extracted for training. After the training is completed, the black sample images used in the last model training can be extracted, and the white sample images can be re-extracted for model training. That is, the white sample images used for each model training can be different, thereby improving the robustness of the model.

[0132] In one embodiment, the detection model to be trained is trained based on the sample image, and the process of obtaining the detection model can be as follows:

[0133] Information is extracted from the sample background image in the sample image from N information extraction dimensions to obtain sample image information of the sample background image in each of the N information extraction dimensions; the detection model to be trained is called to perform information fusion processing on the sample image information of the sample background image in each information extraction dimension to generate sample fusion image information of the sample background image; the detection model to be trained is called to perform anomaly detection on the environment where the sample information code is located based on the sample fusion image information to obtain a sample detection result of the environment where the sample information code is located; the sample detection result is used to indicate whether the environment where the sample information code is located is abnormal or normal; based on the difference between the sample detection result and the sample label, the model parameters of the detection model to be trained are corrected to obtain the detection model.

[0134] Among them, the N information extraction dimensions may include, for example, an object extraction dimension, a text extraction dimension, and other extraction dimensions. By extracting features from the sample background image from multiple dimensions, different sample image information can be extracted from each dimension, so that when the extracted sample image information is used for anomaly detection, the accuracy of anomaly detection can be improved. The sample detection result of the environment where the sample information code is located may refer to the model prediction value, and the sample label may refer to the sample true value. The purpose of training the detection model is to make the model prediction value and the sample true value as consistent as possible. When the model prediction value is inconsistent with the sample true value, the model parameters in the detection model to be trained are continued to be corrected, and the loss function in the detection model to be trained is reduced, so that the model prediction value and the sample true value are as consistent as possible. When the model prediction value is consistent with the sample true value, the model parameters of the detection model to be trained can be stopped to obtain the detection model, and the detection model obtained by the training can be called later to perform anomaly detection on the environment where the information code is located to obtain the detection result of the environment where the information code is located.

[0135] Optionally, the process of training the detection model to be trained is the process of continuously adjusting the model parameters in the detection model to be trained. By continuously adjusting the model parameters in the detection model to be trained, the loss function in the detection model to be trained can be reduced. When the loss function in the detection model to be trained is less than the loss threshold, or when the detection model to be trained reaches the convergence condition, or when the number of iterative training of the detection model to be trained reaches the target number, then the model parameters of the detection model to be trained can be stopped to obtain the detection model.

[0136] In one embodiment, image information extracted from multiple information extraction dimensions can be extracted in the following manner. For example, the image at the location of the sample information code in the sample image can be removed to obtain a sample background image; item detection can be performed on the sample background image from the item extraction dimension to obtain a sample detection frame of the item contained in the sample background image; the sample detection frame belongs to the image information of the sample background image under the item extraction dimension; and text extraction can be performed on the sample background image from the text extraction dimension to obtain text information contained in the sample background image; the text information belongs to the image information of the sample background image under the text extraction dimension. By performing feature extraction on the sample background image from the item extraction dimension and the text extraction dimension, information in the sample background image can be extracted from multiple dimensions, and the accuracy of anomaly detection can be improved when performing anomaly detection later.

[0137] In the embodiment of the present application, by extracting information from the sample background image from the object extraction dimension and the text extraction dimension respectively, more complete image information can be extracted, thereby improving the accuracy of subsequent anomaly detection.

[0138] In one embodiment, the detection model to be trained is called to perform information fusion processing on the sample image information of the sample background image in each information extraction dimension, and the method of generating sample fusion image information of the sample background image can be as follows:

[0139] The object image in the sample detection frame is cut out from the sample background image, and the detection model is called to perform embedding processing on the sample object image to generate a sample object embedding feature of the sample object image; the detection model is called to perform embedding processing on the sample text information to generate a sample text embedding feature of the sample text information; the sample object embedding feature and the sample text embedding feature are subjected to feature fusion processing to generate sample fusion image information; wherein the fusion processing of the sample object embedding feature and the sample text embedding feature is any one of the following: splicing processing, summing processing.

[0140] It is understandable that in order to ensure that the model detection results are more accurate, the method of using the detection model for fusion processing is the same as the method of using the fusion processing for training the detection model to be trained. For example, when training the detection model to be trained, the method of fusion processing for the sample item embedding features and the sample text embedding features is splicing processing, then when using the detection model, the method of fusion processing for the item embedding features and the text embedding features is splicing processing. For example, when training the detection model to be trained, the method of fusion processing for the sample item embedding features and the sample text embedding features is summing processing, then when using the detection model, the method of fusion processing for the item embedding features and the text embedding features is summing processing.

[0141] In one embodiment, the detection model to be trained is called to perform abnormality detection on the environment where the sample information code is located based on the sample fusion image information, and the method of obtaining the sample detection result of the environment where the sample information code is located can be as follows:

[0142] The detection model is called to perform abnormality detection on the environment where the sample information code is located based on the sample fusion image information, and the sample detection probability of the environment where the sample information code is located is obtained; the sample detection probability is the probability that the environment where the sample information code is located is abnormal; if the sample detection probability is greater than or equal to the probability threshold, the sample detection result is determined to be a result indicating that the environment where the sample information code is located is abnormal; if the sample detection probability is less than the probability threshold, the sample detection result is determined to be a result indicating that the environment where the sample information code is located is normal. By obtaining the sample detection probability of the environment where the sample information code is located, the sample detection result can be obtained, and then it can be determined whether the environment where the sample information code is located is normal or abnormal.

[0143] In an embodiment of the present application, by acquiring the detection model to be trained and the sample images, the detection model to be trained can be trained, and finally the detection model can be obtained. Then, the detection model can be used to perform abnormal detection on the environment where the information code is located, and the detection result of the environment where the information code is located can be obtained.

[0144] S202, the client scans the information code to obtain the target image.

[0145] In the embodiment of the present application, the client may refer to a target application installed on a terminal device, and the target application may be provided with a function for scanning an information code, for example, it may refer to a scan function in the target application, and the target application may be run on the terminal device, thereby activating the scanning function in the target application to scan the information code. Since when scanning an information code, not only the information code is scanned, but also the environmental information where the information code is located is scanned, the target image obtained by scanning includes the information code and the background image of the information code.

[0146] S203: The client reports the target image to the server.

[0147] In the embodiment of the present application, after the client scans and obtains the target image, it can report the target image to the server for subsequent processing.

[0148] S204, the server extracts information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions.

[0149] Optionally, after acquiring the target image, the server may use the environment recognition system to perform image recognition on the background image to determine the object image contained in the background image; and use the environment recognition system to perform OCR recognition on the background image to determine the text information contained in the background image.

[0150] Furthermore, the server may use the risk identification system to extract information from the background image in the target image, and determine whether the environment in which the information code is located is abnormal based on the extracted image information. The risk identification system may include a detection model, which may be used to perform abnormality detection on the background image of the information code to determine whether the information code is risky, that is, whether the business executed based on the information code is risky.

[0151] It is understandable that the server can also identify the information code and determine the link associated with the information code, thereby determining whether the information code is an information code provided by the development object of the target application based on the link associated with the information code. The scanning and identification results of the information code will not affect the subsequent results of detecting whether the environment in which the information code is located is abnormal.

[0152] S205, the server performs information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image.

[0153] S206, the server calls the detection model to perform anomaly detection on the environment where the information code is located based on the fused image information, and obtains the detection result of the environment where the information code is located.

[0154] In the embodiment of the present application, the specific implementation method of steps S201 to S206 can refer to the implementation method of the above-mentioned steps S101 to S104, which will not be repeated here.

[0155] S207: If the detection result indicates that the environment in which the information code is located is abnormal, the server intercepts the transaction executed based on the information code.

[0156] In the embodiment of the present application, the abnormal environment of the information code may refer to the information code providing object being an abnormal object, and the abnormal object using the information code to perform abnormal business. For example, the information code contains information for performing resource transfer business such as transaction business, and the detection result is used to indicate that the environment of the information code is abnormal. In order to reduce property losses and reduce complaints against the target application, the server can intercept and process transactions based on the information code. Avoid the negative impact on the target application caused by a large number of abnormal objects using the information code for transactions.

[0157] S208: If the detection result indicates that the environment in which the information code is located is normal, the server continues to execute the transaction based on the information code.

[0158] In the embodiment of the present application, the normal environment of the information code may refer to the object providing the information code being a normal object, such as an object in a normal business place using the information code to conduct a transaction, and receiving the electronic resources transferred by the transaction initiator based on the information code. Since the environment of the information code is normal, the server continues to execute the transaction based on the information code, which can ensure the normal execution of the transaction.

[0159] In one embodiment, the target object is used to initiate a target service based on the information code; when the detection result is used to indicate that the environment in which the information code is located is abnormal, the processing method for the target service can also be determined based on the operation of the target object.

[0160] For example, if the detection result is used to indicate that the environment in which the information code is located is abnormal, a prompt message indicating that the execution environment of the target business is abnormal is sent to the object terminal of the target object, so that the object terminal associates the output prompt message and the execution inquiry information for the target business; when the confirmation message sent by the object terminal is received, the target business is executed; the confirmation message is generated by the object terminal after detecting that the output execution inquiry information is subjected to the confirmation operation; when the termination message sent by the object terminal is received, the execution of the target business is terminated; the termination message is generated by the object terminal after detecting that the output execution inquiry information is subjected to the termination operation.

[0161] Among them, the target object may refer to the object that scans the information code. For example, when the target business is a resource transfer business, the target object may refer to the object that requests the transfer of electronic resources, that is, the transaction initiator, and the information code providing object may refer to the object that receives the electronic resources, that is, the transaction recipient. The object terminal may refer to the terminal device used by the target object, for example, it may refer to the terminal device used to scan the information code. When an abnormality is detected in the environment where the information code is located, a prompt message indicating that the execution environment of the target business is abnormal may be sent to the object terminal, and the object terminal may associate the output prompt message and the execution inquiry information for the target business. The prompt message is used to prompt that the execution environment of the target business is abnormal. The execution inquiry information is used to inquire the target object whether to continue to execute the target business. When the target object sees the prompt message and the execution inquiry information on the object terminal, it can trigger confirmation information or termination information according to needs. The confirmation message is used to indicate the continuation of the execution of the target business, and the termination message is used to indicate the termination of the execution of the target business.

[0162] In the embodiment of the present application, when it is detected that the environment in which the information code is located is abnormal, information security can be improved by prompting the target object.

[0163] In one possible implementation, the information code provider can also file a complaint based on the detection results of the environment in which the information code is located. The management object can then manually detect the environment in which the information code is located to determine whether the environment in which the information code is located is abnormal, thereby reducing the probability of misjudgment.

[0164] In the embodiment of the present application, by performing abnormal detection on the environment where the information code is located, the information code illegally used by the abnormal object can be identified and managed. The embodiment of the present application has high accuracy, so the impact on the normal business execution is low, and the execution of abnormal business can be accurately intercepted. Since each business executed by the information code is processed, the coverage rate can be improved. In addition, the embodiment of the present application has high timeliness, and the environment where the information code is located can be found to be abnormal in the pre-stage link of executing the business, so as to determine that the business execution is at risk and terminate it.

[0165] In the embodiment of the present application, by obtaining a target image containing an information code and a background image of the information code; the background image is an image of the environment in which the information code is located, the information code can be judged in combination with the image of the environment in which the information code is located to determine whether the environment in which the information code is located is safe. By extracting information from the background image in the target image from N information extraction dimensions, the image information of the background image in each information extraction dimension in the N information extraction dimensions is obtained, and information extraction from the background image can be realized from multiple dimensions, and image information of more dimensions is obtained, thereby increasing the accuracy of subsequent abnormality detection. Since the image information extracted from multiple information extraction dimensions is subjected to information fusion processing to obtain fused image information, the fused image information may include more image information, and then when the environment in which the information code is located is detected for abnormality based on the fused image information, that is, the environment in which the information code is located is detected for abnormality from multiple dimensions, the accuracy of abnormality detection for the information code can be improved. Furthermore, since the environment in which the abnormal information code is located usually has specific environmental characteristics, the present application can realize abnormality detection of the information code from the environment in which the information code is located, so as to ensure the security of the environment in which the information code is located, thereby improving the accuracy of abnormality detection of the information code. In addition, since the detection model required for sample image training is acquired in advance before the detection model is used to perform anomaly detection on the environment where the information code is located based on the fused image information, the model prediction accuracy can be improved when the detection model is subsequently used to perform anomaly detection on the environment where the information code is located based on the fused image information.

[0166] The method of the embodiment of the present application is introduced above, and the device of the embodiment of the present application is introduced below.

[0167] See also Figure 6 , Figure 6 : is a schematic diagram of the composition structure of an information processing device provided in an embodiment of the present application. The above-mentioned information processing device can be deployed on a computer device, and the computer device can be, for example, a server; the information processing device can be used to execute the corresponding steps in the information processing method provided in the embodiment of the present application. The information processing device 60 includes:

[0168] The image acquisition unit 601 is used to acquire a target image to be processed; the target image includes an information code and a background image of the information code, and the background image is an image of the environment where the information code is located;

[0169] An information extraction unit 602 is used to extract information of the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions; N is a positive integer;

[0170] An information fusion unit 603 is used to perform information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image;

[0171] The abnormality detection unit 604 is used to perform abnormality detection on the environment where the information code is located based on the fused image information to obtain a detection result of the environment where the information code is located; the detection result is used to indicate whether the environment where the information code is located is abnormal or normal.

[0172] Optionally, the N information extraction dimensions include an object extraction dimension and a text extraction dimension; the information extraction unit 602 is specifically used to:

[0173] The image at the position where the information code is located in the target image is removed to obtain the background image;

[0174] Performing object detection on the background image from the object extraction dimension to obtain a detection frame of the object contained in the background image; the detection frame belongs to the image information of the background image at the object extraction dimension; and,

[0175] Text extraction is performed on the background image from the text extraction dimension to obtain text information contained in the background image; the text information belongs to the image information of the background image under the text extraction dimension.

[0176] Optionally, the information fusion unit 603 is specifically configured to:

[0177] Extracting the object image in the detection frame from the background image, and calling the detection model to embed the object image to generate an object embedding feature of the object image;

[0178] Calling the detection model to perform embedding processing on the text information to generate text embedding features of the text information;

[0179] Performing feature fusion processing on the object embedding feature and the text embedding feature to generate the fused image information;

[0180] The fusion processing of the object embedding feature and the text embedding feature is any one of the following: splicing processing and addition processing.

[0181] Optionally, the anomaly detection unit 604 is specifically configured to:

[0182] The detection model is called to perform abnormality detection on the environment where the information code is located based on the fused image information, and a detection probability of the environment where the information code is located is obtained; the detection probability is the probability that the environment where the information code is located is abnormal;

[0183] If the detection probability is greater than or equal to the probability threshold, determining that the detection result is a result indicating that the environment in which the information code is located is abnormal;

[0184] If the detection probability is less than the probability threshold, it is determined that the detection result is a result indicating that the environment in which the information code is located is normal.

[0185] Optionally, the information processing device 60 further includes: a model training unit 605, wherein the model training unit 605 is configured to:

[0186] Acquire a detection model to be trained and a sample image; the sample image includes a sample information code and a sample background image of the sample information code, the sample background image is an image of the environment where the sample information code is located, and the sample image has a sample label, and the sample label is used to indicate whether the environment where the sample information code is located is actually normal or abnormal;

[0187] Extract information from the sample background image in the sample image from the N information extraction dimensions to obtain sample image information of the sample background image in each information extraction dimension in the N information extraction dimensions;

[0188] Calling the detection model to be trained to perform information fusion processing on the sample image information of the sample background image under the various information extraction dimensions to generate sample fused image information of the sample background image;

[0189] Calling the detection model to be trained to perform abnormality detection on the environment where the sample information code is located based on the sample fusion image information, and obtaining a sample detection result of the environment where the sample information code is located; the sample detection result is used to indicate whether the detected environment where the sample information code is located is abnormal or normal;

[0190] Based on the difference between the sample detection result and the sample label, the model parameters of the detection model to be trained are corrected to obtain the detection model.

[0191] Optionally, there are multiple sample images, and the multiple sample images include a positive sample image and a negative sample image. The sample label of the positive sample image is used to indicate that the environment in which the sample information code included in the positive sample image is actually normal, and the sample label of the negative sample image is used to indicate that the environment in which the sample information code included in the negative sample image is actually abnormal.

[0192] The ratio between the number of the negative sample images and the number of the positive sample images satisfies a target ratio, and the target ratio is used to make the number of the negative sample images and the number of the positive sample images in a balanced state.

[0193] Optionally, the information code includes information for executing a target service; the information processing device 60 further includes: a service processing unit 606, the service processing unit 606 being configured to terminate execution of the target service if the detection result indicates that the environment in which the information code is located is abnormal;

[0194] The target business is any one of the following: resource transfer business, relationship building business, and information query business.

[0195] Optionally, the target object is used to initiate the target service based on the information code; the service processing unit 606 is further used to:

[0196] If the detection result indicates that the environment in which the information code is located is abnormal, a prompt message indicating that the execution environment of the target service is abnormal is sent to the target terminal of the target object, so that the target terminal outputs the prompt message and the execution inquiry information for the target service in association;

[0197] When receiving the confirmation information sent by the target terminal, executing the target service; the confirmation information is generated by the target terminal after detecting that the output execution inquiry information is confirmed;

[0198] When the termination information sent by the object terminal is received, the execution of the target service is terminated; the termination information is generated by the object terminal after detecting that the output execution inquiry information is terminated.

[0199] It should be noted that Figure 6 For the contents not mentioned in the corresponding embodiments, please refer to the description of the method embodiments, which will not be repeated here.

[0200] In the embodiment of the present application, by obtaining a target image containing an information code and a background image of the information code; the background image is an image of the environment in which the information code is located, the information code can be judged in combination with the image of the environment in which the information code is located to determine whether the environment in which the information code is located is safe. By extracting information from the background image in the target image from N information extraction dimensions, the image information of the background image in each information extraction dimension in the N information extraction dimensions is obtained, and information extraction from the background image can be realized from multiple dimensions, and image information of more dimensions is obtained, thereby increasing the accuracy of subsequent abnormality detection. Since the image information extracted from multiple information extraction dimensions is subjected to information fusion processing to obtain fused image information, the fused image information may include more image information, and then when the environment in which the information code is located is detected for abnormality based on the fused image information, that is, the environment in which the information code is located is detected for abnormality from multiple dimensions, the accuracy of abnormality detection for the information code can be improved. Furthermore, since the environment in which the abnormal information code is located usually has specific environmental characteristics, the present application can realize abnormality detection of the information code from the environment in which the information code is located, so as to ensure the security of the environment in which the information code is located, thereby improving the accuracy of abnormality detection of the information code. In addition, since the detection model required for sample image training is acquired in advance before the detection model is used to perform anomaly detection on the environment where the information code is located based on the fused image information, the model prediction accuracy can be improved when the detection model is subsequently used to perform anomaly detection on the environment where the information code is located based on the fused image information.

[0201] See also Figure 7 , Figure 7 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 7 As shown, the computer device 70 may include: a processor 701 and a memory 702. The processor 701 is connected to the memory 702, for example, the processor 701 may be connected to the memory 702 via a bus. Optionally, the computer device 70 may further include: a network interface 703, wherein the network interface 703 is connected to the processor 701 and the memory 702, for example, the processor 701 may be connected to the memory 702 and the network interface 703 via a bus. The computer device may be a terminal device or a server.

[0202] The processor 701 is configured to support the information processing device to perform the corresponding functions in the above-mentioned information processing method. The processor 701 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above-mentioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0203] The memory 702 is used to store program codes, etc. The memory 702 may include a volatile memory (VM), such as a random access memory (RAM); the memory 702 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); the memory 702 may also include a combination of the above-mentioned types of memory.

[0204] The network interface 703 is used to provide network communication function.

[0205] The processor 701 may call the program code to perform the following operations:

[0206] Acquire a target image to be processed; the target image includes an information code and a background image of the information code, and the background image is an image of the environment in which the information code is located;

[0207] Extract information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions; N is a positive integer;

[0208] Performing information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image;

[0209] Based on the fused image information, an abnormality detection is performed on the environment where the information code is located to obtain a detection result of the environment where the information code is located; the detection result is used to indicate whether the environment where the information code is located is abnormal or normal.

[0210] Optionally, the N information extraction dimensions include an object extraction dimension and a text extraction dimension; the processor 701 is specifically configured to:

[0211] The image at the position where the information code is located in the target image is removed to obtain the background image;

[0212] Performing object detection on the background image from the object extraction dimension to obtain a detection frame of the object contained in the background image; the detection frame belongs to the image information of the background image at the object extraction dimension; and,

[0213] Text extraction is performed on the background image from the text extraction dimension to obtain text information contained in the background image; the text information belongs to the image information of the background image under the text extraction dimension.

[0214] Optionally, the processor 701 is specifically configured to:

[0215] Extracting the object image in the detection frame from the background image, and calling the detection model to embed the object image to generate an object embedding feature of the object image;

[0216] Calling the detection model to perform embedding processing on the text information to generate text embedding features of the text information;

[0217] Performing feature fusion processing on the object embedding feature and the text embedding feature to generate the fused image information;

[0218] The fusion processing of the object embedding feature and the text embedding feature is any one of the following: splicing processing and addition processing.

[0219] Optionally, the processor 701 is specifically configured to:

[0220] The detection model is called to perform abnormality detection on the environment where the information code is located based on the fused image information, and a detection probability of the environment where the information code is located is obtained; the detection probability is the probability that the environment where the information code is located is abnormal;

[0221] If the detection probability is greater than or equal to the probability threshold, determining that the detection result is a result indicating that the environment in which the information code is located is abnormal;

[0222] If the detection probability is less than the probability threshold, it is determined that the detection result is a result indicating that the environment in which the information code is located is normal.

[0223] Optionally, the processor 701 is further configured to:

[0224] Acquire a detection model to be trained and a sample image; the sample image includes a sample information code and a sample background image of the sample information code, the sample background image is an image of the environment where the sample information code is located, and the sample image has a sample label, and the sample label is used to indicate whether the environment where the sample information code is located is actually normal or abnormal;

[0225] Extract information from the sample background image in the sample image from the N information extraction dimensions to obtain sample image information of the sample background image in each information extraction dimension in the N information extraction dimensions;

[0226] Calling the detection model to be trained to perform information fusion processing on the sample image information of the sample background image under the various information extraction dimensions to generate sample fused image information of the sample background image;

[0227] Calling the detection model to be trained to perform abnormality detection on the environment where the sample information code is located based on the sample fusion image information, and obtaining a sample detection result of the environment where the sample information code is located; the sample detection result is used to indicate whether the detected environment where the sample information code is located is abnormal or normal;

[0228] Based on the difference between the sample detection result and the sample label, the model parameters of the detection model to be trained are corrected to obtain the detection model.

[0229] Optionally, there are multiple sample images, and the multiple sample images include a positive sample image and a negative sample image. The sample label of the positive sample image is used to indicate that the environment in which the sample information code included in the positive sample image is actually normal, and the sample label of the negative sample image is used to indicate that the environment in which the sample information code included in the negative sample image is actually abnormal.

[0230] The ratio between the number of the negative sample images and the number of the positive sample images satisfies a target ratio, and the target ratio is used to make the number of the negative sample images and the number of the positive sample images in a balanced state.

[0231] Optionally, the information code includes information for executing a target service; the processor 701 is further configured to terminate the execution of the target service if the detection result indicates that the environment in which the information code is located is abnormal;

[0232] The target business is any one of the following: resource transfer business, relationship building business, and information query business.

[0233] Optionally, the target object is used to initiate the target service based on the information code; the processor 701 is further used to:

[0234] If the detection result indicates that the environment in which the information code is located is abnormal, a prompt message indicating that the execution environment of the target service is abnormal is sent to the target terminal of the target object, so that the target terminal outputs the prompt message and the execution inquiry information for the target service in association;

[0235] When receiving the confirmation information sent by the target terminal, executing the target service; the confirmation information is generated by the target terminal after detecting that the output execution inquiry information is confirmed;

[0236] When the termination information sent by the object terminal is received, the execution of the target service is terminated; the termination information is generated by the object terminal after detecting that the output execution inquiry information is terminated.

[0237] It should be understood that the computer device 70 described in the embodiment of the present application can execute the above Figure 3 and Figure 5 The description of the above information processing method in the corresponding embodiment can also be performed as described above. Figure 6 The description of the above-mentioned information processing device in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here.

[0238] In the embodiment of the present application, by obtaining a target image containing an information code and a background image of the information code; the background image is an image of the environment in which the information code is located, the information code can be judged in combination with the image of the environment in which the information code is located to determine whether the environment in which the information code is located is safe. By extracting information from the background image in the target image from N information extraction dimensions, the image information of the background image in each information extraction dimension in the N information extraction dimensions is obtained, and information extraction from the background image can be realized from multiple dimensions, and image information of more dimensions is obtained, thereby increasing the accuracy of subsequent abnormality detection. Since the image information extracted from multiple information extraction dimensions is subjected to information fusion processing to obtain fused image information, the fused image information may include more image information, and then when the environment in which the information code is located is detected for abnormality based on the fused image information, that is, the environment in which the information code is located is detected for abnormality from multiple dimensions, the accuracy of abnormality detection for the information code can be improved. Furthermore, since the environment in which the abnormal information code is located usually has specific environmental characteristics, the present application can realize abnormality detection of the information code from the environment in which the information code is located, so as to ensure the security of the environment in which the information code is located, thereby improving the accuracy of abnormality detection of the information code. In addition, since the detection model required for sample image training is acquired in advance before the detection model is used to perform anomaly detection on the environment where the information code is located based on the fused image information, the model prediction accuracy can be improved when the detection model is subsequently used to perform anomaly detection on the environment where the information code is located based on the fused image information.

[0239] Optionally, when the program instructions are executed by the processor, other steps of the method in the above embodiment may also be implemented, which will not be repeated here.

[0240] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, which includes program instructions, which, when executed by a computer, cause the computer to execute the method of the aforementioned embodiment, and the computer may be part of the computer device mentioned above. As an example, the program instructions may be deployed on a computer device for execution, or deployed on multiple computer devices located at one location for execution, or, executed on multiple computer devices distributed at multiple locations and interconnected by a communication network, and multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain network.

[0241] The embodiment of the present application also provides a computer program product, which includes a computer program / instruction, and when the computer program / instruction is executed by a processor, some or all of the steps in the above method can be implemented. For example, the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the steps performed in the embodiments of the above methods.

[0242] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0243] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0244] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0245] Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0246] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An information processing method, characterized in that: The method comprises: Acquire a target image to be processed; the target image includes an information code and a background image of the information code, and the background image is an image of the environment in which the information code is located; Extract information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions; N is a positive integer; Performing information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image; Based on the fused image information, an abnormality detection is performed on the environment where the information code is located to obtain a detection result of the environment where the information code is located; the detection result is used to indicate whether the environment where the information code is located is abnormal or normal.

2. The method according to claim 1, characterized in that The N information extraction dimensions include an object extraction dimension and a text extraction dimension; extracting information from the background image in the target image from the N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions includes: Removing the image at the location of the information code in the target image to obtain the background image; Performing object detection on the background image from the object extraction dimension to obtain a detection frame of the object contained in the background image; the detection frame belongs to the image information of the background image under the object extraction dimension; and Text extraction is performed on the background image from the text extraction dimension to obtain text information contained in the background image; the text information belongs to the image information of the background image under the text extraction dimension.

3. The method according to claim 2, characterized in that The performing information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image includes: Extracting the object image in the detection frame from the background image, and calling the detection model to perform embedding processing on the object image to generate an object embedding feature of the object image; Calling the detection model to perform embedding processing on the text information to generate text embedding features of the text information; Performing feature fusion processing on the object embedding feature and the text embedding feature to generate the fused image information; The fusion processing of the object embedding feature and the text embedding feature is any one of the following: splicing processing and addition processing.

4. The method according to claim 3, characterized in that The performing abnormality detection on the environment where the information code is located based on the fused image information to obtain the detection result of the environment where the information code is located includes: Calling the detection model to perform abnormality detection on the environment where the information code is located based on the fused image information, and obtaining a detection probability of the environment where the information code is located; the detection probability is the probability that the environment where the information code is located is abnormal; If the detection probability is greater than or equal to the probability threshold, determining that the detection result is a result indicating that the environment in which the information code is located is abnormal; If the detection probability is less than the probability threshold, it is determined that the detection result is a result indicating that the environment in which the information code is located is normal.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: Acquire a detection model to be trained and a sample image; the sample image includes a sample information code and a sample background image of the sample information code, the sample background image is an image of the environment where the sample information code is located, and the sample image has a sample label, and the sample label is used to indicate whether the environment where the sample information code is located is actually normal or abnormal; Extract information from the sample background image in the sample image from the N information extraction dimensions to obtain sample image information of the sample background image in each of the N information extraction dimensions; Calling the detection model to be trained to perform information fusion processing on the sample image information of the sample background image under the various information extraction dimensions to generate sample fused image information of the sample background image; Calling the detection model to be trained to perform abnormality detection on the environment where the sample information code is located based on the sample fusion image information, and obtaining a sample detection result of the environment where the sample information code is located; the sample detection result is used to indicate whether the detected environment where the sample information code is located is abnormal or normal; Based on the difference between the sample detection result and the sample label, the model parameters of the detection model to be trained are corrected to obtain the detection model.

6. The method according to claim 5, characterized in that There are multiple sample images, and the multiple sample images include positive sample images and negative sample images. The sample labels of the positive sample images are used to indicate that the environment in which the sample information code included in the positive sample images is actually normal, and the sample labels of the negative sample images are used to indicate that the environment in which the sample information code included in the negative sample images is actually abnormal. The ratio between the number of the negative sample images and the number of the positive sample images satisfies a target ratio, and the target ratio is used to make the number of the negative sample images and the number of the positive sample images in a balanced state.

7. The method according to claim 1, characterized in that The information code includes information for executing the target service; the method further includes: If the detection result indicates that the environment in which the information code is located is abnormal, then terminating the execution of the target service; The target service is any one of the following: resource transfer service, relationship establishment service, and information query service.

8. The method according to claim 1, characterized in that The target object is used to initiate the target service based on the information code; the method further includes: If the detection result indicates that the environment in which the information code is located is abnormal, a prompt message indicating that the execution environment of the target service is abnormal is sent to the target terminal of the target object, so that the target terminal outputs the prompt message and execution inquiry information for the target service in association; When receiving the confirmation information sent by the target terminal, executing the target service; the confirmation information is generated by the target terminal after detecting that the output execution query information is confirmed; When the termination information sent by the object terminal is received, the execution of the target service is terminated; the termination information is generated by the object terminal after detecting that the output execution inquiry information is terminated.

9. An information processing device, characterized in that: The device comprises: An image acquisition unit is used to acquire a target image to be processed; the target image includes an information code and a background image of the information code, and the background image is an image of the environment in which the information code is located; An information extraction unit, configured to extract information from the background image in the target image from N information extraction dimensions to obtain image information of the background image in each of the N information extraction dimensions; N is a positive integer; An information fusion unit, used to perform information fusion processing on the image information of the background image in each information extraction dimension to generate fused image information of the background image; An abnormality detection unit is used to perform abnormality detection on the environment where the information code is located based on the fused image information to obtain a detection result of the environment where the information code is located; the detection result is used to indicate whether the environment where the information code is located is abnormal or normal.

10. A computer device, characterized in that: The method comprises a processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer program executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images