Network asset mapping method and apparatus, and storage medium and electronic device

Abstract

The present application relates to the technical field of computers. Disclosed are a network asset mapping method and apparatus, and a storage medium and an electronic device. In the method, a device receives at least one piece of response information sent by a target station on the basis of a network asset request, wherein the at least one piece of response information includes station service information of the target station and multimedia information borne on the target station, the multimedia information at least including text information and image information; and then the device extracts, from the text information, target entity information of a target entity to which the target station belongs, extracts, from the image information, external application information of an external application associated with the target station, and associates the extracted entity object information and external application information and the obtained station service information as network asset information of the target station, so as to obtain a corresponding target mapping result. By means of the present application, multi-dimensional mining of network asset information of a target station is realized, thereby improving the accuracy and comprehensiveness of network asset mapping.

Description

A method, device, storage medium and electronic device for surveying and mapping network assets

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on November 13, 2023, with application number 202311500251.1 and application name "A method, device, storage medium and electronic device for surveying and mapping network assets", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of Internet technology, and in particular to a method, device, storage medium, and electronic device for surveying and mapping network assets. Background Art

[0004] In recent years, with the rapid development of the Internet, network asset (such as network asset information of target sites, etc.) mapping methods have emerged.

[0005] In related technologies, a request is usually made to a target site for the IP address of the target site, and the received IP address is used as the network asset information of the target site to obtain corresponding target mapping results.

[0006] However, the above method only considers basic site information such as IP addresses, and has the problem of a relatively single surveying dimension. The surveying results obtained based on this method are difficult to provide rich and comprehensive effective information for the subsequent network asset association analysis, thereby affecting the accuracy of the subsequent network asset association analysis.

[0007] Summary of the Invention

[0008] The present application provides a network asset mapping method, device, storage medium and electronic device for mining multi-dimensional network asset information and improving the accuracy and comprehensiveness of network asset mapping.

[0009] In a first aspect, the present application provides a method for surveying and mapping network assets, the method comprising:

[0010] Receiving at least one response message sent by a target site based on a network asset request; wherein the at least one response message includes: site service information of the target site and multimedia information carried by the target site, the multimedia information including at least text information and image information;

[0011] Extracting target entity information of a target entity to which the target site belongs from the text information; and

[0012] extracting external application information of the external application associated with the target site from the image information;

[0013] The entity object information, the external application information and the site service information are associated with each other as the network asset information of the target site to obtain a corresponding target mapping result.

[0014] In a second aspect, the present application provides a network asset mapping device, the device comprising:

[0015] a receiving unit, configured to receive at least one response message sent by a target site based on a network asset request; wherein the at least one response message includes: site service information of the target site and multimedia information carried by the target site, the multimedia information including at least text information and image information;

[0016] an extraction unit, configured to extract target entity information of a target entity to which the target site belongs from the text information; and to extract external application information of an external application associated with the target site from the image information;

[0017] The obtaining unit is configured to associate the entity object information, the external application information, and the site service information into the network asset information of the target site to obtain a corresponding target mapping result.

[0018] Optionally, the extraction unit is configured to extract external application information of an external application associated with the target site from the image information, specifically to:

[0019] Based on a preset target classification index, at least one target image belonging to a target category is selected from the image information; wherein the target classification index represents: a classification method for the target category;

[0020] For the at least one target image, the following operations are respectively performed: in a target image, at least one target sub-image represented by a specified graphic is extracted; wherein each target sub-image is generated according to a geometric pattern indicated by a preset information encoding method;

[0021] Based on the external application associated with each target sub-image, image coding and parsing processing is performed on each extracted target sub-image to obtain the external application information of the target site.

[0022] Optionally, if the image information includes text information, the extraction unit is further configured to:

[0023] Using optical character recognition, converting the text information in the image information from an image representation to a text representation to obtain a text representation in the text information;

[0024] In the text representation, entity description information of the associated entity of the target site is extracted, and the entity description information is associated with the network asset information of the target site.

[0025] Optionally, the multimedia information further includes file information, and the file information includes at least one document file;

[0026] Then the extraction unit is further used for:

[0027] For the at least one document file, respectively performing the following operations: acquiring document description information of a document file, and extracting document content information of the document file to obtain corresponding document file information;

[0028] The document file information corresponding to each of the at least one document file is associated with the network asset information of the target site.

[0029] Optionally, the multimedia information further includes audio information, and the audio information includes at least one audio file;

[0030] Then the extraction unit is further used for:

[0031] For the at least one audio file, respectively performing the following operations: acquiring audio description information of an audio file, and extracting audio content information of the audio file to obtain corresponding audio file information;

[0032] The audio file information corresponding to each of the at least one audio file is associated with the network asset information of the target site.

[0033] Optionally, the multimedia information further includes video information, the video information includes at least one video file, and each video file includes: a plurality of consecutive image frames;

[0034] Then the extraction unit is further used for:

[0035] For the at least one video file, respectively performing the following operations: acquiring video description information of a video file, and extracting video content information of a head image frame and a tail image frame for the video file to obtain corresponding video file information;

[0036] The video file information corresponding to each of the at least one video file is associated with the network asset information of the target site.

[0037] Optionally, the one video file further includes: audio frames corresponding to the multiple image frames;

[0038] Then the extraction unit is further used for:

[0039] Using a speech recognition method, performing speech recognition processing on each audio frame in the video file to obtain a corresponding speech text representation;

[0040] Extracting entity description information of associated entities of the target site from the speech text representation, and associating the entity description information with network asset information of the target site;

[0041] The video file further includes: subtitle information corresponding to the plurality of image frames;

[0042] Then the extraction unit is further used for:

[0043] Using subtitle extraction technology, performing subtitle extraction processing on each subtitle information in the video file to obtain corresponding subtitle text representation;

[0044] In the subtitle text representation, entity description information of the associated entity of the target site is extracted, and the entity description information is associated with the network asset information of the target site.

[0045] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the network asset mapping methods described in the first aspect is implemented.

[0046] In a fourth aspect, the present application provides a computer storage medium, wherein the computer-readable storage medium stores computer program instructions, and the computer program instructions are executed by a processor to implement any one of the network asset mapping methods described in the first aspect.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, which, when executed by a processor, implement any one of the network asset surveying and mapping methods described in the first aspect.

[0048] The beneficial effects of this application are as follows:

[0049] In an embodiment of the present application, a network asset mapping method, apparatus, device, and storage medium are provided to mine multi-dimensional network asset information and improve the accuracy and comprehensiveness of network asset mapping.

[0050] In the network asset mapping method provided in the embodiment of the present application, the device receives at least one response message sent by the target site based on the network asset request, which includes: the site service information of the target site and the multimedia information carried by the target site, and the multimedia information at least includes text information and image information. It is easy to understand that the embodiment of the present application not only focuses on the site service information of the target site, but also focuses on the multimedia information carried by the target site, especially the text information and image information carried by the target site. Then, by mapping and processing the text information and image information, the network asset information of the target site can be more accurately and comprehensively reflected; for example, for information security reasons, when performing network asset association analysis based on the mapped network asset information, not only can the site service information of the target site be analyzed to trace the network security, but also the graphic information of the target site can be combined for analysis, thereby obtaining accurate and rich analysis results to improve the management efficiency of the corresponding network asset information.

[0051] Then, the target entity information of the target entity to which the target site belongs is extracted from the text information, and the external application information of the external application associated with the target site is extracted from the image information. The extracted entity object information, external application information, and the obtained site service information are then associated with the network asset information of the target site to obtain the corresponding target mapping result. In this way, multi-dimensional mining of the network asset information of the target site is achieved. On the one hand, the target entity to which the target site belongs is mined from the text information carried on the target site. This facilitates the subsequent acquisition of the corresponding target entity information based on the target mapping result for information security considerations. On the other hand, the external application associated with the target site is mined from the image information carried on the target site. This facilitates the subsequent acquisition of the corresponding external application information based on the target mapping result for information security considerations. In this way, the target mapping result obtained in the embodiment of the present application can not only reflect the site service information of the target site, but also reflect the site content information of the target site, such as target entity information, external application information, etc., achieving multi-dimensional mapping of the corresponding network asset information, thereby effectively improving the processing efficiency and accuracy of subsequent network asset association analysis.

[0052] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0054] FIG1 is a schematic diagram of an optional application scenario in an embodiment of the present application;

[0055] FIG2 is a flow chart of a method for surveying and mapping network assets according to an embodiment of the present application;

[0056] FIG3 is a schematic diagram of a process for obtaining site service information according to an embodiment of the present application;

[0057] FIG4 is a schematic diagram of a process for obtaining target entity information according to an embodiment of the present application;

[0058] FIG5 is a schematic diagram of a process for extracting image information in an embodiment of the present application;

[0059] FIG6 is a schematic diagram of a process for obtaining document file information according to an embodiment of the present application;

[0060] FIG7 is a schematic diagram of a process for obtaining audio file information according to an embodiment of the present application;

[0061] FIG8 is a schematic diagram of a process for extracting video information in an embodiment of the present application;

[0062] FIG9 is a schematic diagram of a possible target mapping result in an embodiment of the present application;

[0063] FIG10 is a schematic diagram of the structure of a network asset surveying and mapping device provided in an embodiment of the present application;

[0064] FIG11 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0066] In the embodiments of the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0067] The embodiments of this application relate to artificial intelligence technology, mainly to computer vision technology, speech technology, natural language processing technology, and machine learning technology in artificial intelligence technology.

[0068] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0069] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0070] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects, and further image processing to transform the computer into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Large model technology has brought significant changes to the development of computer vision technology. Pre-trained models in the field of vision, such as the Swin Transformer, ViT, V-MOE, and MAE, can be fine-tuned to quickly and widely apply to specific downstream tasks. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0071] Key technologies in speech technology include automatic speech recognition, speech synthesis, and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech becoming one of the most promising methods of human-computer interaction.

[0072] Natural language processing (NLP) is an important field in the fields of computer science and artificial intelligence. It studies various theories and methods that enable effective communication between humans and computers using natural language. Natural language processing involves natural language, that is, the language people use in daily life, and is closely related to linguistic research; it also involves computer science and mathematics. The pre-training model, an important technology for model training in the field of artificial intelligence, is developed from the large language model (LLM) in the field of NLP. After fine-tuning, the large language model can be widely used in downstream tasks. Natural language processing technology generally includes text processing, semantic understanding, machine translation, robot question answering, knowledge graphs and other technologies.

[0073] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning. Pretrained models are the latest development in deep learning, integrating these techniques.

[0074] In an embodiment of the present application, artificial intelligence technology is applied to the field of asset mapping, specifically to generate target mapping results of corresponding network asset information for target sites, so as to achieve comprehensiveness and accuracy of asset mapping.

[0075] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are first explained.

[0076] Multimedia information: A medium for interactive information exchange and dissemination between humans and computers, or a combination of two or more media, for interactive information exchange and dissemination between humans and computers. Media includes images, text, sound, and video. In the embodiments of this application, multimedia information includes at least text and image information. Of course, multimedia information may also include audio and video information, and may specifically take the form of images, videos, advertisements, articles, news, music, and other content, though this application does not impose any restrictions on this.

[0077] Target subgraph: In an embodiment of the present application, the target subgraph is an image extracted for a specified image representation in a target image that is determined to be a target category; and the target subgraph is generated according to the geometric pattern indicated by the preset information encoding method. Specifically, common URLs, telephone numbers, email addresses and other information can be presented in the form of a specified image representation for quick access by the user in a triggering manner (such as scanning, etc.). Exemplarily, the specified image representation can be a QR code (QR Code), which is a black and white graphic representation of data information distributed regularly on a plane (two-dimensional direction) using specific geometric figures according to an information encoding method.

[0078] Hypertext Markup Language (HTML): A standard markup language for creating web pages. HTML uses a series of tags and attributes to describe the structure, content, and layout of a document, making it interactive and readable across platforms. HTML text is descriptive text composed of HTML commands that can describe text, images, audio, video, and other elements.

[0079] Transmission Control Protocol / Internet Protocol (TCP / IP): is a collection of network communication protocols and a reference standard that enables various types of computers and devices to communicate with each other and transmit information over the Internet.

[0080] Secure Socket Layer (SSL): A network transmission security protocol.

[0081] Transport Layer Security (TLS): A standardized transport layer security protocol.

[0082] Certificate: A digital certificate used for secure network communications.

[0083] Hyper Text Transfer Protocol (HTTP): A network application layer protocol.

[0084] Hyper Text Transfer Protocol Secure (HTTPS): A protocol that uses SSL / TLS to provide website authentication and encrypted data transmission.

[0085] Uniform Resource Locator (URL): A URL is a unique address for a resource on the Internet. It can be used to find and retrieve information on the Internet using a web browser.

[0086] Optical Character Recognition (OCR) is a technology that converts printed or handwritten text into digital text that can be edited and processed by a computer. OCR technology can convert text in scanned paper or images into editable text.

[0087] Internet Content Provider (ICP): ICP filing is a mandatory registration procedure for Internet Content Providers (ICPs) providing internet information services, such as websites and applications, within a designated territory. ICP filing is designed to oversee and manage internet information services and ensure the legality, security, and reliability of online content.

[0088] The following is a brief introduction to the design concept of the embodiments of this application.

[0089] With the rise of the internet, the inventors discovered through creative work that the current field of asset mapping suffers from a single dimension. The key issue is that with the widespread dissemination of multimedia information on the internet, information security-related information, such as large amounts of information and data about organizations, enterprises, groups, and individuals, has been widely circulated in the form of multimedia information on various websites (sites) on the internet. However, there is currently a lack of a network asset mapping method that integrates multimedia information. Furthermore, internet sites serve as a medium and transit point for users to obtain information and consult with them, but there is currently a lack of a way to understand the network assets associated with the sites (i.e., associated external applications such as official accounts, Weibo, mini-programs, and clients).

[0090] In view of this, an embodiment of the present application provides a method for surveying and mapping network assets, in which a multi-dimensional mining method for network asset information is provided. In this method, a device receives at least one response message sent by a target site based on a network asset request, which includes: site service information of the target site and multimedia information carried by the target site, and the multimedia information includes at least text information and image information. Then, the target entity information of the target entity to which the target site belongs is extracted from the text information, and the external application information of the external application associated with the target site is extracted from the image information. The extracted entity object information, external application information, and the obtained site service information are then associated with the network asset information of the target site to obtain the corresponding target surveying and mapping result. It is easy to understand that the embodiments of the present application not only focus on the site service information of the target site, but also focus on the multimedia information carried by the target site, especially the text information and image information carried by the target site. Subsequently, by mapping and processing the text information and image information, the network asset information of the target site can be more accurately and comprehensively reflected; in addition, in the embodiments of the present application, on the one hand, for the text information carried on the target site, the target entity to which the target site belongs is mined, so that for subsequent information security considerations, the corresponding target entity information can be directly obtained based on the target mapping results; on the other hand, for the image information carried on the target site, the external application associated with the target site is mined, so that for subsequent information security considerations, the corresponding external application information can be directly obtained based on the target mapping results, thereby realizing multi-dimensional mining of the network asset information of the target site.

[0091] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0092] The solution provided by the embodiment of the present application can be applied to most scenarios involving asset mapping, such as asset mapping scenarios of target sites, etc. As shown in Figure 1, an application scenario provided by the embodiment of the present application is shown, in which a terminal device 110 and a server 120 may be included.

[0093] The terminal device 110 may be, for example, a mobile phone, a tablet computer (PAD), a laptop computer, a desktop computer, a smart TV, a smart vehicle-mounted device, or a smart wearable device. The terminal device 110 may be installed with or connected to an asset mapping platform capable of performing asset mapping. The asset mapping platform involved in the embodiments of the present application may be a mini-program application, or a client such as a webpage or software. The server 120 is a backend server corresponding to the mini-program, webpage, or software, and the specific type of the platform is not limited.

[0094] The server 120 may be a backend server corresponding to the asset mapping platform installed or connected to the terminal device 110, which may implement the backend functions of network asset mapping to implement the steps of the network asset mapping method provided in the embodiment of the present application. For example, it may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, i.e., content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited thereto.

[0095] The network asset mapping method provided in the embodiment of the present application can be executed by the terminal device 110 or the server 120, or can also be executed by the terminal device 110 and the server 120 in combination. The terminal device 110 or the server 120 may include one or more processors, memories, and an I / O interface for interacting with the terminal, etc. In addition, the terminal device 110 or the server 120 may also be configured with a database, or interact with the server 120 configured with a database, and the database may be used to store preset component nodes and component tree disassembly records, etc. Among them, the program instructions of the network asset mapping method provided in the embodiment of the present application may also be stored in the memory of the terminal device 110 or the server 120. When these program instructions are executed by the processor, they can be used to implement the steps of the network asset mapping method provided in the embodiment of the present application, so as to realize multi-dimensional mapping of network assets.

[0096] The terminal device 110 and the server 120 can be directly or indirectly connected to each other through one or more networks 130. The network 130 can be a wired network or a wireless network, for example, a mobile cellular network or a Wireless Fidelity (WIFI) network. Of course, other possible networks are also possible, and the embodiment of the present invention does not limit this.

[0097] It should be noted that in the embodiment of the present application, the number of terminal devices 110 can be one or more, and similarly, the number of servers 120 can be one or more, that is, there is no restriction on the number of terminal devices 110 or servers 120.

[0098] In a possible application scenario, in order to reduce the communication delay of retrieval, servers 120 can be deployed in various regions, or for load balancing, different servers 120 can serve terminal devices 110 in different regions respectively. For example, the terminal device 110 is located at location a and establishes a communication connection with the server 120 serving location a. The terminal device 110 is located at location b and establishes a communication connection with the server 120 serving location b. Multiple servers 120 form a data sharing system, and data sharing is achieved through blockchain.

[0099] Each server 120 in the data sharing system has a node identifier corresponding to that server 120. Each server 120 in the data sharing system can store the node identifiers of other servers 120 in the data sharing system, so that the generated blocks can be broadcast to other servers 120 in the data sharing system based on the node identifiers of other servers 120. Each server 120 can maintain a node identifier list, storing the server 120 name and node identifier in the node identifier list. The node identifier can be an Internet Protocol (IP) address for interconnecting networks or any other information that can be used to identify the node.

[0100] Of course, the method provided in the embodiment of the present application is not limited to the application scenario shown in Figure 1, and can also be used in other possible application scenarios, which are not limited by the embodiment of the present application. The functions that can be implemented by each device in the application scenario shown in Figure 1 will be described in the subsequent method embodiments, and will not be described in detail here.

[0101] Below, in conjunction with the accompanying drawings, the method flow provided in each embodiment of the present application can be executed by the server 120 or the terminal device 110 in Figure 1, or can be executed jointly by the terminal device 110 and the server 120. Here, the execution by the server 120 is mainly introduced as an example.

[0102] See FIG2 , which is a flow chart of the application development method provided in an embodiment of the present application.

[0103] Step 201: Receive at least one response message sent by a target site based on a network asset request; wherein the at least one response message includes: site service information of the target site and multimedia information carried by the target site, the multimedia information including at least text information and image information.

[0104] Among them, the site service information of the target site includes but is not limited to: IP address, service port, service protocol, SSL / TLS certificate, and key information of the SSL / TLS certificate (such as: organization name, geographical location, domain name information, email address, IP list, etc.).

[0105] In one possible implementation, refer to Figure 3, which is a schematic diagram of a process for obtaining site service information in an embodiment of the present application; wherein, in response to establishing a network connection with the target site, the terminal device receives response information sent by the target site for the site service information request, and when the response information includes TCP / IP network transmission protocol information, the terminal device parses the service IP address, service port, service protocol and other site service information of the target site based on the TCP / IP network transmission protocol information contained therein.

[0106] Optionally, as shown in Figure 3, when the above service protocol is HTTPS service, the server-side SSL / TLS digital certificate asset data of the server where the target site is located is obtained, and based on this, the key information of the SSL / TLS digital certificate is parsed and extracted, such as the organization name, geographic location, domain name information, email address, IP list, etc. of the SSL / TLS digital certificate, together as the corresponding site service information.

[0107] Furthermore, in response to establishing a network connection with the target site, the terminal device receives a response message sent by the target site in response to the site content information request, parses the load multimedia information about the target site in the response message, and obtains the dynamic or static multimedia information carried by the target site, including at least text information and image information, and of course, may also include: file information, audio information, video information, etc., and the embodiments of the present application do not impose specific restrictions on this.

[0108] It is worth noting that the aforementioned request information, such as the site service information request and site content request, can be generated after determining the target site's access address, which includes but is not limited to the target site's website domain name, IP address, or URL. Alternatively, the aforementioned request information can be generated using legal and compliant website crawler technology, targeting the target site, or by simulating a browser and issuing a request to the target site.

[0109] Step 202: extracting target entity information of a target entity to which the target site belongs from the text information; and extracting external application information of an external application associated with the target site from the image information.

[0110] In an embodiment of the present application, the multimedia information includes at least text information and image information. By mining the multimedia information carried by the target site, multi-dimensional mapping of the network asset information of the target site is achieved. For ease of understanding, the following is divided into two parts (such as: the first part and the second part) to respectively explain the process of mining network asset information for text information and image information.

[0111] In the first part, the target entity information of the target entity to which the target site belongs is extracted from the text information.

[0112] The target entity to which the target site belongs may be: the owner, registrant, operator, etc. of the target site (website / domain name). The target entity information is descriptive information of the target entity, and may include but is not limited to: the name of the organization, the website ICP filing number, the network security filing number, the government website identification code, email address, phone number, and mailing address.

[0113] Refer to Figure 4, which is a schematic diagram of a process for obtaining target entity information in an embodiment of the present application; wherein, based on natural language processing technology and pattern matching technology, target entity information related to the target entity to which it belongs is extracted from text information, such as: unit organization name, website ICP registration number, network security registration number, government website identification code, email information, telephone information, communication address, etc., and the obtained target entity information is converted into structured text for storage.

[0114] Specifically, based on natural language processing technology, the text information is segmented to obtain corresponding segmentation results, and then based on pattern matching technology, target segmentations related to the target entity are selected from the segmentation results to obtain corresponding target entity information.

[0115] It should be noted that the text information involved in the above word segmentation process includes but is not limited to: the text title in the target site, the HTML text content carried by the target site, and other text content carried by the target site.

[0116] The second part extracts the external application information of the external application associated with the target site from the image information.

[0117] Among them, external application information includes but is not limited to: application description information of external applications (such as email, address, etc.), application platform information of external applications (such as official accounts, mini-programs, clients, web pages, etc.), and application link information of web applications (such as address coding information, etc.).

[0118] Specifically, based on a preset target classification indicator, at least one target image belonging to a target category is selected from the image information, wherein the target classification indicator represents: a classification method for the target category, and then for at least one target image, the following operations are performed respectively: in a target image, at least one target sub-image represented by a specified graphic is extracted, wherein each target sub-image: is generated according to the geometric pattern indicated by the preset information encoding method, and then based on the external application associated with each target sub-image, each extracted target sub-image is subjected to image encoding and parsing processing to obtain the external application information of the target site.

[0119] It should be noted that the above-mentioned target sub-graph is an image extracted for a specified image representation in a target image that is determined to be a target category; and the target sub-graph is generated according to the geometric pattern indicated by the preset information encoding method. In detail, the target sub-graph may contain information such as a website, a telephone number, an email address, etc. The target sub-graph is presented in the form of a specified image representation and can be triggered by an object for quick access. Exemplarily, the specified image representation can be in the form of a QR code, then the target category is a QR code, and the target sub-graph is a screenshot of the QR code, wherein the QR code is a black and white graphic representation of data information distributed regularly on a plane (two-dimensional direction) using specific geometric figures according to an information encoding method.

[0120] In an optional implementation, the above-mentioned image information also includes text information. Then, optical character recognition is further used to convert the text information in the image information from image representation to text representation to obtain text representation in the text information. Then, in the text representation, entity description information of the associated entity of the target site is extracted, the entity description information is stored, and the entity description information is subsequently associated with the network asset information of the target site.

[0121] Furthermore, if the text information in the image information is watermark text, then content recognition (image recognition) technology is used to extract the watermark text, obtain the corresponding text representation, and then extract the entity description information related to the associated entity of the target site (such as: telephone, ID nickname, name, etc.).

[0122] In an optional implementation, image description information of each image can also be extracted for the image information. The image description information includes but is not limited to: image format type (such as PNG, JPG, GIF), image type, image resolution, image acquisition geographic location, and text content contained in the image.

[0123] In summary, as a relatively complete specific example, refer to FIG5 , which is a schematic diagram of a process for extracting image information in an embodiment of the present application. First, for the image information, extract the image description information of each image (such as image format type, image type, image resolution, image acquisition geographical location and other text content, etc.); second, use content recognition technology to extract the watermark text from the image information as entity description information related to the associated entity of the target site (such as telephone number, ID nickname, name, etc.); third, for the target image, extract at least one target sub-graph represented by a specified graphic (such as: QR code form), and obtain the external application information of the target site by performing image encoding and parsing processing on each extracted target sub-graph.

[0124] Furthermore, in an optional implementation method, the multimedia information may also include any one or a combination of file information, audio information and video information. Based on the above two parts, the above multi-dimensional mapping of the network asset information of the target site can be extended into the following three parts (such as: the third part, the fourth part, and the fifth part) to respectively explain the process of mining network asset information for file information, audio information and video information.

[0125] The third part, the multimedia information also includes file information, which includes at least one document file. Then, the document file information of each document file carried by the target site is extracted from the file information.

[0126] The file format of a document file includes, but is not limited to, Office, PDF, and Markdown. The document file information of a document file includes document description information and document content information. The document description information includes, but is not limited to, the document author, document publication date, organization information, and contact information. The document content information includes, but is not limited to, the document title, document keywords, and document content summary.

[0127] Referring to FIG6 , which is a schematic diagram of a process for obtaining document file information in an embodiment of the present application, the following operations are performed for at least one document file in the file information: obtaining document description information for the document file, extracting document content information for the document file, obtaining corresponding document file information, storing the extracted document file information, and subsequently associating the document file information corresponding to the at least one document file with the network asset information of the target site.

[0128] The fourth part, the multimedia information also includes audio information, which includes at least one audio file. The audio file information of each audio file carried by the target site is extracted from the audio information.

[0129] The file formats of audio files include, but are not limited to, WMV, MP3, and AVI. The audio file information of an audio file includes audio description information and audio content information. The audio description information includes, but is not limited to, the audio format, audio duration, and audio type. The audio content information includes, but is not limited to, the audio name, audio keywords, audio text within the audio, and a short text summary (which can be used to reduce storage space usage and improve the efficiency of subsequent analysis of the target site's network asset information).

[0130] Referring to FIG. 7 , a schematic diagram of a process for obtaining audio file information according to an embodiment of the present application is shown. For at least one audio file, the following operations are performed: obtaining audio description information for the audio file; extracting audio content information for the audio file; obtaining corresponding audio file information; and storing the extracted audio file information. Subsequently, the audio file information corresponding to each of the at least one audio file is associated with the network asset information of the target site.

[0131] The fifth part, the multimedia information also includes video information, and the video file information of each video file carried by the target site is extracted from the video information.

[0132] In one possible implementation, the above-mentioned video information includes at least one video file, and each video file includes: multiple consecutive image frames. Then, for at least one video file, the following operations are performed respectively: video description information of a video file is obtained, and video content information of a head image frame and a tail image frame of a video file is extracted to obtain corresponding video file information, and the obtained video file information is stored. Subsequently, the video file information corresponding to each of the at least one video file is associated with the network asset information of the target site.

[0133] Among them, the above-mentioned video description information includes but is not limited to: video format (such as: MP4, AVI, WMV, FLV, RMVB, etc.), video length, video frame rate, video resolution, and the above-mentioned video content information includes but is not limited to: video name, video type, video frame summary, video text summary, other text information in the video, head image frame, and tail image frame.

[0134] It should be noted that the number of the head image frame and the tail image frame can be determined according to actual conditions. The head image frame and the tail image frame can include relevant information of the video producer, etc., which is not specifically limited here.

[0135] Optionally, a video file also includes: audio frames corresponding to multiple image frames. In this case, voice recognition will be used to perform voice recognition processing on each audio frame in a video file to obtain a corresponding voice text representation, and from the voice text representation, extract the entity description information of the associated entity of the target site, store the entity description information, and subsequently associate the entity description information with the network asset information of the target site.

[0136] The entity description information includes but is not limited to: entity name, entity ID, unit introduction information, contact information, audio summary, etc.

[0137] Optionally, a video file also includes: subtitle information corresponding to multiple image frames, and subtitle extraction technology will also be used to perform subtitle extraction processing on each subtitle information in a video file to obtain the corresponding subtitle text representation, and from the subtitle text representation, extract the entity description information of the associated entity of the target site, store the entity description information, and subsequently associate the entity description information with the network asset information of the target site.

[0138] The entity description information includes but is not limited to: video subtitle text and video subtitle summary.

[0139] It should be noted that when an image frame in a video file is of the target type, that is, the target image described in the second part above, the relevant contents of the second part above are executed and will not be repeated here.

[0140] In summary, as a relatively complete specific example, refer to FIG8 , which is a schematic diagram of a process for extracting video information in an embodiment of the present application. First, for the video information, extract the video description information of each video file (such as: video format, video duration, video frame rate, video resolution, etc.); second, use video processing technology and image recognition technology to extract the video content information of the head image frame and the tail image frame for each video file (such as: video name, video type, video frame summary, video text summary, other text information in the video, head image frame, tail image frame); third, use speech recognition to extract the entity description information of the associated entity of the target site (such as: entity name, entity ID, unit introduction information, contact information, audio summary, etc.) for each audio frame in each video file; fourth, use subtitle extraction technology to extract the entity description information of the associated entity of the target site (such as: video subtitle text, video subtitle summary, etc.) for each subtitle information in each video file.

[0141] Step 203: Associate the entity object information, external application information, and site service information into the network asset information of the target site to obtain a corresponding target mapping result.

[0142] Optionally, the multimedia information of the target site includes text information, image information, file information, audio information and video information, and the corresponding target mapping result can be shown in FIG9 .

[0143] As shown in FIG9 , the target mapping results are stored in structured text, wherein the first row includes: title information of site service information and title information from multimedia information, and each column is the extracted information under the corresponding title information, as follows.

[0144] Site service information includes: service IP, host name, service port, service protocol, service application, HTTPS certificate, and other information.

[0145] The information extracted from text information (i.e., the entity object information of the target entity to which the target site belongs) includes: website URL, website title, navigation menu, text content, website fingerprint, ICP registration number, public security network security registration number, government website identification code, unit name, mailing address, email, telephone number, and other information.

[0146] The information extracted from the file includes: file URL, file name, file format, file size, file author, file time, file fingerprint, file title, file keywords, file content summary, organizational unit, included pictures, included audio, included video, and other information.

[0147] The information extracted from image information includes: image URL, image name, image format, image size, resolution, image time, image fingerprint, geographic location, image type, image text, external application information (such as: official account QR code, mini program QR code, client QR code, other QR codes), image watermark, and other information.

[0148] The information extracted from audio information includes: audio URL, audio name, audio format, audio size, sampling rate, audio duration, modification time, voice text, keywords, summary, audio fingerprint, and other information.

[0149] The information extracted from the video includes: video URL, video name, video format, video size, frame rate, video duration, modification time, video fingerprint, video classification, video subtitle text, video voice text, head image frame, tail image frame, external application frame, external application information (such as: official account QR code, mini program QR code, client QR code, other QR code), video summary, and other information.

[0150] In summary, the embodiments of the present application provide a method for mapping network assets, which performs multi-dimensional asset data mining on the resource content and transport layer data contained in the website webpage of the target site, including but not limited to TCP / IP protocol, TLS / SSL digital certificates, text, files, pictures, audio and video resources, so that the mapping resource types are more comprehensive, the mapping content is more in-depth, and the asset dimension data is richer. It can fully utilize the latest mature technologies such as natural language processing technology, image-to-text, audio processing, and video processing, and the collected asset information clues are more accurate and rich. Ultimately, an asset database covering multi-dimensional information is established for network assets, which can provide more comprehensive key clue information for asset association and asset discovery, improve the effect of network asset discovery and asset association, and play a greater role in cyberspace mapping, as follows:

[0151] 1. In terms of mapping digital assets of websites (i.e., network resource information of the target site), the embodiments of the present application comprehensively collect the resource content and transport layer data contained in the website web pages of the target site, including but not limited to TCP / IP protocol, TLS / SSL digital certificates, text, files, pictures, audio and video resources. The mapping dimensions are more extensive and the coverage of mapping resource types is more comprehensive. The mapping content is more in-depth and adopts the latest mature technologies such as natural language processing technology, image-to-text, audio processing, and video processing. The asset information clues obtained by mapping are more accurate and rich.

[0152] 2. When mapping the website digital assets (i.e., network asset information) of the target site, the embodiment of the present application uses emerging technologies such as image classification and recognition to deeply extract key information of image assets in the website, and performs recognition and analysis on QR code images. It can associate emerging mobile assets such as WeChat public accounts, Weibo accounts, mini-program accounts, and clients. The mapping and collection covers important data assets such as images, and expands the information clues contained in the website image digital asset data.

[0153] 3. When mapping the website digital assets (i.e., network asset information) of the target site, the embodiment of the present application uses emerging technologies such as voice recognition and video processing to deeply extract the key information of the audio and video assets in the website, extract key information and generate summaries for audio and video respectively. The mapping and collection covers the mapping of important data assets such as audio and video, and expands the information clues contained in the website's audio digital asset data.

[0154] 4. When mapping the website digital assets (i.e., network asset information) of the target site, the embodiment of the present application deeply extracts the key information of the text file, forming refined text file metadata. The mapping and collection covers data assets such as files, and expands the clue information contained in the website text data.

[0155] Referring to FIG. 10 , based on the same inventive concept, an embodiment of the present application further provides a network asset mapping device 1000 , which includes:

[0156] The receiving unit 1001 is configured to receive at least one response message sent by a target site based on a network asset request; wherein the at least one response message includes: site service information of the target site and multimedia information carried by the target site, wherein the multimedia information includes at least text information and image information;

[0157] The extraction unit 1002 is configured to extract target entity information of a target entity to which the target site belongs from the text information; and to extract external application information of an external application associated with the target site from the image information;

[0158] The obtaining unit 1003 is configured to associate the entity object information, the external application information and the site service information into the network asset information of the target site to obtain the corresponding target mapping result.

[0159] Optionally, the extraction unit 1002 is configured to extract external application information of an external application associated with the target site from the image information, specifically to:

[0160] Based on a preset target classification index, at least one target image belonging to the target category is selected from the image information; wherein the target classification index represents: a classification method for the target category;

[0161] For at least one target image, the following operations are performed respectively: extracting at least one target sub-image represented by a specified graphic from the target image; wherein each target sub-image is generated according to a geometric pattern indicated by a preset information encoding method;

[0162] Based on the external applications associated with each target sub-image, image coding and parsing processing is performed on each extracted target sub-image to obtain the external application information of the target site.

[0163] Optionally, if the image information includes text information, the extraction unit 1002 is further configured to:

[0164] Using optical character recognition, the text information in the image information is converted from image representation to text representation to obtain text representation in the text information;

[0165] In the text representation, entity description information of the associated entities of the target site is extracted, and the entity description information is associated as the network asset information of the target site.

[0166] Optionally, the multimedia information further includes file information, and the file information includes at least one document file;

[0167] The extraction unit 1002 is further configured to:

[0168] For at least one document file, the following operations are performed respectively: obtaining document description information of a document file, and extracting document content information of a document file to obtain corresponding document file information;

[0169] The document file information corresponding to the at least one document file is associated with the network asset information of the target site.

[0170] Optionally, the multimedia information further includes audio information, and the audio information includes at least one audio file;

[0171] The extraction unit 1002 is further configured to:

[0172] For at least one audio file, performing the following operations respectively: obtaining audio description information of an audio file, and extracting audio content information of an audio file to obtain corresponding audio file information;

[0173] The audio file information corresponding to each of the at least one audio file is associated with the network asset information of the target site.

[0174] Optionally, the multimedia information further includes video information, the video information includes at least one video file, and each video file includes: a plurality of consecutive image frames;

[0175] The extraction unit 1002 is further configured to:

[0176] For at least one video file, the following operations are performed respectively: obtaining video description information of a video file, and extracting video content information of a head image frame and a tail image frame of a video file to obtain corresponding video file information;

[0177] The video file information corresponding to each of the at least one video file is associated with the network asset information of the target site.

[0178] Optionally, a video file further includes: audio frames corresponding to the plurality of image frames;

[0179] The extraction unit 1002 is further configured to:

[0180] Using speech recognition, each audio frame in a video file is processed for speech recognition to obtain the corresponding speech text representation;

[0181] Extracting entity description information of associated entities of the target site from the speech-to-text representation, and associating the entity description information with network asset information of the target site;

[0182] A video file also includes: subtitle information corresponding to multiple image frames;

[0183] The extraction unit 1002 is further configured to:

[0184] Using subtitle extraction technology, subtitle extraction and processing are performed on each subtitle information in a video file to obtain the corresponding subtitle text representation;

[0185] In the subtitle text representation, entity description information of the associated entity of the target site is extracted, and the entity description information is associated with the network asset information of the target site.

[0186] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, please refer to the description of the aforementioned embodiments and no further details will be given.

[0187] Please refer to Figure 11. Based on the same technical concept, an embodiment of the present application also provides a computer device 1100. The computer device 1100 can be the terminal device or server shown in Figure 1. The computer device 1100 can include a memory 1101 and a processor 1102.

[0188] Memory 1101 is used to store computer programs executed by processor 1102. Memory 1101 may primarily include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the like; the data storage area may store data generated based on the use of the computer device. Processor 1102 may be a central processing unit (CPU), a digital processing unit, or the like. The specific connection medium between the memory 1101 and processor 1102 is not limited in the embodiments of the present application. In FIG11 , the memory 1101 and processor 1102 are connected via bus 1103, which is represented by a bold line in FIG11 . The connection methods between other components are for illustrative purposes only and are not intended to be limiting. Bus 1103 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, FIG11 uses only one bold line, but this does not imply that there is only one bus or only one type of bus.

[0189] Memory 1101 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1101 may be a combination of the aforementioned memories.

[0190] The processor 1102 is configured to execute the method executed by the device in each embodiment of the present application when calling the computer program stored in the memory 1101 .

[0191] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to the various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the method executed by the device in each embodiment of the present application.

[0192] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0193] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0194] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for mapping network assets, characterized in that: The method comprises: Receiving at least one response message sent by the target site based on the network asset request; wherein the at least one response message includes: site service information of the target site and multimedia information carried by the target site, the multimedia information at least including text information and image information; Extracting target entity information of a target entity to which the target site belongs from the text information; and extracting external application information of an external application associated with the target site from the image information; The entity object information, the external application information and the site service information are associated with the network asset information of the target site to obtain a corresponding target mapping result.

2. The method according to claim 1, characterized in that The step of extracting the external application information of the external application associated with the target site from the image information includes: Based on a preset target classification index, at least one target image belonging to a target category is selected from the image information; wherein the target classification index represents: a classification method for the target category; For the at least one target image, the following operations are respectively performed: in a target image, at least one target sub-image represented by a specified graphic is extracted; wherein each target sub-image: is generated according to a geometric pattern indicated by a preset information encoding method; Based on the external application associated with each target sub-image, image coding and parsing processing is performed on each extracted target sub-image to obtain the external application information of the target site.

3. The method according to claim 1, characterized in that If the image information includes text information, the method further includes: Using optical character recognition, converting the text information in the image information from image representation to text representation, thereby obtaining the text representation in the text information; In the text representation, entity description information of the associated entity of the target site is extracted, and the entity description information is associated as the network asset information of the target site.

4. The method according to any one of claims 1 to 3, characterized in that: The multimedia information also includes file information, and the file information includes at least one document file; The method further comprises: For the at least one document file, the following operations are performed respectively: obtaining document description information of a document file, and extracting document content information of the document file to obtain corresponding document file information; The document file information corresponding to each of the at least one document file is associated with the network asset information of the target site.

5. The method according to any one of claims 1 to 3, characterized in that: The multimedia information further includes audio information, and the audio information includes at least one audio file; The method further comprises: For the at least one audio file, respectively perform the following operations: obtain audio description information of an audio file, and extract audio content information of the audio file to obtain corresponding audio file information; The audio file information corresponding to each of the at least one audio file is associated with the network asset information of the target site.

6. The method according to any one of claims 1 to 3, characterized in that: The multimedia information also includes video information, the video information includes at least one video file, each video file includes: a plurality of continuous image frames; The method further comprises: For the at least one video file, respectively perform the following operations: obtain video description information of a video file, and extract video content information of a head image frame and a tail image frame for the video file to obtain corresponding video file information; The video file information corresponding to each of the at least one video file is associated with the network asset information of the target site.

7. The method according to claim 6, characterized in that The one video file also includes: audio frames corresponding to the multiple image frames; The method further comprises: Using a speech recognition method, performing speech recognition processing on each audio frame in the video file to obtain a corresponding speech text representation; In the speech text representation, extracting entity description information of an associated entity of the target site, and associating the entity description information as network asset information of the target site; The one video file also includes: subtitle information corresponding to the multiple image frames; The method further comprises: Using subtitle extraction technology, performing subtitle extraction processing on each subtitle information in the video file to obtain corresponding subtitle text representation; In the subtitle text representation, entity description information of the associated entity of the target site is extracted, and the entity description information is associated as network asset information of the target site.

8. A network asset mapping device, characterized in that: The device comprises: A receiving unit, configured to receive at least one response message sent by a target site based on a network asset request; wherein the at least one response message includes: site service information of the target site and multimedia information carried by the target site, wherein the multimedia information includes at least text information and image information; an extraction unit, configured to extract target entity information of a target entity to which the target site belongs from the text information; and to extract external application information of an external application associated with the target site from the image information; The obtaining unit is used to associate the entity object information, the external application information and the site service information into the network asset information of the target site to obtain the corresponding target mapping result.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images