Webpage picture acquisition method and device, electronic equipment and storage medium

Abstract

The invention provides a webpage picture obtaining method and device, electronic equipment and a storage medium, and the method comprises the steps: receiving a picture conversion request which comprises a target website needing to be converted; calling a headless browser, and performing screen capture operation on a webpage corresponding to the target website to obtain target picture data; and storing the target picture data to a preset storage position, and returning the preset storage position to a sender of the picture conversion request. After the picture conversion request is received, the server calls the headless browser to perform the screenshot operation on the original webpage corresponding to the target website to obtain the target picture data, and since the screenshot operation is performed on the original webpage, the screenshot operation does not need to be performed on the basis of the conversion result after other conversion is performed on the original webpage, so that the user experience is improved. And the definition loss of the picture in the webpage is reduced, so that the definition of the obtained webpage picture is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer software technology, and in particular to a web page image acquisition method, device, electronic equipment and storage medium. Background Art

[0002] In actual applications, for advertising purposes, companies may need to display and promote internally produced images on other web pages. Specifically, the image producer will upload the produced image files to the company's internal network platform, and relevant personnel can obtain the images from the network platform and send them to the outside. In order to ensure the security of internal corporate data, companies usually do not provide original image files to other web pages. Therefore, it is usually necessary to obtain images from the network platform and send them to the outside.

[0003] In order to obtain images from a web page, in the related technology, the image conversion and printing are usually performed after the DOM (Document Object Model) elements are parsed based on the web page. Among them, DOM is a programming interface that allows people to create, change or delete elements from a web page. It converts HTML or XML documents into a tree structure composed of nodes, each node represents an HTML element, and the HTML element includes images, tables, links, etc. This allows developers to read, modify and delete documents through programming languages ​​​​(such as JavaScript), thereby realizing dynamic updates and interactive effects of web pages.

[0004] Specifically, the related technology is to serialize the DOM elements of the web page, and perform canvas (a web graphics drawing technology based on HTML5 that allows developers to create and manipulate image content on the web page through JavaScript) rendering conversion based on the serialized DOM elements. After the rendering is completed, the entire canvas element is converted into base64 encoding, and the base64 encoding is uploaded to the server for storage.

[0005] However, when converting web page DOM serialization to canvas, the web page needs to be preprocessed before obtaining the image, which is inefficient. The rendering of some images may fail, and the canvas rendering is uncontrollable. There are differences in the canvas rendering of different elements, resulting in unstable quality of the rendering results. In addition, when converting canvas to base64 format images, the clarity is uncontrollable and low in most cases. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a method, device, electronic device and storage medium for obtaining a web page image, so as to improve the clarity of the obtained web page image.

[0007] According to one aspect of the present invention, a method for obtaining a web page image is provided, which is applied to a server, and the method comprises:

[0008] Receiving an image conversion request, wherein the image conversion request includes a target website address to be converted;

[0009] Calling a headless browser to take a screenshot of the webpage corresponding to the target URL to obtain target image data;

[0010] The target image data is stored in a preset storage location, and the preset storage location is returned to the sender of the image conversion request.

[0011] In a possible embodiment, the method further includes:

[0012] Storing the target image data and the corresponding target URL in a shared cache;

[0013] The calling of the headless browser to take a screenshot of the webpage corresponding to the target URL to obtain the target image data also includes:

[0014] Querying the target website in the shared cache, and when the target website is found, obtaining the target image data corresponding to the target cache from the shared cache and returning the data to the sender of the image conversion request;

[0015] In the case where the target URL is not found, the step of calling the headless browser, taking a screenshot of the web page corresponding to the target URL, and obtaining the target image data is executed.

[0016] In a possible embodiment, the method further includes:

[0017] Determining whether the usage rate of the shared cache exceeds a preset usage rate threshold, and if the usage rate of the shared cache does not exceed the preset usage rate threshold, storing the target image data and the corresponding target website in the shared cache;

[0018] When the usage rate of the shared cache exceeds a preset usage rate threshold, a cache error message is thrown;

[0019] The shared cache is updated based on the query frequency of each target image data stored in the shared cache, and the target image data whose query frequency is lower than a preset query frequency threshold is eliminated.

[0020] In a possible embodiment, the method further includes:

[0021] Encapsulating the image conversion request into token data, wherein the token data includes the target website, information of the user who sends the image conversion request, and callback user information, wherein the callback user information includes a callback user identifier and contact information of the callback user;

[0022] The token data is stored in a preset message queue, and when the headless browser is in an idle state, the step of calling the headless browser, taking a screenshot of the web page corresponding to the target URL, and obtaining the target image data is executed.

[0023] In a possible embodiment, the method further includes:

[0024] When the image conversion request is received, multiple child processes are created in the Node server, wherein the child processes include a proxy process and a working process; the working process is used to receive the image conversion request and send the image conversion request to the proxy process, and the proxy process is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data; the target image data is stored in a preset storage location, and the preset storage location is returned to the working process, so that the working process sends the preset storage location to the sender of the image conversion request.

[0025] In a possible embodiment, the method further includes:

[0026] After the working process sends the image conversion request to the proxy process, execution prompt information is returned to the sender of the image conversion request.

[0027] According to another aspect of the present invention, a web page image acquisition device is provided, which is applied to a server, and the device includes:

[0028] A receiving module, used for receiving a picture conversion request, wherein the picture conversion request includes a target website address to be converted;

[0029] A calling module is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data;

[0030] The storage module is used to store the target image data in a preset storage location and return the preset storage location to the sender of the image conversion request.

[0031] In a possible embodiment, the device also includes a creation module, which is used to create multiple child processes in the Node server when the image conversion request is received, wherein the child processes include a proxy process and a working process; the working process is used to receive the image conversion request and send the image conversion request to the proxy process, and the proxy process is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data; the target image data is stored in a preset storage location, and the preset storage location is returned to the working process, so that the working process sends the preset storage location to the sender of the image conversion request.

[0032] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0033] Processor; and

[0034] Memory for storing programs,

[0035] Wherein, the program includes instructions, and when the instructions are executed by the processor, the processor executes any of the above-mentioned methods for obtaining web page images.

[0036] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute any of the above-mentioned methods for obtaining web page images.

[0037] One or more technical solutions provided in an embodiment of the present invention receive an image conversion request, wherein the image conversion request includes a target URL to be converted; call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data; store the target image data in a preset storage location, and return the preset storage location to the sender of the image conversion request.

[0038] By applying the embodiment of the present invention, after obtaining the image conversion request, the server calls the headless browser to perform a screenshot operation on the original web page corresponding to the target URL, so as to obtain the target image data. Since the screenshot operation is performed on the original web page, there is no need to perform other transformations on the original web page and then perform the screenshot operation based on the transformation result, which reduces the clarity loss of the image in the web page and thereby improves the clarity of the obtained web page image. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Further details, features and advantages of the invention are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0040] Figure 1 A schematic diagram of a flow chart of a method for obtaining web page images provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of another flow chart of a web page image acquisition method provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the structure of a web page image acquisition device provided by an embodiment of the present invention;

[0043] Figure 4 A block diagram of an exemplary electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0045] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0046] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0047] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0048] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0049] In order to improve the quality of web page image acquisition, the embodiments of the present invention provide a web page image acquisition method, device, electronic device and storage medium. The web page image acquisition method provided by the embodiments of the present invention can be applied to any electronic device with a web page image acquisition function, and the electronic device can be a server, a computer or a mobile terminal. In a possible embodiment, the web page image acquisition method can be applied to a Node server, wherein the Node server, also known as Node.js, is a JavaScript operating environment based on the Chrome V8 engine, mainly used to create a network server. The following describes the solution of the present invention with reference to the accompanying drawings:

[0050] Figure 1 A schematic flow chart of a web page image acquisition method provided by an embodiment of the present invention may include the following steps:

[0051] S101, receiving an image conversion request, wherein the image conversion request includes a target website to be converted;

[0052] S102, calling a headless browser to take a screenshot of the webpage corresponding to the target URL to obtain target image data;

[0053] S103: Store the target image data in a preset storage location, and return the preset storage location to the sender of the image conversion request.

[0054] By applying the embodiment of the present invention, after receiving the image conversion request, the server calls the headless browser to perform a screenshot operation on the original web page corresponding to the target URL, so as to obtain the target image data. Since the screenshot operation is performed on the original web page, there is no need to perform other transformations on the original web page and then perform the screenshot operation based on the transformation result, which reduces the clarity loss of the image in the web page and thereby improves the clarity of the acquired web page image.

[0055] The above S101-S103 are exemplarily described below:

[0056] The above-mentioned image conversion request can be sent by the user through a web page. As a possible implementation method, the web page image acquisition method provided in the embodiment of the present invention can be implemented by a plug-in. The plug-in can be inserted into each browser used by the user after obtaining the user's authorization. The browser can be set according to the actual application scenario. Exemplarily, the plug-in can display a function entrance in the browser, and the user can send an image conversion request through the function entrance while browsing the web page using the browser. The plug-in can also be inserted into a preset network platform, which can be a network platform for internal users of an enterprise to publish output images. The network platform can include multiple web pages, and each web page can include one or more images.

[0057] The above-mentioned image conversion request may include a user identifier and a target URL to be converted, and the above-mentioned user identifier may be a user device number, a user ID, etc. As a possible implementation, after the user clicks on the above-mentioned function entrance, the URL of the webpage currently browsed by the user may be defaulted as the target URL. As another possible implementation, after receiving the user's click, the function entrance may display a URL copy location, and the user may copy the URL to be converted to the URL copy location, and the URL copied by the user is used as the target URL to be converted.

[0058] In a possible embodiment, after receiving the image conversion request, the server can return execution prompt information to the sender of the image conversion request to remind the user that his request is being processed. Exemplarily, "executing instructions" can be displayed in the browser or network platform to inform the user of the execution stage of his request, so that the user can perceive the processing progress of the request and improve the user experience.

[0059] After obtaining the image conversion request, a headless browser can be called to process the image conversion request, wherein a headless browser refers to a browser that can run in a graphical interface, which accesses a web page by rendering the page in memory and then sending the result back to the user or program that requested it, without displaying the web page on the screen. The headless browser supports programming to control it to automatically perform various tasks. In this embodiment, the headless browser can be set as a headless browser for screenshots. Further, the headless browser can also be set to add a watermark to the screenshot after the screenshot is taken to further protect the security of the image data. The headless browser can be flexibly selected and configured by relevant personnel according to the actual application scenario. For example, the headless browser can be PhantomJS, Puppeteer, etc., and the present invention does not specifically limit this.

[0060] In a possible embodiment, the headless browser can parse the target URL to obtain the webpage corresponding to the target URL, and take a screenshot of the webpage to obtain the target image data. The target image data can be data in base64 format or other formats, which can be set according to the actual application scenario.

[0061] After obtaining the target image data, the target image data can be stored in a preset storage location, which is specifically a storage location in the cloud server and can be flexibly set for different images. After storing the target image data, the preset storage location can be returned to the user who sent the image conversion request. The user can access the target image based on the storage location and convert the target image into .jpg, .png and other formats according to their needs.

[0062] In a possible embodiment, after the target image data is obtained and stored in the cloud, the target image data can be stored in a shared cache. Exemplarily, the storage path of the target image data in the cloud and the target URL can be stored in the shared cache in correspondence, and the shared cache can be in the Node server. In a possible embodiment, the above method can also include:

[0063] Querying the target website in the shared cache, and when the target website is found, obtaining the target image data corresponding to the target cache from the shared cache and returning the data to the sender of the image conversion request;

[0064] In the case where the target URL is not found, the step of calling the headless browser, taking a screenshot of the web page corresponding to the target URL, and obtaining the target image data is executed.

[0065] In this way, through the pre-query stage, if the target URL that needs to be converted is found in the shared cache, the storage path can be directly returned to the user without converting the target URL again, reducing the data processing volume on the backend and thereby improving image processing efficiency.

[0066] In a possible embodiment, before the target URL and its corresponding storage address need to be stored in the shared cache, the usage rate of the shared cache can be queried. If the usage rate of the shared cache reaches a preset usage rate threshold, an error message can be thrown to remind that the content in the current shared cache has exceeded the capacity. If the usage rate of the shared cache does not exceed the preset usage rate threshold, the target image data and the corresponding target URL are stored in the shared cache.

[0067] In a possible embodiment, since the storage space in the memory is limited, the content stored in the shared cache can be updated regularly. As a possible implementation, the shared cache can be updated according to a preset update cycle, or it can be updated when new target image data is generated and stored in the shared cache.

[0068] In a possible embodiment, according to the query frequency of each URL in the shared cache, the URLs whose query frequency is lower than a preset frequency threshold and their corresponding storage addresses can be deleted from the shared cache, and the URLs whose generation time is earlier than a preset time threshold and their corresponding storage addresses can also be deleted from the shared cache.

[0069] In a possible embodiment, when storing the storage address information of the target image data, the storage address information with the same target URL can be deduplicated and merged, that is, multiple results are obtained for multiple identical target URLs, and only one result can be stored in the shared cache to minimize the space occupied by the shared cache.

[0070] In a possible embodiment, in order to avoid processing conflicts that may be caused by processing multiple image conversion requests simultaneously, the web page image acquisition method may further include the following steps:

[0071] S201, encapsulating the image conversion request into token data, wherein the token data includes the target website, information of the user who sends the image conversion request, and callback user information, wherein the callback user information includes a callback user identifier and contact information of the callback user;

[0072] S202, storing the token data in a preset message queue, and when the headless browser is in an idle state, executing the step of calling the headless browser, taking a screenshot of the web page corresponding to the target URL, and obtaining target image data.

[0073] After obtaining the image conversion request, the image conversion request can be encapsulated into token data, which may include the target address included in the above image conversion request, the user information sending the image conversion request, and the callback user information of the target image data storage location to be sent later. The callback user information may be customized by the user sending the image conversion request, or may be preset in the server, and the callback user information may include the contact information of the callback user, including email, instant messaging ID, etc.

[0074] Afterwards, each token data can be stored in a message queue, which is used to store pending image conversion requests, and the requests in the message queue are consumed in a first-in-first-out manner. In a possible embodiment, the server can query the current working status of the headless browser. When the headless browser is in operation, the requests in the message queue are not consumed. When the headless browser is in an idle state, the token data in the message queue can be obtained for consumption. Specifically, the token data can be obtained, the target URL contained in the token data can be extracted, the web page corresponding to the target URL can be screenshot, the target image can be obtained, and after the target image is stored, its storage address is sent to the callback user based on the contact information of the callback user.

[0075] In a possible embodiment, when the headless browser is in an idle state, the server can obtain token data from the message queue for consumption according to a preset time period.

[0076] In a possible embodiment, the web page image acquisition method can be implemented by a preset work process and an agent process, wherein the agent process is used to monitor and manage the processing of requests and their processing status, and the work process is used to receive image conversion requests. As a possible implementation method, when the image conversion request is received, multiple sub-processes can be created in the Node server, wherein the sub-processes include an agent process and a work process; the work process is used to receive the image conversion request and send the image conversion request to the agent process, and the agent process is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain the target image data; the target image data is stored in a preset storage location, and the preset storage location is returned to the work process, so that the work process sends the preset storage location to the sender of the image conversion request.

[0077] As a possible implementation, the worker process and the agent process can be generated by the following steps:

[0078] S301. Based on the number of CPU cores of the server, multiple subprocesses are created, and each subprocess can occupy a portion of the number of CPU cores.

[0079] The above server refers to a Node server. In a possible embodiment, a subprocess can be created when a user accesses a web page. The number of the above subprocesses can be flexibly defined according to the actual application scenario, and the CPU resources occupied by each subprocess can also be set at will, as long as the number of subprocesses is not less than 2. For example, if the number of server cores is 8, 8 subprocesses can be created, and each subprocess occupies 1 CPU core.

[0080] S302: Determine an agent process and a worker process from each sub-process.

[0081] In a possible embodiment, the subprocess that occupies the most resources can be determined as the agent process, and the other subprocesses can be determined as the worker process. If the CPU resources occupied by each subprocess are the same, a subprocess can be randomly selected as the agent process, and the remaining processes can be selected as the worker process. The worker process is used to receive the image conversion request. By setting multiple worker processes, the reliability of receiving the image conversion request can be improved.

[0082] like Figure 2 As shown, Figure 2A flow chart of a web page image acquisition method provided by an embodiment of the present invention is provided. The web page image acquisition method is implemented by an agent process and a work process created in a Node server. Specifically, when a user browser access or a user request is detected, a scheduling process is created in the Node server. The scheduling process includes multiple sub-processes, specifically an agent process and multiple work processes. The agent process and the work process can acquire images through the following steps:

[0083] The work process receives the image conversion request sent by the user, which is issued by the user in the browser.

[0084] Each worker process sends the received image conversion request to the agent process, and the agent process performs subsequent processing of the image conversion request.

[0085] In a possible embodiment, after the worker process sends the image conversion request to the agent request, it can return a request processing indicator to the user to remind the user that the image conversion request is being processed, so that the user can understand the image conversion process and improve the user experience.

[0086] The agent process checks whether there is an image corresponding to the target URL that has been converted in the shared cache. If so, the image is sent to the worker process, which then sends the image to the user through email notification or other means. If not, execute step 4.

[0087] The agent process encapsulates the image conversion requests sent by each worker process into token data and pushes it to the message queue. The message request queue is used to store pending image conversion requests. The token data includes the target URL, sent user messages, and callback user messages.

[0088] The agent process determines whether the token conversion operation is currently being performed. If not, it executes step 6. If so, it waits.

[0089] The agent process periodically obtains a token in the idle state, calls a headless browser to take a screenshot, obtains the converted data stream, and saves the data stream to the cloud service.

[0090] The agent process pushes the address generated by the cloud service to the worker process, writes it into the shared memory, selects cached data that can be eliminated, and removes overloaded data.

[0091] The work process pushes messages to the user by sending emails.

[0092] By applying the embodiment of the present invention, a headless browser is used on the server side to render the converted web page, and a screenshot of the web page is saved as an image without performing other pre-processing on the web page, thereby ensuring the clarity of the acquired image, avoiding rendering differences and image rendering failures in the web page.

[0093] Furthermore, a queue is used on the server side to encapsulate the conversion capability into a token form. The user sends a conversion command, pushes it into the queue, obtains the conversion capability of the token, writes it to the local cache after printing, and pushes it to the user's mailbox, thus realizing asynchronous processing of requests and achieving peak shaving effect, avoiding the impact of traffic peaks on server operation and ensuring server stability.

[0094] In addition, in the embodiment of the present invention, a sub-process is dynamically created for the Node server, and the sub-process is divided into an agent process and a work process. The agent process is responsible for the conversion capability in the token and the cache management capability (automatic elimination, etc.), and the work process is responsible for collecting user requests and pushing them into the queue. After the agent process prints, it sends a completion instruction to the work process, and the work process is responsible for pushing messages to the user. The centralized and convergent way of converting images improves the stability and high availability of the backend, improves the overall conversion efficiency, and reduces the anxiety of users waiting.

[0095] Based on the same inventive concept, the embodiment of the present invention also provides a web page image acquisition device, such as Figure 3 As shown, the device 300 may include:

[0096] The receiving module 301 is used to receive a picture conversion request, wherein the picture conversion request includes a target website to be converted;

[0097] The calling module 302 is used to call a headless browser to take a screenshot of the webpage corresponding to the target URL to obtain target image data;

[0098] The storage module 303 is used to store the target image data in a preset storage location, and return the preset storage location to the sender of the image conversion request.

[0099] In a possible embodiment, the device also includes a creation module, which is used to create multiple child processes in the Node server when the image conversion request is received, wherein the child processes include a proxy process and a working process; the working process is used to receive the image conversion request and send the image conversion request to the proxy process, and the proxy process is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data; the target image data is stored in a preset storage location, and the preset storage location is returned to the working process, so that the working process sends the preset storage location to the sender of the image conversion request.

[0100] Among them, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0101] The exemplary embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform a method according to an embodiment of the present invention when executed by the at least one processor.

[0102] Exemplary embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0103] An exemplary embodiment of the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform a method according to an embodiment of the present invention.

[0104] refer to Figure 4 , a block diagram of an electronic device 400 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0105] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0106] A plurality of components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 may be any type of device capable of inputting information to the electronic device 400, and the input unit 406 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 407 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 may include, but is not limited to, a disk, an optical disk. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0107] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned web page image acquisition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via the ROM 402 and / or the communication unit 409. In some embodiments, the computing unit 401 may be configured to perform the above-mentioned web page image acquisition method by any other appropriate means (e.g., by means of firmware).

[0108] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0109] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0111] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0112] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0113] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

Claims

1. A web page image acquisition method, characterized in that: Applied to the server, the method includes: Receiving an image conversion request, wherein the image conversion request includes a target website address to be converted; Calling a headless browser to take a screenshot of the webpage corresponding to the target URL to obtain target image data; The target image data is stored in a preset storage location, and the preset storage location is returned to the sender of the image conversion request.

2. The method according to claim 1, characterized in that The method further comprises: Storing the target image data and the corresponding target URL in a shared cache; The calling of the headless browser to take a screenshot of the webpage corresponding to the target URL to obtain the target image data also includes: Querying the target website in the shared cache, and when the target website is found, obtaining the target image data corresponding to the target cache from the shared cache and returning the data to the sender of the image conversion request; In the case where the target URL is not found, the step of calling the headless browser, taking a screenshot of the web page corresponding to the target URL, and obtaining the target image data is executed.

3. The method according to claim 2, characterized in that The method further comprises: Determining whether the usage rate of the shared cache exceeds a preset usage rate threshold, and if the usage rate of the shared cache does not exceed the preset usage rate threshold, storing the target image data and the corresponding target website in the shared cache; When the usage rate of the shared cache exceeds a preset usage rate threshold, a cache error message is thrown; The shared cache is updated based on the query frequency of each target image data stored in the shared cache, and the target image data whose query frequency is lower than a preset query frequency threshold is eliminated.

4. The method according to claim 1, characterized in that The method further comprises: Encapsulating the image conversion request into token data, wherein the token data includes the target website, information of the user who sends the image conversion request, and callback user information, wherein the callback user information includes a callback user identifier and contact information of the callback user; The token data is stored in a preset message queue, and when the headless browser is in an idle state, the step of calling the headless browser, taking a screenshot of the web page corresponding to the target URL, and obtaining the target image data is executed.

5. The method according to claim 1, characterized in that: The method further comprises: When the image conversion request is received, multiple child processes are created in the Node server, wherein the child processes include a proxy process and a working process; the working process is used to receive the image conversion request and send the image conversion request to the proxy process, and the proxy process is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data; the target image data is stored in a preset storage location, and the preset storage location is returned to the working process, so that the working process sends the preset storage location to the sender of the image conversion request.

6. The method according to claim 5, characterized in that The method further comprises: After the working process sends the image conversion request to the proxy process, execution prompt information is returned to the sender of the image conversion request.

7. A web page image acquisition device, characterized in that: Applied to the server, the device comprises: A receiving module, used for receiving a picture conversion request, wherein the picture conversion request includes a target website address to be converted; A calling module is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data; The storage module is used to store the target image data in a preset storage location and return the preset storage location to the sender of the image conversion request.

8. The device according to claim 7, characterized in that The device also includes a creation module, which is used to create multiple child processes in the Node server when receiving the image conversion request, wherein the child processes include a proxy process and a working process; the working process is used to receive the image conversion request and send the image conversion request to the proxy process, and the proxy process is used to call a headless browser to take a screenshot of the web page corresponding to the target URL to obtain target image data; the target image data is stored in a preset storage location, and the preset storage location is returned to the working process, so that the working process sends the preset storage location to the sender of the image conversion request.

9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-6.

Images