Data acquisition method and device, electronic equipment and storage medium

Abstract

The application provides a data acquisition method and device, electronic equipment and a storage medium, which can be used in the field of big data. The method comprises the following steps: sending a data acquisition request to a first storage device, wherein the data acquisition request is used for the first storage device to feed back data; when complete target data fed back by the first storage device is not acquired within a preset time length, acquiring a configuration file associated with the first storage device and parameter information of the target data; determining a second storage device and a third storage device according to the configuration file; determining a target storage device in the second storage device and the third storage device according to the parameter information; and sending a data acquisition request to the target storage device. In the application, when data cannot be acquired from the first storage device, a data acquisition request is sent to a second storage device in the same region or a third storage device in different regions which can acquire data within a preset time length, so that the data acquisition time length is ensured to be not too long, and the data acquisition efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of big data, and more particularly to a data acquisition method, apparatus, electronic device, and storage medium. Background Technology

[0002] When a client needs to retrieve data, it can obtain the data from storage devices such as databases.

[0003] When a client requests data from a storage device, if the storage device is experiencing network issues or malfunctions, it will be unable to send complete data back to the client. This will result in the client being unable to obtain the data for an extended period, leading to low data retrieval efficiency and impacting the client's operation. Summary of the Invention

[0004] This application provides a data acquisition method, apparatus, electronic device, and storage medium to solve the problem of low data acquisition efficiency.

[0005] Firstly, this application provides a data acquisition method, including:

[0006] Send a data acquisition request to the first storage device, the data acquisition request being used by the first storage device to return data;

[0007] If the complete target data is not received from the first storage device within a preset time period, the configuration file associated with the first storage device and the parameter information of the target data are obtained. The parameter information includes at least one of the following: the download progress of the target data, the cumulative download time, and the data volume.

[0008] The second storage device and the third storage device are determined according to the configuration file. The second storage device is located in the same area as the first storage device, and the third storage device is located in a different area from the first storage device.

[0009] Based on the parameter information, a target storage device is determined among the second storage device and the third storage device. The target storage device is either the second storage device or the third storage device that returns complete target data within the preset time period.

[0010] Send a data acquisition request to the target storage device to obtain the target data fed back by the target storage device.

[0011] Secondly, this application provides a data acquisition device, comprising:

[0012] A first sending module is configured to send a data acquisition request to a first storage device, wherein the data acquisition request is used by the first storage device to provide feedback data.

[0013] The acquisition module is used to acquire the configuration file associated with the first storage device and the parameter information of the target data when the complete target data fed back by the first storage device is not acquired within a preset time period. The parameter information includes at least one of the following: the download progress of the target data, the cumulative download time, and the data volume.

[0014] A first determining module is configured to determine a second storage device and a third storage device according to the configuration file, wherein the second storage device is located in the same area as the first storage device, and the third storage device is located in a different area from the first storage device;

[0015] The second determining module is used to determine a target storage device among the second storage device and the third storage device based on the parameter information. The target storage device is either the second storage device or the third storage device that returns complete target data within the preset time period.

[0016] The second sending module is used to send a data acquisition request to the target storage device to obtain the target data fed back by the target storage device.

[0017] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0018] The memory stores computer-executed instructions;

[0019] The processor executes computer execution instructions stored in the memory to implement the method described above.

[0020] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described above.

[0021] The data acquisition method, apparatus, electronic device, and storage medium provided in this application send a data acquisition request to a first storage device to obtain target data. If complete target data is not obtained from the first storage device within a preset time period, the application acquires the configuration file associated with the first storage device, as well as parameter information such as the download progress, cumulative download time, and data volume of the target parameters. Based on the configuration file, a second and third storage device are determined. Based on the parameter information, a target storage device is determined from the second and third storage devices, and a data acquisition request is then sent to the target storage device to obtain the target data. In this application, when data cannot be obtained from the first storage device, a data acquisition request is sent to a second storage device in the same region or a third storage device in a different region that can obtain data within the preset time period, ensuring that the data acquisition time is not too long and improving data acquisition efficiency. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] Figure 1 This is a schematic diagram illustrating a scenario involving the data acquisition method described in this application.

[0024] Figure 2 A flowchart illustrating the first embodiment of the data acquisition method provided in this application;

[0025] Figure 3 A flowchart illustrating a second embodiment of the data acquisition method provided in this application;

[0026] Figure 4 A flowchart illustrating a third embodiment of the data acquisition method provided in this application;

[0027] Figure 5 A flowchart illustrating the fourth embodiment of the data acquisition method provided in this application;

[0028] Figure 6 This is a schematic diagram of the data acquisition device of this application;

[0029] Figure 7 This is a schematic diagram of the structure of the electronic device of this application.

[0030] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0033] It should be noted that the data acquisition method, apparatus, electronic device and storage medium of this application can be used in the field of big data, or in any field other than big data. The application fields of the data acquisition method, apparatus, electronic device and storage medium of this application are not limited.

[0034] When a client needs to retrieve data, it can obtain the data from storage devices such as databases.

[0035] The inventors of this application have discovered that when a client requests data from a storage device, if the storage device is connected to a network or malfunctions, the storage device cannot send complete data back to the client, resulting in the client being unable to obtain data for an extended period of time. This leads to low data retrieval efficiency and affects the operation of the client.

[0036] The inventors of this application therefore conceived of sending a data acquisition request to a second storage device in the same area or a third storage device in a different area that can acquire data within a preset time period when data cannot be obtained from the first storage device. This ensures that the data acquisition time will not be too long and improves the data acquisition efficiency.

[0037] Reference Figure 1 , Figure 1This is a schematic diagram illustrating the data acquisition method of this application. The first storage device 100 and the second storage device 200 are located in the same area, while the second storage device 100 and the third storage device 300 are located in different areas. That is, the first storage device 100 and the second storage device 200 are in the same location, while the first storage device 100 and the third storage device 300 are in different locations. There can be multiple third storage devices 300 in different locations. The first storage device 100, the second storage device 200, and the third storage device 300 can be a data center or a database, and they all store the same data. This means the data needs to be synchronized across all storage devices. The first storage device 100 serves as the primary and backup storage device, and all clients obtain data from the first storage device 100 by default. The second storage device 200 serves as the backup storage device for the first storage device 100. The data acquisition device 400 (client) sends a data acquisition request to the first storage device 100 to download data from the first storage device 100. If the data acquisition device 400 fails to acquire complete target data from the first storage device 100 within a preset time period, the data acquisition device 400 acquires the configuration file associated with the first storage device 100 and parameter information of the target data. The parameter information includes at least one of the following: download progress, cumulative download time, and data volume. The data acquisition device determines the second storage device 200 and the third storage device 300 based on the configuration file, that is, it determines the access paths of the two storage devices to facilitate data acquisition. The data processing device 400 determines the target storage device among the second storage device 200 and the third storage device 300 based on the parameter information. For example, if the target storage device is the third storage device 300, the data processing device 100 sends a data acquisition request to the target storage device 300 to acquire the target data from the target storage device within a preset time period.

[0038] The above illustrations of a data acquisition method using a data acquisition device are for general purposes. However, the execution entity of the method in this application is not limited to a data acquisition device; it can be any electronic device with data acquisition capabilities, or a data processing system. It should be noted that the data processing system can be deployed independently on an electronic device in any environment (e.g., independently on an edge server in an edge environment), or entirely in a cloud environment, or distributed across different environments.

[0039] For example, a data processing system can be logically divided into multiple parts, each with different functions. These parts can be deployed in any two or three of the following environments: electronic devices (located on the user side), edge environments, and cloud environments. An edge environment is defined as a collection of edge electronic devices located close to the electronic devices, including edge servers and edge stations with computing power. The various parts of the data processing system deployed in different environments or devices work together to achieve the system's functions.

[0040] It should be understood that this application does not restrict the specific deployment environment of which parts of the data processing system are deployed. In actual application, the deployment can be adapted according to the computing power of electronic devices, the resource availability of edge and cloud environments, or specific application requirements.

[0041] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0042] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the data acquisition method of this application. The data acquisition method includes the following steps:

[0043] Step S201: Send a data acquisition request to the first storage device. The data acquisition request is used by the first storage device to provide feedback data.

[0044] In this embodiment, the executing entity is a data processing device. For ease of description, the term "device" will be used to refer to the data processing device below. The device can be a terminal device that loads the application or a client.

[0045] The device sends a data retrieval request to a first storage device to retrieve data corresponding to the data retrieval request, which is defined as target data. For example, an application in the device sends a data retrieval request to a corresponding server, which retrieves the target data from the first storage device.

[0046] The server hosts a site on ANS (Network Attached Storage). This site can be an FTP (File Transfer Protocol) or SFTP (SSH File Transfer Protocol) site, allowing devices to retrieve data from the storage device containing the ANS disk. The storage device has a configuration file containing local and remote working paths. The remote working path can be the access path of a storage device located in the same region as the storage device, or it can be the access path of a storage device located in a different region. Furthermore, permissions are configured for the path to ensure that registered devices and users have permission to create subfolders and files under that path to retrieve data from the storage device. The configuration file also specifies the registered device's IP address, username, password, retry interval, and number of retries. The retry interval is the time interval after a failed data retrieval attempt before the device can request data again. The number of retries is the maximum number of times the device can fail to retrieve data from the storage device.

[0047] The server is equipped with multiple NAS drives, each corresponding to a storage device. The first and second storage devices are located in the same area, while the first and third storage devices are located in different areas. The data stored in each storage device is identical. For example, the first storage device serves as a hot standby device, and the device retrieves data from the first storage device by default. The second storage device serves as a cold standby device, and is not operational when the hot standby data is initiated. Of course, the first and second storage devices can also operate simultaneously.

[0048] Step S202: If the complete target data fed back by the first storage device is not obtained within the preset time period, the configuration file associated with the first storage device and the parameter information of the target data are obtained. The parameter information includes at least one of the following: the download progress of the target data, the cumulative download time, and the data volume.

[0049] After sending a data acquisition request, the device will start a timer to obtain the time duration. When the time duration reaches the preset duration, the device will detect whether the first storage device has returned complete target data, that is, whether the device has completely downloaded the target data from the first storage device.

[0050] If the device fails to obtain complete target data from the first storage device within the preset time period, the device obtains the configuration file associated with the first storage device and the parameter information of the target data. The configuration file is stored in the device, and the parameter information includes at least one of the following: the download progress of the target data, the cumulative download time, and the data volume.

[0051] Furthermore, due to network issues, if the first storage device fails to send data to the device, the device will resend the data retrieval request to the first storage device, meaning the device can send multiple data retrieval requests to the first storage device. Based on download record information, the device can determine the number of times the target data has been downloaded from the first storage device, i.e., the number of retries for downloading the target data. When the preset number of downloads is reached, the device will stop sending data retrieval requests to the first storage device and instead obtain the configuration file associated with the first storage device and the parameter information of the target data.

[0052] Step S203: Determine the second storage device and the third storage device according to the configuration file. The second storage device is in the same area as the first storage device, and the third storage device is in a different area from the first storage device.

[0053] The configuration file stores remote access paths. Based on these paths, the device can determine the second and third storage devices. The second storage device and the first storage device are located in the same region (i.e., co-located); the first storage device and the third storage device are located in different regions (i.e., the third storage device is located in a different region). For example, the device obtains the local storage path and the remote storage path based on the configuration file, identifies the storage device corresponding to the local storage path as the second storage device, and identifies the storage device corresponding to the abnormal storage path as the third storage device.

[0054] Step S204: Based on the parameter information, determine the target storage device among the second storage device and the third storage device. The target storage device is either the second storage device or the third storage device that returns complete target data within a preset time period.

[0055] The device determines the target storage device through parameter information in the third storage device and the second storage device. The target storage device is either the second storage device or the third storage device that can return complete target data within a preset time period.

[0056] For example, the second storage device is located in the same area as the first storage device, while the third storage device is located in a different area. If the first storage device fails, retrieving the target data directly from the second storage device in the same area might be faster than using the third storage device in a different location. However, the second storage device acts as a cold backup; it is not activated while the first storage device is running, and waking it up requires a certain amount of time. Furthermore, the target data might be partially downloaded, with the remainder undownloaded, and the first storage device might be experiencing network issues. If the amount of undownloaded data is small, waking up the second storage device is unnecessary. Therefore, the device determines the amount of undownloaded target data based on parameter information. For example, the amount of undownloaded data can be determined based on the download progress and the total amount of target data. If the amount of downloaded data is lower than a preset amount, the third storage device is designated as the target storage device; if the amount of downloaded data is higher than the preset amount, the second storage device is designated as the target storage device.

[0057] Step S205: Send a data acquisition request to the target storage device to obtain the target data returned by the target storage device.

[0058] After identifying the target storage device, the device sends a data acquisition request to the target storage device, thereby enabling the device to acquire the target data within a preset time period.

[0059] In this embodiment, a data acquisition request is sent to the first storage device to obtain the target data. If the complete target data is not obtained from the first storage device within a preset time period, the download progress, cumulative download time, and data volume of the configuration file associated with the first storage device and the target parameters are obtained. Based on the configuration file, a second and third storage device are determined. Based on the parameter information, the target storage device is determined from the second and third storage devices, and then a data acquisition request is sent to the target storage device to obtain the target data. In this embodiment, if data cannot be obtained from the first storage device, a data acquisition request is sent to the second storage device in the same area or the third storage device in a different area that can obtain data within the preset time period. This ensures that the data acquisition time is not too long and improves the data acquisition efficiency.

[0060] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the data acquisition method of this application. Based on the first embodiment, step S204 includes:

[0061] Step S301: Obtain the first operating status information of the second storage device and the second operating status information of the third storage device.

[0062] In this embodiment, after determining the second and third storage devices, the device acquires the first operating status information of the second storage device and the second operating status information of the third storage device. For example, when downloading target data, the first storage device stores the first and second operating status information of the second and third storage devices in a configuration file, and the device directly extracts the first and second operating status information from the configuration file. The first operating status information indicates the operating status of the second storage device; for example, it indicates whether the second storage device is in a startup state, its resource utilization rate when in a startup state, and its data processing speed. The second operating status information indicates the operating status of the third storage device.

[0063] Step S302: Determine the first processing speed of the second storage device based on the first operating status information, and determine the second processing speed of the third storage device based on the second operating information.

[0064] After obtaining the first operating status information, the device determines the first processing speed of the second storage device based on the first operating status information. The first processing speed refers to the speed at which the second storage device processes data. The device then determines the second processing speed of the third storage device using the second operating status information. The second processing speed is used to indicate the speed at which the third storage device processes data.

[0065] For example, the first operating status information includes the idle resources of the second storage device, and the first processing speed of the second storage device can be calculated based on these resources. Similarly, the second processing speed can also be determined based on the idle resources of the third storage device included in the second operating status information.

[0066] Step S303: Determine the required download duration of the target data based on the parameter information.

[0067] The device determines the required download duration for the target data based on parameter information. The required download duration refers to the maximum time the device requires to acquire the target data. In one example, the device obtains the cumulative download duration from the parameter information, while the device sets a maximum download duration for the target data. The difference between the maximum duration and the cumulative download duration can be used as the required download duration.

[0068] Step S304: Determine the target processing speed that matches the required download duration from the first processing speed and the second processing speed.

[0069] The device determines a first processing speed or a second processing speed that matches the required download duration, and uses this as the target processing speed.

[0070] For example, the device divides the first processing speed by the required download time to obtain a first data volume, and divides the second processing speed by the required download time to obtain a second data volume. The first data volume is the data transmission volume of the second storage device within the required download time, and the second data volume is the data transmission volume of the third storage device within the required download time. The first or second data volume, which is greater than the target data volume, is taken as the target data volume, and the processing speed corresponding to the target data volume is the target processing speed.

[0071] Step S305: Determine the second or third storage device corresponding to the target processing speed as the target storage device.

[0072] After determining the target processing speed, the second or third storage device corresponding to the target processing speed is determined as the target storage device. That is, the target storage device is capable of feeding back the target data to the device within the required download time, and the required download time is less than the preset time.

[0073] In this embodiment, the device accurately determines the target storage device based on the operating status information of the second storage device and the operating status information of the third storage device, enabling the device to obtain data from the target storage device.

[0074] Reference Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the data acquisition method of this application. Based on the second embodiment, step S303 includes:

[0075] Step S401: Based on the download record information, determine the number of downloads required to download the target data from the first storage device.

[0076] In this embodiment, due to network issues, the first storage device fails to report data failure to the device. The device then resends the data retrieval request to the first storage device, meaning the device can send multiple data retrieval requests to the first storage device. The device can determine the number of times the target data was downloaded from the first storage device based on download record information.

[0077] Step S402: Determine the priority parameter of the target data based on the parameter information. The priority parameter is used to indicate the urgency of downloading the target data.

[0078] The device determines the priority parameter of the target data based on the parameter information. The priority parameter is used to indicate the urgency of downloading the target data.

[0079] Step S403: Based on the priority parameter and the number of downloads, the preset duration is adjusted to obtain the required download duration.

[0080] The device adjusts the preset duration based on priority parameters and the number of downloads to obtain the required download duration.

[0081] In one example, a higher priority parameter indicates a higher urgency of the download, and thus a greater reduction in the preset duration due to the priority parameter. Additionally, the number of downloads determines a coefficient; the more downloads, the smaller the coefficient, with a coefficient less than 1. The device determines a correction value based on the priority parameter. Multiplying this correction value by the coefficient determined by the number of downloads yields the final correction value. Subtracting the final correction value from the preset duration gives the required download duration.

[0082] In another example, the device determines a correction value based on a priority parameter and the number of downloads, with these parameters being positively correlated. The device can obtain the correction value based on the mapping relationship between the priority parameter, the number of downloads, and the correction value. The device then determines the target duration based on the correction value, with a larger correction value resulting in a larger target duration; that is, the target duration and the correction value are positively correlated. The device can determine the required download duration based on the difference between the preset duration and the target duration.

[0083] In this embodiment, the device accurately corrects the preset duration based on the number of downloads of the target data and parameter information to obtain the required download duration.

[0084] Reference Figure 5 , Figure 5 This is the fourth embodiment of the data acquisition method of this application. Based on the third embodiment, step S402 includes:

[0085] Step S501: Determine the weight and score of each reference factor based on the corresponding value of the reference factor. The reference factors are download progress, cumulative download time, and data volume.

[0086] In this embodiment, the parameter information includes download progress, cumulative download duration, and data volume. The device uses download progress, cumulative download duration, and data volume as reference factors; that is, the reference factors include download progress, cumulative download duration, and data volume.

[0087] The device determines the weight and score of each reference factor based on its corresponding numerical value. For example, the numerical value of the reference factor can be converted into a score, thus standardizing the measurement of each reference factor. Each reference factor has a corresponding preset score, and the device determines the difference between the score of each reference factor and the preset score; the larger the difference, the greater the corresponding weight.

[0088] Step S502: Determine the target value based on the score and weight of each reference factor.

[0089] After determining the score and weight of each reference factor, the target value can be calculated by weighting each score and its corresponding weight.

[0090] Step S503: Determine the priority parameters of the target data based on the target value.

[0091] The device can determine the priority parameter of the target data based on the target value. For example, the target value can be used as the priority parameter; or the priority level of the target value can be determined and used as the priority parameter.

[0092] In this embodiment, the device determines the weight and score of each reference factor based on the value corresponding to the reference factor. The target value can be determined by the score and weight, thereby accurately determining the priority parameter of the target data through the target value.

[0093] This application also provides a data acquisition device, referring to... Figure 6 The data acquisition device 600 includes:

[0094] The first sending module 610 is used to send a data acquisition request to the first storage device, and the data acquisition request is used for the first storage device to feed back data.

[0095] The acquisition module 620 is used to acquire the configuration file associated with the first storage device and the parameter information of the target data when the complete target data is not received from the first storage device within a preset time period. The parameter information includes at least one of the following: the download progress of the target data, the cumulative download time, and the data volume.

[0096] The first determining module 630 is used to determine the second storage device and the third storage device according to the configuration file. The second storage device is in the same area as the first storage device, and the third storage device is in a different area from the first storage device.

[0097] The second determining module 640 is used to determine the target storage device in the second storage device and the third storage device according to the parameter information. The target storage device is the second storage device or the third storage device that returns complete target data within a preset time period.

[0098] The second sending module 650 is used to send a data acquisition request to the target storage device in order to obtain the target data fed back by the target storage device.

[0099] In one embodiment, the second determining module 640 includes:

[0100] The first acquisition unit is used to acquire the first operating status information of the second storage device and the second operating status information of the third storage device.

[0101] The first determining unit is configured to determine the first processing speed of the second storage device based on the first operating status information, and to determine the second processing speed of the third storage device based on the second operating information.

[0102] The second determining unit is used to determine the required download duration of the target data based on the parameter information;

[0103] The third determining unit is used to determine the target processing speed that matches the download requirement duration between the first processing speed and the second processing speed.

[0104] The fourth determining unit is used to determine the second or third storage device corresponding to the target processing speed as the target storage device.

[0105] In one embodiment, the second determining unit includes:

[0106] The first determining subunit is used to determine the number of times the target data is downloaded from the first storage device based on the download record information;

[0107] The second determining subunit is used to determine the priority parameter of the target data based on the parameter information. The priority parameter is used to indicate the urgency of downloading the target data.

[0108] The correction subunit is used to correct the preset duration based on the priority parameter and the number of downloads to obtain the required download duration.

[0109] In one embodiment, the second determining subunit includes:

[0110] The first module is used to determine the weight and score of each reference factor based on the value of the reference factor. The reference factors are download progress, cumulative download time and data volume.

[0111] The second determination module is used to determine the target value based on the score and weight of each reference factor.

[0112] The third module is used to determine the priority parameters of the target data based on the target value.

[0113] In one embodiment, the third determining unit includes:

[0114] The fourth module determines the correction value based on the priority parameter and the number of downloads. The priority parameter, the number of downloads, and the correction value are positively correlated.

[0115] The fifth module determines the target duration based on the correction value, and the target duration is positively correlated with the correction value.

[0116] The sixth module is used to determine the difference between the preset duration and the target duration as the required download duration.

[0117] In one embodiment, the first determining module 630 includes:

[0118] The second acquisition unit is used to obtain the local storage path and the off-site storage path according to the configuration file.

[0119] The fifth determining unit is used to determine the storage device corresponding to the local storage path as the second storage device and the storage device corresponding to the remote storage path as the third storage device.

[0120] In one embodiment, the acquisition module 620 includes:

[0121] The sixth determining unit is used to determine the number of times the target data is downloaded from the first storage device based on the download record information;

[0122] The third acquisition unit is used to acquire the configuration file and target data parameter information associated with the first storage device when the number of downloads reaches a preset number.

[0123] Figure 7 This is a hardware structure diagram of an electronic device according to an exemplary embodiment.

[0124] Electronic device 700 may include: a processor 71, such as a CPU, a memory 72, and a transceiver 73. Those skilled in the art will understand that... Figure 7 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. Memory 72 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0125] The processor 71 can call computer programs stored in the memory 72 or execute computer instructions to complete all or part of the steps of the above data acquisition method.

[0126] Transceiver 73 is used to receive information sent by external devices and to send information to external devices.

[0127] An electronic device includes: a processor, and a memory communicatively connected to the processor;

[0128] The memory stores the instructions that the computer executes;

[0129] The processor executes computer execution instructions stored in memory to implement the data acquisition method as described in any of the above embodiments.

[0130] A non-transitory computer-readable storage medium, wherein instructions (computer-executable instructions) in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the aforementioned data acquisition method.

[0131] A computer program product includes a computer program that, when executed by a processor of an electronic device, enables the electronic device to perform the aforementioned data acquisition method.

[0132] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0133] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data acquisition method, characterized in that, include: Send a data acquisition request to the first storage device, the data acquisition request being used by the first storage device to return data; If the complete target data is not received from the first storage device within a preset time period, the configuration file associated with the first storage device and the parameter information of the target data are obtained. The parameter information includes at least one of the following: the download progress of the target data, the cumulative download time, and the data volume. The second storage device and the third storage device are determined according to the configuration file. The second storage device is located in the same area as the first storage device, and the third storage device is located in a different area from the first storage device. The download time requirement for the target data is determined based on the parameter information, and a target storage device matching the download time requirement is determined among the second storage device and the third storage device. The target storage device is either the second storage device or the third storage device that returns complete target data within the preset time. Send a data acquisition request to the target storage device to obtain the target data returned by the target storage device; The step of determining the download duration of the target data based on the parameter information includes: Based on the download record information, determine the number of times the target data was downloaded from the first storage device; The priority parameter of the target data is determined based on the parameter information, and the priority parameter is used to indicate the urgency of downloading the target data. The required download duration is obtained by adjusting the preset duration based on the priority parameter and the number of downloads.

2. The data acquisition method according to claim 1, characterized in that, In the second storage device and the third storage device, the step of determining the target storage device that matches the download requirement duration includes: Obtain the first operating status information of the second storage device and the second operating status information of the third storage device; The first processing speed of the second storage device is determined based on the first operating status information, and the second processing speed of the third storage device is determined based on the second operating information. Among the first processing speed and the second processing speed, a target processing speed that matches the required download duration is determined; The second or third storage device corresponding to the target processing speed is determined as the target storage device.

3. The data acquisition method according to claim 1, characterized in that, The step of determining the priority parameter of the target data based on the parameter information includes: Based on the numerical values ​​of the reference factors, the weight and score of each reference factor are determined, wherein the reference factors are the download progress, the cumulative download duration, and the data volume; The target value is determined based on the score and weight corresponding to each of the reference factors; The priority parameter of the target data is determined based on the target value.

4. The data acquisition method according to claim 1, characterized in that, The step of adjusting the preset duration based on the priority parameter and the number of downloads to obtain the required download duration includes: A correction value is determined based on the priority parameter and the number of downloads, wherein the priority parameter, the number of downloads, and the correction value are positively correlated. The target duration is determined based on the correction value, and the target duration is positively correlated with the correction value; The difference between the preset duration and the target duration is determined as the required download duration.

5. The data acquisition method according to claim 1, characterized in that, The step of determining the second storage device and the third storage device according to the configuration file includes: Obtain the local storage path and the off-site storage path according to the configuration file; The storage device corresponding to the local storage path is designated as the second storage device, and the storage device corresponding to the off-site storage path is designated as the third storage device.

6. The data acquisition method according to any one of claims 1-5, characterized in that, The step of obtaining the configuration file associated with the first storage device and the parameter information of the target data includes: Based on the download record information, determine the number of times the target data was downloaded from the first storage device; When the number of downloads reaches a preset number, the configuration file associated with the first storage device and the parameter information of the target data are obtained.

7. A data acquisition device, characterized in that, include: A first sending module is configured to send a data acquisition request to a first storage device, wherein the data acquisition request is used by the first storage device to provide feedback data. The acquisition module is used to acquire the configuration file associated with the first storage device and the parameter information of the target data when the complete target data fed back by the first storage device is not acquired within a preset time period. The parameter information includes at least one of the following: the download progress of the target data, the cumulative download time, and the data volume. A first determining module is configured to determine a second storage device and a third storage device according to the configuration file, wherein the second storage device is located in the same area as the first storage device, and the third storage device is located in a different area from the first storage device; The second determining module is used to determine the download requirement duration of the target data based on the parameter information, and to determine a target storage device that matches the download requirement duration among the second storage device and the third storage device. The target storage device is either the second storage device or the third storage device that returns complete target data within the preset duration. The second sending module is used to send a data acquisition request to the target storage device in order to obtain the target data fed back by the target storage device; The second determining module is specifically used for: Based on the download record information, determine the number of times the target data was downloaded from the first storage device; The priority parameter of the target data is determined based on the parameter information, and the priority parameter is used to indicate the urgency of downloading the target data. The required download duration is obtained by adjusting the preset duration based on the priority parameter and the number of downloads.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product stores a computer program that, when executed by a processor, is used to implement the method as described in any one of claims 1 to 6.

Images