File uploading method and device, electronic equipment and storage medium

Abstract

The application provides a file uploading method and device, electronic equipment and storage medium. The method comprises: obtaining a to-be-uploaded file, and determining a context vector corresponding to the to-be-uploaded file; obtaining a transmission strategy set; for any candidate transmission strategy in the transmission strategy set, determining a strategy tendency score corresponding to the candidate transmission strategy based on a preset association matrix, a preset decision tensor and the context vector; determining a target transmission strategy corresponding to the to-be-uploaded file based on the strategy tendency scores corresponding to the candidate transmission strategies; and uploading the to-be-uploaded file according to the target transmission strategy. Thus, the file uploading process can be intelligently and adaptively optimized without user participation, so as to ensure the uploading success rate, simplify user operation and improve user experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a file upload method, apparatus, electronic device, and storage medium. Background Technology

[0002] File upload is a fundamental function of various applications and is widely used in scenarios such as instant messaging, cloud storage, social media, and online office.

[0003] In related technologies, existing file upload methods mainly adopt a fixed encoding strategy, such as saving small files directly to a micro server or a separate file server, while large files use a breakpoint resume method with fixed slice size.

[0004] However, the aforementioned existing technologies have the following drawbacks: 1. Fixed file upload strategies: a single transmission link is determined during the upload initialization phase, making dynamic switching based on real-time status impossible. 2. Strong coupling between rules and execution: business compliance rules are tightly bound to upload execution logic, making it difficult for different business lines to reuse the same upload infrastructure, increasing system redundancy and maintenance costs. 3. Coarse-grained resuming of interrupted uploads: using fixed slice sizes can easily lead to upload failures in weak network environments, while reducing transmission efficiency due to excessive requests in strong network environments. 4. Imbalance between accuracy and performance in determining instant upload speeds: relying solely on file size can easily lead to misjudgments. 5. Lack of closed-loop self-optimization capabilities: unable to automatically adjust based on actual upload results. Summary of the Invention

[0005] The purpose of this application is to provide a file upload method, apparatus, electronic device, and storage medium to solve at least one of the technical problems in the prior art caused by the use of fixed encoding hard-coded strategies, including policy rigidity, strong coupling between rules and execution, coarse granularity of breakpoint resume upload, imbalance in instant upload judgment, and lack of closed-loop self-optimization capability. The specific technical solution is as follows: In a first aspect of this application, a file upload method is provided, the method comprising: Obtain the file to be uploaded and determine the context vector corresponding to the file to be uploaded; Obtain a set of transmission strategies. For any candidate transmission strategy in the set of transmission strategies, determine the strategy preference score corresponding to the candidate transmission strategy based on a preset correlation matrix, a preset decision tensor, and the context vector. Based on the policy preference scores corresponding to each of the candidate transmission strategies, the target transmission strategy corresponding to the file to be uploaded is determined. Upload the file to be uploaded according to the target transmission strategy.

[0006] In one optional implementation, before obtaining the set of transport policies, the following steps are included: Determine the file size of the file to be uploaded; Construct a query key based on the file size of the file to be uploaded; Based on the query key, perform a first-level verification on the file to be uploaded and generate a first verification result; If the first verification result is a hit, a second verification is performed on the file to be uploaded to generate a second verification result; If the second verification result is inconsistent, the step of obtaining the transmission strategy set is executed.

[0007] In an optional implementation, a query key is constructed based on the file size of the file to be uploaded, including: If the file size of the file to be uploaded is less than or equal to a preset length threshold, then the hash value of the full file is determined based on the file content corresponding to the file to be uploaded. The query key is constructed based on the hash value of the full file and the file size of the file to be uploaded. If the file size of the file to be uploaded is greater than the preset length threshold, then the header file content and the footer file content are extracted from the file content corresponding to the file to be uploaded. Determine the header file hash value corresponding to the header file content and the tail file hash value corresponding to the tail file content; The query key is constructed based on the header file hash value corresponding to the header file content, the tail file hash value corresponding to the tail file content, and the file size of the file to be uploaded.

[0008] In one optional implementation, a set of transmission policies is obtained, including: The rules in the preset rule list are traversed to determine at least one candidate rule that matches the context vector. Obtain the transmission strategy corresponding to each candidate rule, and determine the transmission strategy set based on the transmission strategy corresponding to each candidate rule.

[0009] In an optional implementation, the transmission strategy corresponding to each candidate rule is obtained, and a transmission strategy set is determined based on the transmission strategy corresponding to each candidate rule, including: Based on the priority of each candidate rule, the target rule is determined from each candidate rule; Obtain the transmission policy constraints corresponding to the target rule, wherein the transmission policy constraints include a mandatory allow list and / or a mandatory prohibit list; Based on the mandatory allow list and / or mandatory prohibit list, the transmission policies corresponding to the target rules are retained and / or eliminated to obtain the transmission policy set.

[0010] In an optional implementation, the policy preference score corresponding to the candidate transmission policy is determined based on a preset correlation matrix, a preset decision tensor, and the context vector, including: For any feature in the context vector, determine the discretization state corresponding to the feature; Obtain the state influence coefficient corresponding to the discretized state; Obtain the basic weight parameters corresponding to the feature from the preset weight table; The product of the basic weight parameter corresponding to the feature and the state influence coefficient is taken as the basic amplitude corresponding to the feature; The base amplitude corresponding to the feature is corrected according to the preset correlation matrix to obtain the corrected amplitude corresponding to the feature; Based on the preset decision tensor and the correction magnitude corresponding to each feature, the policy preference score corresponding to the candidate transmission strategy is determined.

[0011] In an optional implementation, the policy preference score corresponding to the candidate transmission policy is determined based on the preset decision tensor and the correction magnitude corresponding to each feature, including: Obtain the default value corresponding to the candidate transmission strategy from the preset decision tensor; For any of the aforementioned features, the product of the correction magnitude corresponding to the feature and the default value corresponding to the candidate transmission strategy is determined as the feature tendency component corresponding to the feature. Based on the feature tendency components corresponding to each of the aforementioned features, the strategy tendency score corresponding to the candidate transmission strategy is determined.

[0012] In an optional implementation, uploading the file to be uploaded according to the target transmission strategy includes: Obtain real-time network metrics, including uplink transmission rate, round-trip time, device CPU utilization, and packet loss rate; The initial slice value is determined based on the aforementioned real-time network metrics; Get historical slice values; Candidate slice values ​​are determined based on the historical slice values ​​and the initial slice values; The target slice value is determined based on the preset amplitude limiting function and the candidate slice values; The file to be uploaded is split according to the target slice value to obtain at least one slice; Upload each of the aforementioned fragments according to the target transmission strategy.

[0013] In an optional implementation, after uploading the file to be uploaded according to the target transmission strategy, the method further includes: Obtain the log records corresponding to the file to be uploaded according to the target transmission strategy; Determine the revenue score corresponding to the log record; The preset decision tensor is updated based on the profit score.

[0014] In an optional implementation, the target transmission strategy corresponding to the file to be uploaded is determined based on the strategy preference score corresponding to each of the candidate transmission strategies, including: Obtain a set of correction factors, which includes at least one of the following: strategy type preference coefficient, cost coefficient, business scenario coefficient, device status coefficient, real-time requirement coefficient, and security level coefficient; The policy preference scores corresponding to each candidate transmission strategy are corrected according to the set of correction factors to obtain the corrected policy preference scores corresponding to each candidate transmission strategy. Based on the modified policy preference score corresponding to each of the candidate transmission strategies, the target transmission strategy corresponding to the file to be uploaded is determined.

[0015] In a second aspect of this application, a file uploading device is also provided, the device comprising: The vector determination module is used to obtain the file to be uploaded and determine the context vector corresponding to the file to be uploaded; The strategy preference score determination module is used to obtain a set of transmission strategies and, for any candidate transmission strategy in the set of transmission strategies, determine the strategy preference score corresponding to the candidate transmission strategy based on a preset correlation matrix, a preset decision tensor, and the context vector. The target transmission strategy determination module is used to determine the target transmission strategy corresponding to the file to be uploaded based on the strategy preference score corresponding to each of the candidate transmission strategies. The file upload module is used to upload the file to be uploaded according to the target transmission strategy.

[0016] In a third aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the file upload method described in any one of the first aspects above.

[0017] In a fourth aspect of the embodiments of this application, a storage medium is also provided, the storage medium storing instructions that, when run on a computer, cause the computer to execute any of the file upload methods described in the first aspect above.

[0018] In a fifth aspect of the embodiments of this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the file upload methods described in the first aspect above.

[0019] The technical solution provided in this application involves: acquiring the file to be uploaded and determining the context vector corresponding to the file; acquiring a set of transmission strategies; determining a strategy preference score for any candidate transmission strategy in the set based on a preset association matrix, a preset decision tensor, and the context vector; determining the target transmission strategy for the file to be uploaded based on the strategy preference scores of each candidate transmission strategy; and uploading the file according to the target transmission strategy. This technical solution achieves adaptive selection of the target transmission strategy by jointly deciding using a preset association matrix, a preset decision tensor, and the context vector. It eliminates the need for manually preset fixed transmission methods, significantly reducing maintenance costs and the frequency of manual intervention. Simultaneously, since the decision is executed immediately after determining the context vector, and the target strategy is output through a combination of the preset association matrix, the preset decision tensor, and the context vector, it avoids low transmission efficiency or failures caused by rigid strategies. Therefore, it enables intelligent and adaptive optimization of the file upload process without user intervention, thereby simplifying user operations and improving user experience while ensuring upload success rates. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0023] Figure 1 A schematic diagram illustrating the implementation process of a file upload method provided in this application embodiment; Figure 2 A schematic diagram illustrating the implementation process of a method for verifying a file to be uploaded, provided in an embodiment of this application; Figure 3A schematic diagram illustrating the implementation process of a method for obtaining a set of transmission strategies provided in this application embodiment; Figure 4 A schematic diagram illustrating the implementation process of a method for determining a strategy preference score provided in an embodiment of this application; Figure 5 A schematic diagram illustrating the implementation process of a method for determining a target transmission strategy provided in an embodiment of this application; Figure 6 A schematic diagram illustrating the implementation process of another file upload method provided in this application embodiment; Figure 7 A schematic diagram illustrating the implementation process of a preset decision tensor update method provided in this application embodiment; Figure 8 This application provides a schematic flowchart of a file upload system according to an embodiment of the present application. Figure 9 This is a schematic diagram of the structure of a file upload device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0026] The file upload method provided in this application will be further explained and described below with reference to the accompanying drawings and specific embodiments. The embodiments do not constitute a limitation on the embodiments of the present invention.

[0027] To address at least one of the technical problems in existing technologies caused by the use of fixed encoding hard-coded strategies, including policy rigidity, strong coupling between rules and execution, coarse granularity of breakpoint resume, imbalance in instant transmission judgment, and lack of closed-loop self-optimization capability, this application provides a file upload method. The method involves: acquiring the file to be uploaded and determining its corresponding context vector; acquiring a set of transmission strategies; determining a policy preference score for any candidate transmission strategy in the set based on a preset association matrix, a preset decision tensor, and the context vector; determining the target transmission strategy for the file to be uploaded based on the policy preference scores of each candidate transmission strategy; and uploading the file according to the target transmission strategy.

[0028] This technical solution achieves adaptive selection of the target transmission strategy by jointly deciding using a preset association matrix, a preset decision tensor, and a context vector. It eliminates the need for manually preset fixed transmission methods, significantly reducing operational costs and the frequency of manual intervention. Simultaneously, because the decision is executed immediately after determining the context vector, and the target strategy is output through a combination of the preset association matrix, preset decision tensor, and context vector, it avoids transmission inefficiencies or failures caused by rigid strategies. Therefore, it enables intelligent and adaptive optimization of the file upload process without user intervention, thereby simplifying user operations and improving user experience while ensuring upload success rates.

[0029] like Figure 1 The diagram shown illustrates the implementation flow of a file upload method provided in this application embodiment, which may specifically include the following steps: S101, obtain the file to be uploaded and determine the context vector corresponding to the file to be uploaded.

[0030] The aforementioned file to be uploaded refers to any data object that the user or application needs to transfer from the client to the server, such as a video file, an image, a document, or a compressed file.

[0031] The aforementioned context vector refers to multidimensional feature data describing the file to be uploaded and its upload environment. This includes, but is not limited to: business identifier, file size, internet media type, and current network quality level.

[0032] The business identifier is a unique string used to distinguish different business lines (which can be understood as different product lines or business departments, such as short video business and live streaming business) or application scenarios (which can be understood as different functional modules under the same product line, such as user avatar upload, content publishing upload, log upload, etc.). For example, "SHORT_VIDEO" represents the short video business, and "LIVE" represents the live streaming business.

[0033] Internet media type refers to a standard identifier describing the type of file to be uploaded. For example, "video / mp4" represents MP4 video, "image / jpeg" represents JPEG image, and "application / pdf" represents PDF document.

[0034] In this embodiment of the application, the file to be uploaded can be obtained, and the context vector corresponding to the file to be uploaded can be determined.

[0035] As an optional implementation, the client can read the number of bytes of the file to be uploaded as the file size by calling its operating system file interface; parse the MIME type from the file extension or file header of the file to be uploaded; obtain the service identification code from the current application session; measure the round-trip latency by sending ICMP probe packets or using the system network callback interface, and convert the round-trip latency into a network quality level according to a preset threshold mapping rule (e.g., round-trip latency ≤ 50ms is EXCELLENT, round-trip latency ≥ 500ms is POOR); finally, combine the data from the above dimensions into a dictionary or JSON structure as a context vector.

[0036] As an alternative implementation, only the file size and current network type (such as WiFi / 4G / 5G) can be collected, while other features can use default values ​​or be inferred from historical records to obtain the context vector corresponding to the file to be uploaded. For example, if the current network is WiFi and the signal strength is high, the default network quality level is excellent; the service identification code is obtained directly from the link parameters of the upload request.

[0037] S102, obtain the transmission strategy set, and for any candidate transmission strategy in the transmission strategy set, determine the strategy preference score corresponding to the candidate transmission strategy based on the preset correlation matrix, preset decision tensor and context vector.

[0038] The aforementioned set of transmission strategies refers to the current set of candidate upload methods. Each transmission strategy represents a file data transfer mode, such as direct upload (i.e., the client directly establishes a connection with the business gateway to upload), bypass upload (i.e., directly uploads to object storage via temporary credentials from the cloud storage security token service), P2P transmission (point-to-point assisted transmission), etc. The set of transmission strategies can contain one or more candidate transmission strategies, depending on the matching result between the business rules and the context vector.

[0039] The aforementioned predefined correlation matrix refers to a predefined square matrix used to characterize the coupling strength between different dimensions of context vectors. For example, a predefined correlation matrix can be shown in Table 1. Matrix elements This represents the amplification (or attenuation) coefficient of the contribution of the j-th (e.g., 1, 2, etc.) dimension to the i-th (e.g., 1, 2, etc.) dimension. Using this matrix, the indirect impact of state changes in one dimension on decision-making can be transferred to other dimensions, thereby correcting the bias caused by the assumption of independence in a single dimension. For example, This indicates that the network quality dimension (dimension 1) has a significant amplifying effect on the file size dimension (dimension 0). In other words, when network quality deteriorates, the file size dimension's influence on decision-making is amplified by an additional 30%. 0.05 indicates that the file size has a very small impact on network quality, which can be approximated as negligible.

[0040] Table 1

[0041] It should be noted that for unconfigured elements (including diagonal elements, such as...) The value can be kept at 0. The preset association matrix can be loaded into memory when the gateway service starts, and then called by the decision engine during calculation.

[0042] The aforementioned preset decision tensor refers to a pre-stored multidimensional array with the dimensional structure of [feature index][discrete state index][action index]. This tensor records the basic tendency weights for recommending each candidate transmission strategy when a feature in the context vector is in a specific discretization state, under simplified conditions that only consider a single dimension and do not consider the influence of other dimensions. For example, as shown in Table 2, when the file size dimension is in the "extremely large" state (e.g., greater than 1GB), the default value in the decision tensor for bypass upload is 0.95, for direct upload it is 0.01, and for P2P transmission it is 0.40. This preset decision tensor can be dynamically updated through expert experience configuration or subsequent feedback learning.

[0043] Table 2

[0044] The aforementioned strategy preference score refers to a numerical rating determined by comprehensively considering various features in the current context vector for a given candidate transmission strategy. This score directly reflects the overall recommendation level of adopting the candidate transmission strategy in the current environment; a higher score indicates a better candidate transmission strategy. For example, the strategy preference score for candidate transmission strategy 1 is 0.00067, for candidate transmission strategy 2 it is 0.544, and for candidate transmission strategy 3 it is 0.0118.

[0045] In this embodiment of the application, a set of transmission strategies can be obtained. For any candidate transmission strategy in the set of transmission strategies, a strategy preference score corresponding to the candidate transmission strategy can be determined based on a preset correlation matrix, a preset decision tensor, and a context vector.

[0046] It should be noted that if the transmission strategy set contains only one candidate transmission strategy, it means that there is only one upload method available. In this case, the single candidate transmission strategy can be directly used as the target transmission strategy. If the transmission strategy set does not contain any candidate transmission strategies, it means that there is no candidate transmission strategy suitable for the file to be uploaded. In this case, the preset degraded upload mode (e.g., the default direct upload mode) can be triggered, and the upload data can be recorded for subsequent analysis and optimization.

[0047] Before obtaining the set of transmission strategies, the uniqueness of the files to be uploaded needs to be verified (i.e., the instant upload mechanism) to avoid uploading the same files repeatedly. The specific details of how the uniqueness of uploaded files is verified will be explained below. Figure 2 The process shown will be explained in detail here.

[0048] As for how the set of transmission strategies is obtained, it will be explained below. Figure 3 The process shown will be explained in detail here.

[0049] The specific method for determining the policy preference score corresponding to the candidate transmission strategy based on the preset correlation matrix, preset decision tensor, and context vector will be explained below. Figure 4 The process shown will be explained in detail here.

[0050] S103, based on the policy preference scores corresponding to each candidate transmission strategy, determine the target transmission strategy corresponding to the file to be uploaded.

[0051] The aforementioned target transmission strategy refers to the single transmission method ultimately selected from the set of transmission strategies and actually used to perform file upload, such as direct upload, bypass upload, or P2P transmission.

[0052] In this embodiment of the application, the target transmission strategy corresponding to the file to be uploaded can be determined based on the strategy preference score corresponding to each candidate transmission strategy.

[0053] As an optional implementation, the candidate transmission strategy with the highest strategy preference score can be directly determined as the target transmission strategy.

[0054] As another alternative implementation, the policy preference scores of each candidate transmission strategy can be probabilistically processed first (e.g., the scores can be converted into a probability distribution using a Softmax function), and then random sampling can be performed based on the probability distribution, or the candidate transmission strategy with the highest probability can be directly selected as the target transmission strategy.

[0055] The specific method for determining the target transmission strategy for the file to be uploaded based on the strategy preference scores of each candidate transmission strategy will be explained below. Figure 5The process shown will be explained in detail here.

[0056] S104. Upload the file to be uploaded according to the target transmission strategy.

[0057] In this embodiment of the application, the file to be uploaded can be uploaded according to the target transmission strategy.

[0058] As for the specifics of how the files to be uploaded are uploaded according to the target transmission strategy, please refer to the following text. Figure 6 The process shown will be explained in detail here.

[0059] After uploading the file according to the target transmission strategy, the preset decision tensor can be updated based on the data generated during the upload process. The specific method for updating the preset decision tensor will be explained below. Figure 7 The process shown will be explained in detail here.

[0060] Based on the above description of the technical solution provided in the embodiments of this application, the following steps are taken: First, obtain the file to be uploaded and determine the context vector corresponding to the file to be uploaded. Second, obtain a set of transmission strategies. For any candidate transmission strategy in the set of transmission strategies, determine the strategy preference score corresponding to the candidate transmission strategy based on a preset correlation matrix, a preset decision tensor, and the context vector. Third, determine the target transmission strategy corresponding to the file to be uploaded based on the strategy preference scores corresponding to each candidate transmission strategy. Finally, upload the file to be uploaded according to the target transmission strategy.

[0061] This technical solution achieves adaptive selection of the target transmission strategy by jointly deciding using a preset association matrix, a preset decision tensor, and a context vector. It eliminates the need for manually preset fixed transmission methods, significantly reducing operational costs and the frequency of manual intervention. Simultaneously, because the decision is executed immediately after determining the context vector, and the target strategy is output through a combination of the preset association matrix, preset decision tensor, and context vector, it avoids transmission inefficiencies or failures caused by rigid strategies. Therefore, it enables intelligent and adaptive optimization of the file upload process without user intervention, thereby simplifying user operations and improving user experience while ensuring upload success rates.

[0062] like Figure 2 The diagram shown is a schematic representation of the implementation process of a method for verifying files to be uploaded, as provided in an embodiment of this application. Figure 2 exist Figure 1 Building upon this, the document details how to verify the uniqueness of uploaded files, which may include the following: S201, Determine the file size of the file to be uploaded.

[0063] In this embodiment, the file size of the file to be uploaded can be determined. The file size refers to the number of bytes of storage space occupied by the file to be uploaded, such as 1024 bytes or 1MB.

[0064] Specifically, the client can obtain the file size of the file to be uploaded by calling the file interface provided by its operating system.

[0065] S202, construct the query key based on the file size of the file to be uploaded.

[0066] In this embodiment, a query key can be constructed based on the file size of the file to be uploaded. This query key is used to quickly retrieve whether a file with the same content as the file to be uploaded already exists in the cloud. The query key serves as the primary key for subsequent first-level verification (i.e., fast fingerprint filtering), and its construction method directly affects the efficiency and accuracy of the verification.

[0067] As an optional implementation, constructing a query key based on the file size of the file to be uploaded may specifically include the following steps: Step 11: If the file size of the file to be uploaded is less than or equal to the preset length threshold, then determine the full file hash value based on the file content corresponding to the file to be uploaded.

[0068] The aforementioned preset length threshold refers to the critical number of bytes used to distinguish between "small files" and "large files," such as 2MB (2,097,152 bytes). This threshold can be dynamically adjusted according to business scenarios (such as typical file distribution, user network bandwidth, and device I / O performance).

[0069] The file content mentioned above refers to the complete binary data sequence of the file to be uploaded. For text files, the content is the byte sequence after character encoding; for multimedia files such as images and videos, the content is the corresponding raw pixel or encoded frame data. The file content can be obtained through the file reading interface provided by the operating system.

[0070] The aforementioned full file hash value refers to a fixed-length digest value calculated using a cryptographic hash algorithm (such as MD5, SHA-1, SHA-256, etc.) with the entire file content as input. This value is collision-resistant and can uniquely identify the file content; even the slightest change to the file will result in a completely different hash value.

[0071] In this embodiment of the application, if the file size of the file to be uploaded is less than or equal to a preset length threshold, it indicates that the file size is small, the time required to read the entire file content and calculate the hash value is low, and it will not cause a perceptible wait for the user. Therefore, the hash value of the entire file can be determined based on the file content corresponding to the file to be uploaded.

[0072] Specifically, the operating system's file reading interface can be called to open the file to be uploaded in binary read mode; then, all bytes of the file to be uploaded (i.e., the file content) are read into the buffer in batches, and the state of the hash calculator is continuously updated (or a small file is read into memory at once); finally, after all bytes have been calculated, a fixed-length hash digest (and the hash value of the entire file) is output. For example, a 200KB text file has a hash value of "3c6e0b8a9c..." obtained by SHA-256.

[0073] Step 12: Construct a query key based on the hash value of the full file and the file size of the file to be uploaded.

[0074] In this embodiment of the application, a query key can be constructed based on the hash value of the full file and the file size of the file to be uploaded.

[0075] Specifically, the file size and the hash value of the entire file can be concatenated according to a preset format to obtain the query key. For example: "size=1048576&hash=3c6e0b8a9c...", or a more compact format such as "1048576:3c6e0b8a9c...". For small file scenarios, this query key can uniquely identify a file, and the server can directly store the mapping relationship between this key and file metadata.

[0076] Step 13: If the file size of the file to be uploaded is greater than the preset length threshold, extract the header and footer file content from the file content corresponding to the file to be uploaded.

[0077] In this embodiment, if the file size of the file to be uploaded is greater than a preset length threshold, it means that the file is large (e.g., more than 2MB). If the full hash is still calculated, it will result in high I / O overhead and long preprocessing time, which will affect the user experience. Therefore, the header and footer file content can be extracted from the file content corresponding to the file to be uploaded in order to construct the query key later.

[0078] The header file content refers to a continuous sequence of bytes (e.g., 1MB) read from the beginning of the file content; the tail file content refers to a continuous sequence of bytes (e.g., 1MB) read from the end of the file content. The sampling lengths of the header and tail can be the same, or they can be configured to different values ​​according to actual needs (e.g., 512KB for the header and 1MB for the tail).

[0079] Specifically, by randomly accessing the file, N bytes (where N is a positive integer, such as 32) are read from the beginning of the file as the header content, and then N bytes are read from the end of the file at an offset of N bytes from the end of the file as the tail content. If the total file length is greater than 2N but less than 2N, only the maximum usable length can be used.

[0080] Step 14: Determine the header file hash value corresponding to the header file content and the tail file hash value corresponding to the tail file content.

[0081] In this embodiment of the application, the header file hash value corresponding to the header file content and the tail file hash value corresponding to the tail file content can be determined by a hash algorithm (such as SHA-256).

[0082] Step 15: Construct a query key based on the header file hash value corresponding to the header file content, the tail file hash value corresponding to the tail file content, and the file size of the file to be uploaded.

[0083] In this embodiment, the header file hash value corresponding to the header file content, the tail file hash value corresponding to the tail file content, and the file size of the file to be uploaded can be concatenated according to a preset format to construct a query key.

[0084] S203, perform first-level verification on the file to be uploaded based on the query key, and generate the first verification result.

[0085] The aforementioned first-level verification refers to the process of checking the existence of the query key using a fast fingerprint filter (such as a Bloom filter, Redis hash table, or in-memory dictionary). First-level verification can quickly exclude files that are determined to be non-existent, reducing the query pressure on the backend storage.

[0086] In this embodiment of the application, a first-level verification is performed on the file to be uploaded based on the query key to generate a first verification result.

[0087] Specifically, the query key can be sent to a fast fingerprint filter (such as a Bloom filter or distributed cache) for an existence check. If the query key exists in the fast fingerprint filter, the first verification result is a hit, indicating that a file with the same content as the file to be uploaded may exist in the cloud, but it cannot be completely confirmed and a second-level verification is required for confirmation. If the query key does not exist in the fast fingerprint filter, the first verification result is a miss, indicating that it is almost impossible for a file with the same content to exist in the cloud, and the second-level verification can be skipped directly to proceed to the subsequent normal upload process (i.e., obtaining the transmission strategy set).

[0088] S204. If the first verification result is a hit, perform a second verification on the file to be uploaded to generate a second verification result.

[0089] The aforementioned secondary verification refers to the verification process that uses a strong hash algorithm (such as SHA-256) to calculate the complete content of the file to be uploaded, and then compares the calculation result bit by bit with the full hash value of the candidate file stored on the server.

[0090] In this embodiment, if the first verification result is a match, it indicates that the same query key already exists in the fast fingerprint filter, and the cloud may have already stored files with the same content. However, since the first-level verification may theoretically result in misjudgment (for example, when using sampling hash to construct the query key, different files may have the same beginning and end but different middle parts; or the Bloom filter may produce false positives), a second-level verification is required for the file to be uploaded to generate a second verification result, in order to eliminate the possibility of misjudgment and ensure the accuracy of the instant transmission.

[0091] Specifically, the client can calculate the full file hash value of the file to be uploaded in a background thread and report this value to the instant upload index service. The server locates the candidate file in the cloud based on the query key matched by the first-level check, obtains its pre-stored full hash value, and compares it bit by bit with the full file hash value reported by the client. If they match completely, the second check result is consistent, indicating that the file to be uploaded does indeed exist in the cloud. At this point, the server can directly return a successful instant upload and the file access address, and the entire upload process ends. If they do not match, the second check result is inconsistent, indicating that the first-level check has misjudged, and the file to be uploaded is not actually stored in the cloud.

[0092] S205, if the second verification result is inconsistent, execute the step of obtaining the transmission strategy set.

[0093] In this embodiment of the application, if the second verification result is inconsistent, it indicates that the first-level verification was a false positive, and the file to be uploaded does not exist in the cloud, therefore instant transfer cannot be used. At this time, the instant transfer verification process should be exited, and the step of obtaining the transmission strategy set should be executed, i.e., step S102.

[0094] like Figure 3 The diagram shown is a schematic representation of the implementation flow of a method for obtaining a transmission strategy set according to an embodiment of this application. Figure 3 exist Figure 1 Based on this, a detailed description is provided on how to obtain the set of transmission strategies, which may include the following: S301, traverse each rule in the preset rule chain to determine at least one candidate rule that matches the context vector.

[0095] The aforementioned pre-configured rule list refers to a set of rules pre-configured and sorted by priority. Each rule defines the triggering conditions and corresponding transmission constraints. For example, the pre-configured rule list can be shown in Table 3.

[0096] Table 3

[0097] In this embodiment of the application, each rule in the preset rule chain can be traversed to determine at least one candidate rule that matches the context vector.

[0098] For example, for the context vector {bizCode:"SHORT_VIDEO",fileSize:500MB,mimeType:"video / mp4",networkQuality:"MODERATE"}: Rule 1: bizCode matches, fileSize≥100MB, and mimeType matches, while Rule 2: bizCode does not match, Rule 3: fileSize>1MB does not match, and Rule 4: network quality level does not match. Therefore, the candidate rule is Rule 1.

[0099] S302, obtain the transmission strategy corresponding to each candidate rule, and determine the transmission strategy set according to the transmission strategy corresponding to each candidate rule.

[0100] In this embodiment of the application, for any candidate rule, a transmission strategy corresponding to the candidate rule can be obtained from a preset rule chain, and a transmission strategy set can be determined according to the transmission strategies corresponding to each candidate rule. The transmission strategies may include direct transmission (DIRECT), side-up upload (SIDE_UPLOAD), and P2P transmission.

[0101] Specifically, candidate rules can be filtered according to their priority, and the candidate rule with the highest priority can be selected as the target rule. The transmission policy constraints corresponding to the target rule are obtained from a preset rule list, whereby the transmission policy constraints include a mandatory allow list and / or a mandatory prohibit list. Based on the mandatory allow list and / or mandatory prohibit list, the transmission policies corresponding to the target rule are retained and / or eliminated to obtain a set of transmission policies.

[0102] For example, based on the rules in Table 3, if only rule 1 (priority level 1) is hit, then the target rule is rule 1, and its mandatory allow list is ["SIDE_UPLOAD"], so the transmission strategy set is {bypass upload}. If only rule 2 (priority level 1) is hit, then the target rule is rule 2, and its mandatory prohibit list is ["P2P"], so the transmission strategy set is {direct upload, bypass upload}. If both rule 1 (level 1) and rule 2 (level 1) are hit, then both have the same priority and are both target rules, requiring the constraints to be merged: rule 1's mandatory allow list ["SIDE_UPLOAD"] and rule 2's mandatory prohibit list ["P2P"]. First, obtain {bypass upload} based on the mandatory allow list, and then obtain the transmission strategy set {bypass upload} based on the mandatory prohibit list. If only rule 3 (level 2) is hit, and no higher-level rule is hit, then the target rule is rule 3, its mandatory allow list is ["DIRECT", "SIDE_UPLOAD"], and the transmission strategy set is {direct upload, bypass upload}. If only rule 4 (level 3) is met, the transmission strategy set is {bypass upload}.

[0103] It's important to note that the preset rule list supports hot reloading, meaning it takes effect without requiring a service restart. Specifically, when rule configurations change, the configuration center broadcasts a version change event to all gateway nodes. The rule synchronization component on each node performs the following operations: pulls incremental or full rule data from the configuration center; builds a new version's rule index hash table in memory; and instantly switches the global rule pointer from the old version address to the new version address via an atomic pointer swap operation. Upload requests currently being processed continue to use the old version of the rules, while new requests immediately use the new version, thus achieving real-time effect of rule configurations without impacting online services.

[0104] like Figure 4 The diagram shown is a schematic representation of the implementation process of a method for determining a strategy preference score according to an embodiment of this application. Figure 4 exist Figure 1 Building upon this foundation, the paper details how to determine the policy preference score corresponding to a candidate transmission policy based on a pre-defined correlation matrix, a pre-defined decision tensor, and a context vector. Specifically, this may include the following: S401, for any feature in the context vector, determine the discretization state corresponding to the feature.

[0105] In the embodiments of this application, the discretization state corresponding to any feature in the context vector can be determined.

[0106] For example, the feature corresponding to file size in the context vector can be divided into five states: tiny (0-1MB), small (1-10MB), medium (10-100MB), large (100MB-1GB), and very large (>1GB). If the network quality level is already a discrete enumeration (such as excellent, good, medium, poor), it can be directly used as the discretized state corresponding to the feature.

[0107] S402, obtain the state influence coefficient corresponding to the discretized state.

[0108] In this embodiment, the state influence coefficient corresponding to the discretized state can be obtained. The state influence coefficient represents the degree to which the discretized state amplifies or attenuates the feature. The state influence coefficient can be set through expert experience or adjusted subsequently through feedback learning. For example, for the file size feature, the state influence coefficient corresponding to the "extremely large" state can be set to 1.0 (baseline), and the "small" state to 0.5; for the network quality dimension, the "excellent" state is set to 1.0, and the "poor" state to 0.3. The corresponding state influence coefficient can be found from a preset configuration table based on the feature name and the discretized state index.

[0109] S403: Obtain the basic weight parameters corresponding to the feature from the preset weight table.

[0110] In this embodiment, basic weight parameters corresponding to the features can be obtained from a preset weight table. These basic weight parameters represent the importance of the feature in the decision-making process and can be configured based on business experience, remaining unchanged regardless of the discretization state. For example, the basic weight parameter for the file size feature can be set to 1.0, and the basic weight parameter for the network quality dimension can be set to 1.2, indicating that under the same conditions, changes in network quality are more sensitive to upload success rate than file size. The basic weight parameters can also be dynamically adjusted through subsequent feedback learning.

[0111] S404 uses the product of the basic weight parameter corresponding to the feature and the state influence coefficient as the basic amplitude corresponding to the feature.

[0112] In this embodiment of the application, the product of the basic weight parameter corresponding to the feature and the state influence coefficient can be used as the basic amplitude corresponding to the feature.

[0113] For example, the context vector contains two features: file size and network quality. If the base weight parameter for file size is 1.0, and the current state is "extra large," with a state influence coefficient of 1.0, then the base magnitude of file size is 1.0 × 1.0 = 1.0. If the base weight parameter for network quality is 1.2, and the current state is "poor," with a state influence coefficient of 0.3, then the base magnitude of network quality is 1.2 × 0.3 = 0.36.

[0114] S405, Correct the basic amplitude corresponding to the feature according to the preset correlation matrix to obtain the corrected amplitude corresponding to the feature.

[0115] In this embodiment of the application, the basic amplitude corresponding to the feature can be corrected according to a preset correlation matrix to obtain the corrected amplitude corresponding to the feature.

[0116] Specifically, the matrix element corresponding to the feature can be obtained from the preset correlation matrix. ,according to Determine the correction magnitude corresponding to this feature. The base magnitude of the i-th feature. Let be the coupling influence coefficient between the j-th feature and the i-th feature in the predefined correlation matrix. This is the correction magnitude for the i-th feature.

[0117] For example, the basic magnitude of the file size feature =1.0, the base magnitude of network quality characteristics =0.36. The preset correlation matrix is ​​shown in Table 1, where... =0.05 (the impact of network quality on file size). =0.30 (the impact of file size on network quality), then the correction magnitude for the file size characteristic is 1.0 + 0.05 × 0.36 = 1.018. The correction magnitude for the network quality characteristic is 0.36 + 0.30 × 1.0 = 0.66.

[0118] S406, based on the preset decision tensor and the correction magnitude corresponding to each feature, determine the policy preference score corresponding to the candidate transmission policy.

[0119] In this embodiment, the policy preference score corresponding to the candidate transmission strategy can be determined based on the preset decision tensor and the correction magnitude corresponding to each feature.

[0120] As an optional embodiment, the policy preference score corresponding to the candidate transmission policy is determined based on the preset decision tensor and the correction magnitude corresponding to each feature. Specifically, this may include the following steps: Step 21: Obtain the default value corresponding to the candidate transmission strategy from the preset decision tensor.

[0121] In this embodiment, the default value corresponding to the candidate transmission strategy can be obtained from the preset decision tensor.

[0122] For example, as shown in Table 2, the default value is 0.95 when the file size feature is in the "extra large" state and the candidate strategy is bypass upload; the default value is 0.90 when the network quality feature is in the "poor" state and the candidate strategy is bypass upload.

[0123] Step 22: For any feature, the product of the correction magnitude corresponding to the feature and the default value corresponding to the candidate transmission strategy is determined as the feature tendency component corresponding to the feature.

[0124] In this embodiment of the application, for any feature, the product of the correction magnitude corresponding to the feature and the default value corresponding to the candidate transmission strategy is determined as the feature tendency component corresponding to the feature.

[0125] For example, based on the correction magnitude in step S405 (file size 1.018, network quality 0.66), for the bypass upload strategy: the characteristic bias component of file size = 1.018 × 0.95 = 0.9671. The characteristic bias component of network quality = 0.66 × 0.90 = 0.594.

[0126] Step 23: Based on the feature tendency components corresponding to each feature, determine the policy tendency score corresponding to the candidate transmission policy.

[0127] In this embodiment of the application, the strategy preference score corresponding to the candidate transmission strategy is determined based on the feature preference component corresponding to each feature.

[0128] As an optional embodiment, the feature preference components of all features can be multiplied or summed to obtain the policy preference score of the current candidate transmission strategy. For example, using multiplication, for bypass upload, the policy preference score = 0.9671 × 0.594 = 0.574. Using summation, the policy preference score = 0.9671 + 0.594 = 1.5611.

[0129] As another alternative embodiment, it can be based on ,in, The policy preference score is the score corresponding to the k-th candidate transmission policy. For the first i The correction magnitude of each feature, Obtained from the preset decision tensor and the first i The feature corresponds to and is related to the first feature. k The default values ​​corresponding to each candidate strategy To prevent division by zero constant, such as .

[0130] like Figure 5 The diagram shown is a schematic representation of the implementation flow of a method for determining a target transmission strategy according to an embodiment of this application. Figure 5 exist Figure 1 Based on this, the paper details how to determine the target transmission strategy for the file to be uploaded based on the strategy preference scores corresponding to each candidate transmission strategy. Specifically, this may include the following: S501, Obtain the set of correction factors, which includes at least one of the following: strategy type preference coefficient, cost coefficient, business scenario coefficient, device status coefficient, real-time requirement coefficient, and security level coefficient.

[0131] The aforementioned set of correction factors is used to introduce additional constraints such as business preferences, resource consumption, and equipment status on the basis of the strategy preference score, and to make a secondary adjustment to the recommendation degree of the candidate transmission strategy, so that the final selected target transmission strategy is more in line with actual operational needs and user experience.

[0132] The aforementioned strategy type preference coefficients refer to the preset preference values ​​for different transmission strategies (direct transmission, bypass transmission, P2P). For example, if a business wants to prioritize the use of the lower-cost bypass transmission, it can set its preference coefficient to 1.2, direct transmission to 1.0, and P2P to 0.8.

[0133] The aforementioned cost coefficient refers to an adjustment factor set based on cost factors such as bandwidth consumption and storage costs of the transmission strategy. The higher the cost of the strategy, the smaller the coefficient (reduced weighting). For example, a P2P strategy incurs user-side traffic costs, so its cost coefficient can be set to 0.7.

[0134] The aforementioned business scenario coefficients refer to the weighted coefficients used to assess the applicability of transmission strategies based on business type (such as short videos and live streaming). For example, for business types with high security requirements, the business scenario coefficient for the direct transmission strategy is 1.2, while the business scenario coefficient for the P2P strategy is 0.5.

[0135] The aforementioned device status coefficient refers to the adjustment of the strategy based on the client's battery level, storage space, temperature, and other conditions. For example, when the battery level is below 20%, the device status coefficient for the P2P strategy can be 0.6.

[0136] The aforementioned real-time requirement coefficient refers to a coefficient set based on the latency sensitivity of the upload task corresponding to the file to be uploaded. For example, live streaming has extremely high real-time requirements; the real-time requirement coefficient for direct upload is 1.5, while the real-time requirement coefficient for bypass upload strategy is 0.9.

[0137] The aforementioned security level coefficient refers to the adjustment of the transmission policy based on the confidentiality requirements of the data to be uploaded. For example, confidential data is prohibited from P2P, so its security level can be set to 0, forcibly disabling it; ordinary data can be set to 1.0.

[0138] In this application embodiment, a set of correction factors can be obtained, which includes at least one of the following: strategy type preference coefficient, cost coefficient, business scenario coefficient, device status coefficient, real-time requirement coefficient, and security level coefficient.

[0139] As an optional implementation, the correction factor values ​​corresponding to the current business line and device environment can be obtained by reading local configuration files (such as JSON or XML) or by sending a request to the configuration center. If a coefficient is not configured, it is set to 1.0 by default, indicating that no correction effect is generated. At the same time, the device status coefficient can be obtained in real time through the operating system (such as battery level) to determine the set of correction factors.

[0140] S502, adjust the policy preference score corresponding to each candidate transmission strategy according to the set of adjustment factors to obtain the adjusted policy preference score corresponding to each candidate transmission strategy.

[0141] In this embodiment of the application, the policy preference score corresponding to each candidate transmission strategy can be corrected according to the set of correction factors to obtain the corrected policy preference score corresponding to each candidate transmission strategy.

[0142] For example, the strategy preference scores for each candidate transmission strategy are: direct transmission 0.2, bypass transmission 0.7, and P2P 0.1. The set of correction factors includes: strategy type preference coefficient (direct transmission 1.0, bypass 1.2, P2P 0.8), cost coefficient (bypass 1.0, P2P 0.6), and business scenario coefficient (all 1.0). Therefore, after correction: direct transmission correction score = 0.2 × 1.0 × 1.0 = 0.2, bypass transmission correction score = 0.7 × 1.2 × 1.0 = 0.84, and P2P correction score = 0.1 × 0.8 × 0.6 = 0.048.

[0143] S503, based on the correction strategy preference score corresponding to each candidate transmission strategy, determine the target transmission strategy corresponding to the file to be uploaded.

[0144] In this embodiment, the modified policy preference scores of each candidate transmission strategy can be compared, and the strategy with the highest score can be selected as the target transmission strategy. If multiple strategies have the same score, a secondary selection can be made according to a preset priority (such as security priority or cost priority), or a strategy can be randomly selected.

[0145] like Figure 6 The diagram shown is a schematic representation of the implementation process of another file upload method provided in this application. Figure 6 exist Figure 1 Based on this, a detailed description is provided on how to upload files according to the target transmission strategy, which may include the following: S601 acquires real-time network metrics, including uplink transmission rate, round-trip latency, device CPU utilization, and packet loss rate.

[0146] The aforementioned real-time network metrics refer to data used to describe the current network transmission conditions and device load status, which are used to dynamically adjust the upload slice size and determine whether to trigger a degradation strategy.

[0147] Table 4 explains the parameters in the above real-time network metrics.

[0148] Table 4

[0149] In this embodiment of the application, real-time network metrics can be obtained, including uplink transmission rate, round-trip latency, device CPU utilization, and packet loss rate.

[0150] S602 determines the initial slice value based on real-time network metrics.

[0151] In this embodiment of the application, the initial slice value is determined based on real-time network metrics.

[0152] As an optional implementation, the initial slice value can be calculated based on the uplink transmission rate (BW): Sinitial= ×BW, where This is an empirical scaling factor (e.g., 0.8) used to map bandwidth to a reasonable number of bytes per slice. Additionally, if the round-trip latency is high or the packet loss rate is large, the initial value can be further adjusted downwards: Sinitial' = Sinitial × (1 b×Loss The formula is (c×RTTnorm-d×R), where b, c, and d are empirical scaling factors (e.g., 0.3, 0.4), Loss is the packet loss rate, R is the device CPU utilization, and RTTnorm is the normalized round-trip time. This step aims to adapt the slice size to network capabilities, avoiding the use of excessively large slices in poor network conditions that would increase retransmission costs.

[0153] As another optional embodiment, it can also be based on Determine the initial slice values. These are the initial slice values. is the empirical scaling factor (e.g., 0.8), Loss is the packet loss rate, R is the device CPU utilization, and RTTnorm is the normalized round-trip time.

[0154] S603, retrieve historical slice values.

[0155] In this embodiment, historical slice values ​​are obtained. These historical slice values ​​are the actual slice size used in the previous slice period or the previous upload task, and can be directly obtained from the client's local storage. If this is the first upload or the previous period record does not exist, the historical slice value can be set to a preset default slice size (e.g., 1MB). Historical slice values ​​are used to smooth out abrupt changes in slice size and prevent drastic slice changes due to instantaneous network fluctuations.

[0156] S604, determine candidate slice values ​​based on historical slice values ​​and initial slice values.

[0157] In this embodiment, candidate slice values ​​are determined based on historical slice values ​​and initial slice values. Specifically, the candidate slice values ​​can be calculated using a weighted moving average formula: S = β × Sinitial + (1 Sinitial is the initial slice value, S0 is the historical slice value, and S is the candidate slice value. A larger β indicates greater sensitivity to changes in the current network state, while a smaller β results in smoother slice size changes. This processing can filter out instantaneous noise in network metrics, making slice size adjustments more stable.

[0158] S605 determines the target slice value based on a preset limiting function and candidate slice values.

[0159] In this embodiment, the target slice value is determined based on a preset limiting function and candidate slice values. This step ensures that the final slice value is always within the physically achievable range, avoiding slices that are too large or too small due to abnormal indicators.

[0160] The preset limiting function is: ; 'a' represents the lower limit of the interval, which is the preset minimum sharding threshold (e.g., 128KB, determined by the client's hardware I / O capabilities or the minimum requirements of the cloud storage service). 'b' represents the upper limit of the interval, which is the preset maximum sharding threshold (e.g., 5GB, determined by the cloud storage service provider's single sharding limit or the maximum sharding threshold in the rule configuration). The candidate sharding value is the candidate sharding value. This is the target slice value. This explicitly defined limiting function ensures that the dynamically calculated target slice value will not exceed physical limits or service constraints due to momentary anomalies in network metrics, guaranteeing the stability and compatibility of the upload process. This calculation process is periodically re-executed (e.g., every 5 minutes) after every certain number of slices are uploaded to adapt to real-time changes in network conditions.

[0161] S606: Split the file to be uploaded according to the target slice value to obtain at least one slice.

[0162] In this embodiment of the application, the file to be uploaded can be split according to the target slice value to obtain at least one slice.

[0163] S607, upload each fragment according to the target transmission strategy.

[0164] In this embodiment, each fragment can be uploaded sequentially according to the target transmission strategy. If a fragment upload fails or times out, a limited number of retries are performed according to the retry strategy. If the network is interrupted, the current upload progress is persisted, and after the network is restored, the server's list of successfully uploaded fragments is queried, and the upload continues from the breakpoint. After all fragments are uploaded, a merge request is sent to the server. The server assembles the file according to the fragment sequence number and verifies the hash value of the entire file. After successful verification, a permanent storage address is returned.

[0165] It's important to note that during the upload process, the breakpoint state machine records the current upload stage. The states include: INIT (initialization), HASHING (hash calculation), UPLOADING (uploading), PAUSED (paused), RECOVERING (recovering), MERGING (merging), DONE (complete), and FAILED (failure). When a network interruption is detected, the state machine transitions from UPLOADING to PAUSED, forcing the local persistence of fragment metadata. After network recovery, the state machine transitions from PAUSED to RECOVERING. The client sends a ListParts request to the server to retrieve the list of successfully uploaded fragments, compares it with the local ChunkMeta to locate incomplete fragments, and then transitions the state machine back to UPLOADING to continue uploading from the breakpoint.

[0166] like Figure 7 The diagram shown is a schematic representation of the implementation process of a preset decision tensor update method provided in an embodiment of this application. Figure 7 exist Figure 1 Based on this, a detailed description is provided on how to update the preset decision tensor, which may include the following: S701, obtain the log records corresponding to the file to be uploaded according to the target transmission strategy.

[0167] In this embodiment, log records corresponding to the uploaded files are obtained according to the target transmission strategy. These log records are structured data generated at the end of each file upload and contain at least the following information: Task Identifier: Uniquely identifies an upload task. Context Snapshot: The context vector collected at that time (service code, file size, MIME type, network quality level, etc.). Decision Output: The final selected target transmission strategy, the strategy preference score and probability distribution of each candidate transmission strategy, and the version number of the decision tensor used. Result Indicators: Upload success flag, actual total upload time, number of fragment retransmissions, and whether interruption recovery occurred.

[0168] S702, determine the corresponding profit score for the log record.

[0169] In this embodiment of the application, a revenue score corresponding to the log record can be determined. The revenue score is used to quantify the overall performance of this file upload.

[0170] The revenue score for each log record can be determined using the revenue score formula, which is as follows: .

[0171] As shown in Table 5, Table 5 explains the variables in the above profit score formula.

[0172] Table 5

[0173] S703 updates the preset decision tensor based on the profit score.

[0174] In this embodiment, the preset decision tensor can be updated based on the reward score. Specifically, the reward score can be attributed to the tensor element T corresponding to the actual "dimension-state-action" combination used, based on log records, according to A=Reward. b calculates the advantage difference between the reward score and the average baseline reward, where A is the advantage difference, b is the average baseline reward of recent successful tasks, and Reward is the reward score. Multiply the advantage difference by a preset learning rate (e.g., 0.01) to obtain the gradient increment, and update the preset decision tensor based on the gradient increment.

[0175] As an alternative implementation, if a certain "dimension-state-action" combination produces high returns, its tendency weight is increased; if it produces low returns, its tendency weight is decreased.

[0176] Specifically, it may include the following steps: Step 1: Sample Attribution: For each sample d, the state influence coefficient si(d) and the actual target transmission strategy k(d) are known, as well as the reward score Reward(d). The reward is attributed to the corresponding cell T[i][si(d)][k(d)] in the preset decision tensor.

[0177] Step 2: Gradient increment calculation Calculate the average return of a batch of M samples within the current processing cycle as the baseline. For sample d, calculate the dominance value. .

[0178] The meaning of the advantage value: If A(d)>0, it means that the sample performs better than the average level, and the parameter that leads to the decision should be strengthened; if A(d)<0, it means that the sample performs worse than the average level, and the parameter that leads to the decision should be weakened.

[0179] Calculate the increment: ,in The tensor learning rate (a positive number, such as 0.01).

[0180] Step 3: Batch Accumulation and Smooth Update For each sample d corresponding to the dimension-state-action combination (i, s, k), count all the samples that hit the target in this batch of training samples, and calculate the average increment of the hit samples. Then, the corresponding elements of the decision tensor T are updated using the momentum approach: ,in, This is the momentum decay coefficient (e.g., 0.1). The current value is the decision tensor value before the update. This is the decision tensor value obtained after the previous update.

[0181] Step 4: Boundary Constraints After the update is complete, the decision tensor Enforce nonnegativity constraints: if ,but for ,in, This is a preset, extremely small positive number, such as 0.01. This boundary constraint can prevent zero or negative values ​​from appearing in the decision tensor, ensuring the numerical stability of subsequent probability calculations.

[0182] Step 5: Update the version of the decision tensor T parameters After updating all decision tensors, a new decision tensor matrix is ​​generated, and the version number is incremented according to a preset rule.

[0183] It should be noted that, to prevent the updated parameters from causing online issues, the newly generated version of the decision tensor parameters must undergo the following security verification and release process: 1. Offline playback verification: Upload data from a random period of time is extracted from historical logs to serve as a validation set. In an offline environment, the average simulated performance and other key performance indicators (such as upload success rate) of the new and old parameter versions on this validation set are compared. The release process only proceeds if the performance of the new parameter version is significantly better than or equal to that of the old version.

[0184] 2. Canary Release: Deploy the new parameter version to the online environment, but only apply it to a small portion (e.g., 5%) of newly created upload tasks. Continuously observe for at least 2 hours, monitoring key metrics such as success rate, time consumption, and number of retransmissions.

[0185] 3. Automatic Circuit Breaker Rollback: During the gray-scale release, if the system detects that key indicators (such as upload failure rate, average number of retransmissions) deteriorate and exceed the preset threshold (for example, the total number of upload retries increases by more than 10% compared to the old version), the circuit breaker rollback mechanism will be automatically triggered, instantly switching the parameter version of the decision engine back to the old stable version and issuing an alarm.

[0186] 4. Full rollout: If all indicators are stable and meet expectations during the gray-scale observation period, the traffic ratio of the new parameter version will be gradually increased, for example from 5% to 20%, 50%, until finally 100% full rollout, completing the update of this decision tensor.

[0187] In this application embodiment, the file upload method provided by this application embodiment is described with reference to specific examples: like Figure 8 The diagram shown is a flowchart of a file upload system provided in an embodiment of this application. The file upload system adopts a distributed architecture with client-cloud collaboration. The client is responsible for file acquisition, context awareness, chunked upload, and breakpoint recovery; the cloud consists of multiple functional modules that jointly complete tasks such as rule management, decision calculation, instant upload indexing, upload orchestration, and feedback learning.

[0188] Before executing the following interaction process, the client has completed the collection of context vectors, which are then filtered by the rule center module and calculated by the decision engine module to determine the target transmission strategy (such as direct transmission, bypass transmission, or P2P transmission) for this upload task. Specifically, the interaction process between the client and the cloud upload gateway includes the following three stages: Phase 1: Obtaining Authorization and Establishing a Session Step 1: Client initiates upload request: The client initiates an upload request to the cloud upload gateway, carrying the target transmission strategy (such as direct upload, bypass upload or P2P transmission) in the request.

[0189] Step 2: Request a temporary credential (bypass upload mode): If the target transmission strategy is bypass upload mode, the cloud upload gateway initiates a temporary credential request to the STS credential service.

[0190] Step 3: Return Temporary Credentials: After verifying the legitimacy of the request, the STS credential service uploads temporary credentials to the cloud gateway. The temporary credentials include the access key, security token, and target bucket path.

[0191] Step 4: Return Upload Credentials: The cloud upload gateway returns the obtained temporary credentials to the client. The client uses these temporary credentials to obtain temporary authorization for direct upload to object storage.

[0192] Phase Two: Segmented Upload Step 5: Upload File Shards: The client segments the file to be uploaded according to the target slicing value determined by the adaptive slicing algorithm, resulting in one or more slices. For each slice, the client sends a slice upload request to the cloud upload gateway, carrying a temporary credential, slice sequence number, slice data content, and file offset.

[0193] Step 6: Storage Sharding: After receiving the sharded data, the cloud upload gateway forwards it to the object storage service for persistent storage.

[0194] Step 7: Storage Successful: After the object storage service completes the sharded write, it sends a storage success confirmation message back to the cloud upload gateway.

[0195] Step 8: Return Fragment ETag: After receiving the successful storage confirmation, the cloud upload gateway returns the entity tag corresponding to the fragment to the client. The client associates and stores the entity tag with the fragment metadata for subsequent merge requests.

[0196] Repeat steps 5 through 8 above until all fragments are successfully uploaded.

[0197] Phase 3: Complete Upload and Verification: Step 9: Send a merge request: After all fragments have been uploaded, the client sends a merge request to the cloud upload gateway. The merge request includes a list of the sequence numbers and entity tags of all fragments corresponding to the upload task.

[0198] Step 10: Merge fragments into a complete file: After receiving the merge request, the cloud upload gateway splices the fragments in the object storage into a complete file data block according to the fragment number sequence.

[0199] Step 11: Return file hash and metadata: After the object storage service completes the file merging, it calculates the hash value of the merged complete file and returns the hash value and file metadata to the cloud upload gateway.

[0200] Step 12: Return the final file URL and upload success status: After the cloud upload gateway verifies that the file hash value is consistent with the full hash value reported by the client during the initialization phase, it returns the final file access address and upload success status information to the client, completing this file upload process.

[0201] Corresponding to the above method embodiments, this application also provides a file uploading device, such as... Figure 9 As shown, the device may include a vector determination module 901, a strategy preference score determination module 902, a target transmission strategy determination module 903, and a file upload module 904.

[0202] The vector determination module 901 is used to obtain the file to be uploaded and determine the context vector corresponding to the file to be uploaded; The strategy preference score determination module 902 is used to obtain a set of transmission strategies and, for any candidate transmission strategy in the set of transmission strategies, determine the strategy preference score corresponding to the candidate transmission strategy based on a preset correlation matrix, a preset decision tensor, and a context vector. The target transmission strategy determination module 903 is used to determine the target transmission strategy corresponding to the file to be uploaded based on the strategy preference score corresponding to each candidate transmission strategy. The file upload module 904 is used to upload the file to be uploaded according to the target transmission strategy.

[0203] This application also provides an electronic device, such as... Figure 10 As shown, the system includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004. The processor 1001, communication interface 1002, and memory 1003 communicate with each other via the communication bus 1004. Memory 1003 is used to store computer programs; In one embodiment of this application, when the processor 1001 executes a program stored in the memory 1003, it performs the following steps: Obtain the file to be uploaded and determine the context vector corresponding to the file; obtain a set of transmission strategies, and for any candidate transmission strategy in the set, determine the strategy preference score corresponding to the candidate transmission strategy based on a preset correlation matrix, a preset decision tensor, and the context vector; determine the target transmission strategy corresponding to the file to be uploaded based on the strategy preference scores corresponding to each candidate transmission strategy; upload the file to be uploaded according to the target transmission strategy.

[0204] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0205] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0206] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0207] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0208] In another embodiment provided in this application, a storage medium is also provided, which stores instructions that, when run on a computer, cause the computer to execute any of the file upload methods described in the above embodiments.

[0209] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the file upload methods described in the above embodiments.

[0210] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted from one storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0211] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0212] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0213] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the protection scope of this application.

Claims

1. A file upload method, characterized in that, The method includes: Obtain the file to be uploaded and determine the context vector corresponding to the file to be uploaded; Determining the context vector corresponding to the file to be uploaded includes: parsing the file to be uploaded to obtain the MIME type, obtaining the service identifier code and network quality level, and combining the MIME type, the service identifier code and the network quality level into a dictionary structure to obtain the context vector corresponding to the file to be uploaded; Obtain a set of transmission strategies. For any candidate transmission strategy in the set of transmission strategies, determine the strategy preference score corresponding to the candidate transmission strategy based on a preset correlation matrix, a preset decision tensor, and the context vector. Based on a preset correlation matrix, a preset decision tensor, and the context vector, the strategy preference score corresponding to the candidate transmission strategy is determined, including: for any feature in the context vector, determining the discretized state corresponding to the feature; obtaining the state influence coefficient corresponding to the discretized state; obtaining the basic weight parameter corresponding to the feature from a preset weight table; using the product of the basic weight parameter corresponding to the feature and the state influence coefficient as the basic magnitude corresponding to the feature; correcting the basic magnitude corresponding to the feature according to the preset correlation matrix to obtain the corrected magnitude corresponding to the feature; and determining the strategy preference score corresponding to the candidate transmission strategy based on the preset decision tensor and the corrected magnitude corresponding to each feature. The preset correlation matrix is ​​used to characterize the coupling influence strength between the dimensions of the context vector; the preset decision tensor contains the basic tendency weights of each candidate transmission strategy; Based on the policy preference scores corresponding to each of the candidate transmission strategies, the target transmission strategy corresponding to the file to be uploaded is determined. Upload the file to be uploaded according to the target transmission strategy.

2. The method according to claim 1, characterized in that, Before obtaining the set of transmission policies, the following is included: Determine the file size of the file to be uploaded; Construct a query key based on the file size of the file to be uploaded; Based on the query key, perform a first-level verification on the file to be uploaded and generate a first verification result; If the first verification result is a hit, a second verification is performed on the file to be uploaded to generate a second verification result; If the second verification result is inconsistent, the step of obtaining the transmission strategy set is executed.

3. The method according to claim 2, characterized in that, A query key is constructed based on the file size of the file to be uploaded, including: If the file size of the file to be uploaded is less than or equal to a preset length threshold, then the hash value of the full file is determined based on the file content corresponding to the file to be uploaded. The query key is constructed based on the hash value of the full file and the file size of the file to be uploaded. If the file size of the file to be uploaded is greater than the preset length threshold, then the header file content and the footer file content are extracted from the file content corresponding to the file to be uploaded. Determine the header file hash value corresponding to the header file content and the tail file hash value corresponding to the tail file content; The query key is constructed based on the header file hash value corresponding to the header file content, the tail file hash value corresponding to the tail file content, and the file size of the file to be uploaded.

4. The method according to claim 1, characterized in that, Obtain the set of transmission policies, including: The rules in the preset rule list are traversed to determine at least one candidate rule that matches the context vector. Obtain the transmission strategy corresponding to each candidate rule, and determine the transmission strategy set based on the transmission strategy corresponding to each candidate rule.

5. The method according to claim 4, characterized in that, Obtain the transmission strategy corresponding to each of the candidate rules, and determine a transmission strategy set based on the transmission strategy corresponding to each of the candidate rules, including: Based on the priority of each candidate rule, the target rule is determined from each candidate rule; Obtain the transmission policy constraints corresponding to the target rule, wherein the transmission policy constraints include a mandatory allow list and / or a mandatory prohibit list; Based on the mandatory allow list and / or mandatory prohibit list, the transmission policies corresponding to the target rules are retained and / or eliminated to obtain the transmission policy set.

6. The method according to claim 1, characterized in that, Based on the preset decision tensor and the correction magnitude corresponding to each feature, the policy preference score corresponding to the candidate transmission strategy is determined, including: Obtain the default value corresponding to the candidate transmission strategy from the preset decision tensor; For any of the aforementioned features, the product of the correction magnitude corresponding to the feature and the default value corresponding to the candidate transmission strategy is determined as the feature tendency component corresponding to the feature. Based on the feature tendency components corresponding to each of the aforementioned features, the strategy tendency score corresponding to the candidate transmission strategy is determined.

7. The method according to claim 1, characterized in that, Uploading the file to be uploaded according to the target transmission strategy includes: Obtain real-time network metrics, including uplink transmission rate, round-trip time, device CPU utilization, and packet loss rate; The initial slice value is determined based on the aforementioned real-time network metrics; Get historical slice values; Candidate slice values ​​are determined based on the historical slice values ​​and the initial slice values; The target slice value is determined based on the preset amplitude limiting function and the candidate slice values; The file to be uploaded is split according to the target slice value to obtain at least one slice; Upload each of the aforementioned fragments according to the target transmission strategy.

8. The method according to claim 1, characterized in that, After uploading the file to be uploaded according to the target transmission strategy, the process further includes: Obtain the log records corresponding to the file to be uploaded according to the target transmission strategy; Determine the revenue score corresponding to the log record; The preset decision tensor is updated based on the profit score.

9. The method according to claim 1, characterized in that, Based on the policy preference scores corresponding to each of the candidate transmission strategies, the target transmission strategy corresponding to the file to be uploaded is determined, including: Obtain a set of correction factors, which includes at least one of the following: strategy type preference coefficient, cost coefficient, business scenario coefficient, device status coefficient, real-time requirement coefficient, and security level coefficient; The policy preference scores corresponding to each candidate transmission strategy are corrected according to the set of correction factors to obtain the corrected policy preference scores corresponding to each candidate transmission strategy. Based on the modified policy preference score corresponding to each of the candidate transmission strategies, the target transmission strategy corresponding to the file to be uploaded is determined.

10. A file upload device, characterized in that, The device includes: The vector determination module is used to obtain the file to be uploaded and determine the context vector corresponding to the file to be uploaded; Determining the context vector corresponding to the file to be uploaded includes: parsing the file to be uploaded to obtain the MIME type, obtaining the service identifier code and network quality level, and combining the MIME type, the service identifier code and the network quality level into a dictionary structure to obtain the context vector corresponding to the file to be uploaded; The strategy preference score determination module is used to obtain a set of transmission strategies and, for any candidate transmission strategy in the set of transmission strategies, determine the strategy preference score corresponding to the candidate transmission strategy based on a preset correlation matrix, a preset decision tensor, and the context vector. Based on a preset correlation matrix, a preset decision tensor, and the context vector, the strategy preference score corresponding to the candidate transmission strategy is determined, including: for any feature in the context vector, determining the discretized state corresponding to the feature; obtaining the state influence coefficient corresponding to the discretized state; obtaining the basic weight parameter corresponding to the feature from a preset weight table; using the product of the basic weight parameter corresponding to the feature and the state influence coefficient as the basic magnitude corresponding to the feature; correcting the basic magnitude corresponding to the feature according to the preset correlation matrix to obtain the corrected magnitude corresponding to the feature; and determining the strategy preference score corresponding to the candidate transmission strategy based on the preset decision tensor and the corrected magnitude corresponding to each feature. The preset correlation matrix is ​​used to characterize the coupling influence strength between the dimensions of the context vector; the preset decision tensor contains the basic tendency weights of each candidate transmission strategy; The target transmission strategy determination module is used to determine the target transmission strategy corresponding to the file to be uploaded based on the strategy preference score corresponding to each of the candidate transmission strategies. The file upload module is used to upload the file to be uploaded according to the target transmission strategy.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images