Safety detection method for model service and related equipment

Abstract

The invention provides a model service security detection method and related equipment. The method comprises the following steps: acquiring cache information of a first model service, wherein the cache information is generated based on a first request sent to the first model service; and performing security detection on the first model service based on the life cycle of the cache information to obtain a security detection result for cache sharing of the first model service. According to the security detection method for the model service and the related equipment, security detection can be carried out on the KV cache sharing process in the model service, an effective security analysis mode is provided for a provider of the model service, and potential safety hazards in the KV cache sharing process in the model service are eliminated.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a security detection method for a model service and related equipment. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] With the continuous development of artificial intelligence technology, model services have demonstrated superior performance in conversational question and answer, text generation, language translation, and other aspects.

[0004] During the model reasoning process, the key-value cache (KV cache) sharing method is used to save the computing time and computing resources of the model reasoning process.

[0005] However, this method has certain security issues. Summary of the invention

[0006] In view of this, the purpose of the present disclosure is to propose a security detection method and related equipment for model services, which at least solves one of the technical problems in the related technology to a certain extent.

[0007] Based on the above objectives, the first aspect of the present disclosure provides a security detection method for a model service, comprising:

[0008] Obtain cache information of a first model service, where the cache information is generated based on a first request sent to the first model service;

[0009] A security check is performed on the first model service based on the life cycle of the cache information to obtain a security check result of cache sharing for the first model service.

[0010] In some embodiments, performing security detection on the first model service based on the life cycle of the cache information to obtain a security detection result of cache sharing for the first model service includes:

[0011] If the life cycle of the cache information is extended due to user reasons, it is determined that the security test result of the cache sharing of the first model service is failed.

[0012] In some embodiments, the method further includes: obtaining a second request sent by a user to the first model service;

[0013] If the life cycle of the cache information is extended due to user reasons, determining that the security detection result of the cache sharing of the first model service is failed includes:

[0014] Obtaining the life cycle of the cache information;

[0015] If the life cycle of the cache information is greater than a preset life cycle threshold, determining the abnormal reason for the life cycle of the cache information;

[0016] If the cause of the exception includes that the life cycle of the cache information is extended based on the second request, it is determined that the security check result of the cache sharing of the first model service is failed.

[0017] In some embodiments, the method further comprises:

[0018] In response to the abnormal reason including that the cache information is not cleared based on a preset clearing command, it is determined that a security check result of the cache sharing of the first model service is failed.

[0019] In some embodiments, it also includes:

[0020] Obtaining a third request sent by the user to the first model service;

[0021] Based on the return information of the third request, a security check is performed on the first model service to obtain a security check result.

[0022] In some embodiments, performing security detection on the first model service based on the return information of the third request to obtain a security detection result includes:

[0023] Obtaining a request processing time of the third request;

[0024] If within a first preset time period, the number of the third requests belonging to the same user is greater than a first preset number and the corresponding request processing time is less than a preset time threshold, it is determined that the security detection result of the cache sharing of the first model service is failed.

[0025] In some embodiments, performing security detection on the first model service based on the return information of the third request to obtain a security detection result includes:

[0026] Obtaining return information of the third request belonging to multiple users;

[0027] If within a second preset time period, the order of the returned information of the third request belonging to the target user among multiple users changes and the number of the third requests whose order changes is greater than a second preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

[0028] In some embodiments, it also includes:

[0029] Obtaining a fourth request sent by the user to the first model service;

[0030] A security check is performed on the first model service based on the request content of the fourth request to obtain a security check result.

[0031] In some embodiments, performing a security check on the first model service based on the request content of the fourth request to obtain a security check result includes:

[0032] Obtaining request content of the fourth request;

[0033] If within a third preset time period, the request contents of the fourth requests belonging to the same user are the same or similar, and / or the number of the fourth requests is greater than the third preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

[0034] In some embodiments, the performing security detection on the first model service based on the request content of the fourth request to obtain a security detection result includes:

[0035] Obtaining request content of the fourth request;

[0036] Obtaining user-set request parameters in the request content;

[0037] In response to the user setting request parameter satisfying a preset condition, it is determined that a security detection result of the cache sharing of the first model service is failed.

[0038] In some embodiments, the user setting request parameter satisfies a preset condition, including at least one of the following:

[0039] The length of the returned information of the fourth request is smaller than a preset value;

[0040] The cache information generated based on the fourth request is not stored in the first model service.

[0041] Based on the same inventive concept, the second aspect of the present disclosure provides a security detection device for a model service, comprising:

[0042] An acquisition module is configured to: acquire cache information of a first model service, where the cache information is generated based on a first request sent to the first model service;

[0043] The detection module is configured to: perform security detection on the first model service based on the life cycle of the cache information, and obtain a security detection result of the cache sharing of the first model service.

[0044] Based on the same inventive concept, the third aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0045] Based on the same inventive concept, the fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0046] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of the present disclosure provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the method described in the first aspect.

[0047] From the above, it can be seen that the security detection method and related equipment of the model service provided by the present invention perform a security detection on the first model service based on the life cycle of the existing cache information in the first model service, thereby obtaining a security detection result for the cache sharing of the first model service, and then perform a security detection on the KV cache sharing process in the model service, providing an effective security analysis method for the provider of the model service, and eliminating security risks in the KV cache sharing process in the model service. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 An exemplary schematic diagram provided by an embodiment of the present disclosure is shown.

[0050] Figure 2 A flowchart of an exemplary method provided by an embodiment of the present disclosure is shown.

[0051] Figure 3 A schematic diagram of an exemplary device provided by an embodiment of the present disclosure is shown.

[0052] Figure 4 A schematic diagram of the hardware structure of an exemplary computer device provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0054] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0055] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0056] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0057] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0058] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0059] In the inference process of the model service, when a series of input tokens are given, the model service will gradually generate output tokens through autoregression. The inference process consists of two stages: the prefill stage and the incremental decoding stage.

[0060] During the pre-population phase, the model server processes all input tokens simultaneously and generates the first output tokens.

[0061] During the incremental decoding phase, the model service gradually generates subsequent output tokens, and each new output depends on the previously generated tokens to build a complete response.

[0062] Key-Value Cache (KV Cache) is a key data structure generated during the model service reasoning process. A corresponding KV cache is generated for each word segment generated in the pre-filling phase and incremental decoding phase, which is used for reference when decoding subsequent word segments.

[0063] The features of KV cache include:

[0064] Calculation dependency: The calculation of KV cache depends on all previous word segmentations. The same word segmentation may generate different KV caches in different contexts. For example, in the sentence "How do you do?", the two "do" will generate different KV caches.

[0065] Consistency: When the leading tokens are the same, the generated KV cache is also the same. This can be seen in sentences like “I really enjoy coding” and “I really enjoy debugging”, where “I”, “really”, and “enjoy” in the same model generate the same KV cache.

[0066] Memory consumption: KV cache is a significant bottleneck in model serving because it consumes a lot of memory resources.

[0067] KV cache plays a vital role in model service reasoning, but it also brings challenges in computing and resource management, especially in multi-user and high-concurrency scenarios.

[0068] In a high-concurrency multi-user scenario, when all the pre-participles in the requests of multiple users are the same, the generated KV cache is also the same. In this case, through the KV cache sharing method, subsequent requests can directly use the previously calculated KV cache, thus saving the time and resources of recalculation.

[0069] The key to KV cache sharing is to ensure that the prefix word between requests is fully matched. This match ensures that the KV cache can be reused in different requests, improving computing efficiency.

[0070] However, the inventors of this application discovered that although KV cache sharing improves computing efficiency and reduces memory consumption, there are overlooked security issues: KV cache sharing creates a shared channel between multiple users, and users can infer questions asked by other users to the model service through this shared channel, thereby causing security issues.

[0071] like Figure 1 As shown, the first user sends a request to the model service. When the model service processes the request of the first user, a corresponding KV cache is generated in the model service. The KV cache stores the status information of the request of the first user.

[0072] The second user can carefully design and send a request to the model service, with the purpose of trying to reuse the first user's KV cache. If the second user's request reuses the first user's KV cache, the second user's return information will carry side channel information, where the side channel information may include, for example, the response time and power consumption of the model service. The second user can record and construct a new request based on the side channel information and send it to the model service again, and further analyze it based on the side channel information carried in the return information.

[0073] By analyzing the side channel information, the second user can determine that the word segmentation sequence input by himself is the same as or at least partially the same as the word segmentation sequence input by the first user, and then can know the word segmentation sequence input by the first user, which affects the security of the model service.

[0074] In view of this, an embodiment of the present disclosure provides a security detection method for a model service, which performs security detection on the KV cache sharing process in the model service, provides an effective security analysis method for the provider of the model service, and eliminates security risks in the KV cache sharing process in the model service.

[0075] like Figure 2 As shown, the security detection method of the model service includes:

[0076] Step S101: Obtain cache information of a first model service, where the cache information is generated based on a first request sent to the first model service.

[0077] The cache information of the first model service includes the KV cache of the first model service, and the cache information is generated based on the first request sent to the first model service.

[0078] In this embodiment, the existing cache information of the first model service may be obtained, and the security of the first model service in the KV cache sharing process may be detected based on the existing cache information.

[0079] Step S103: Perform a security check on the first model service based on the life cycle of the cache information to obtain a security check result of cache sharing for the first model service.

[0080] In this embodiment, a security check may be performed on the first model service based on the life cycle of the existing cache information in the first model service, thereby obtaining a security check result for the cache sharing of the first model service.

[0081] In this embodiment, a security check is performed on the first model service based on the life cycle of the existing cache information in the first model service, so as to obtain a security check result for the cache sharing of the first model service, thereby performing a security check on the KV cache sharing process in the model service, providing an effective security analysis method for the provider of the model service, and eliminating security risks in the KV cache sharing process in the model service.

[0082] In some embodiments, the first model service is security checked based on the life cycle of the cache information to obtain a security check result for the cache sharing of the first model service, including: if the life cycle of the cache information is extended due to user reasons, determining that the security check result for the cache sharing of the first model service is failed.

[0083] In this embodiment, if the life cycle of the cached information is extended due to user reasons, it means that the user has been trying to reuse the cached information. This reuse behavior may be caused by the user's attack on the cached information, so there is a security risk. In this case, the security test result of the cache sharing of the first model service is determined to have failed.

[0084] In some embodiments, the method further includes: obtaining a second request sent by the user to the first model service; if the life cycle of the cache information is extended due to user reasons in step S103, determining that the security detection result of the cache sharing of the first model service is failed includes:

[0085] Step S201, obtaining the life cycle of the cache information.

[0086] In this embodiment, when the life cycle of the KV cache exceeds the life cycle it should exist, a security problem of KV cache sharing may occur. Therefore, the potential security risk can be determined by monitoring the life cycle of the KV cache.

[0087] Step S203: If the life cycle of the cache information is greater than a preset life cycle threshold, determine the abnormal reason for the life cycle of the cache information.

[0088] In this embodiment, the life cycle threshold of the KV cache in the first model service can be pre-set. The life cycle thresholds of the KV cache in different first model services can be different; the life cycle thresholds of the KV cache in the same first model service can be the same or different, which is not limited in this embodiment.

[0089] In this embodiment, the generation cycle of the KV cache corresponding to each first request can be compared with the preset life cycle threshold in real time to obtain a KV cache with an abnormal life cycle. When the life cycle of the KV cache is greater than the preset life cycle threshold, it means that the life cycle of the KV cache is abnormal. At this time, it is necessary to further determine the abnormal cause of the life cycle of the KV cache.

[0090] Step S205: If the abnormal reason includes that the life cycle of the cache information is extended based on the second request, it is determined that the security detection result of the cache sharing of the first model service is failed.

[0091] In this embodiment, the source of the KV cache with an abnormal life cycle is traced back to determine the cause of the abnormal life cycle. If the life cycle abnormality is caused by system reasons, it can be ignored; if the life cycle abnormality is caused by a user request, for example, the user repeatedly sends the same or similar second request, causing the KV cache to be reused and the life cycle to be extended, then the user may be trying to obtain the information corresponding to the KV request, and there is a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service is failed.

[0092] In some embodiments, the method further includes: in response to the abnormal cause including cache information not being cleared based on a preset clearing command, determining that a security check result of the cache sharing of the first model service is failed.

[0093] It is also possible to monitor whether the KV cache can be cleared normally. The first model service may configure the clearing conditions of the KV cache, such as scheduled clearing, clearing when the memory usage exceeds the preset value, etc. If the KV cache should be cleared but is not cleared, it may be that the user has prevented the execution of the clearing process in some way, which poses a security risk to KV cache sharing.

[0094] In this embodiment, the log information corresponding to the KV cache with an abnormal life cycle can be queried, and the log information can be used to determine whether the clear command for the KV cache has been awakened or the clear command has been awakened but not executed. When it is detected that the clear command for the KV cache has not been awakened or the clear command has been awakened but not executed normally, there is a security risk of KV cache sharing, so it can be determined that the security test result of the cache sharing of the first model service is failed.

[0095] In some embodiments, the method further includes: obtaining a third request sent by the user to the first model service; performing a security check on the first model service based on return information of the third request to obtain a security check result.

[0096] In this embodiment, a security check may also be performed on the first model service based on the return information of the third request to obtain a security check result.

[0097] In some embodiments, performing security detection on the first model service based on the return information of the third request to obtain a security detection result includes:

[0098] Step S301, obtaining the request processing time of the third request.

[0099] In this embodiment, the request processing time may include the time between sending the third request to the first model service and the time between the first model service returning the return information of the third request, or it may be the time for the first model service to process the third request and generate the return information of the third request. This embodiment does not limit this.

[0100] Step S303, if within the first preset time period, the number of the third requests belonging to the same user is greater than the first preset number and the corresponding request processing time is less than the preset time threshold, it is determined that the security detection result of the cache sharing of the first model service is failed.

[0101] In this embodiment, the return information of each third request can be analyzed in real time. If the request processing time of the first model service for processing the third request is significantly shortened, for example, less than a preset time threshold, then the third request has KV cache sharing.

[0102] If within a time period, for example, within a first preset time period, the same user sends a large number of third requests, for example, the number of third requests is greater than the first preset number, and the request processing time of these third requests is significantly shortened, then it may be that the user is continuously trying to reuse the KV cache, and there is a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service is failed.

[0103] Among them, the first preset time period can be set according to the requirements of the first model service and the requirements of the user. When the first model service is a high-concurrency model service, that is, the first model service will process a large number of user requests in a short time, the first preset time period can be set to a shorter duration, such as a few minutes, one hour, two hours, etc.; when the first model service is a low-concurrency model service, the first preset time period can be set to a longer duration, such as several hours, one day, several days, etc. This embodiment does not limit this.

[0104] In some embodiments, performing security detection on the first model service based on the return information of the third request to obtain a security detection result includes:

[0105] Step S401: Acquire return information of the third request belonging to multiple users.

[0106] Step S403, if within a second preset time period, the order of the returned information of the third request belonging to the target user among multiple users changes and the number of the third requests whose order has changed is greater than a second preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

[0107] In this embodiment, the return information of each third request can be analyzed in real time. If the order of the requests changes when the first model service processes the third requests of multiple users, there is a situation of KV cache sharing.

[0108] Here, assuming that users a, b, and c send the third request to the first model service in sequence, then under normal circumstances, the return information of the first model service is also returned in the order of a, b, and c.

[0109] When the order of the returned information of the first model service changes, for example, it is returned in the order of c, a, and b, it indicates that the second request sent by user c has a KV cache sharing situation.

[0110] If within a time period, for example, within the second preset time period, the same user, for example, user c, sends a large number of third requests, for example, the number of third requests is greater than the second preset number, and the return information of these third requests has changed in order, it may be that user c is continuously trying to reuse the KV cache, and there is a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service is failed.

[0111] Among them, the second preset time period can be set according to the requirements of the first model service and the requirements of the user. When the first model service is a high-concurrency model service, that is, the first model service will process a large number of user requests in a short time, the second preset time period can be set to a shorter duration, such as a few minutes, one hour, two hours, etc.; when the first model service is a low-concurrency model service, the second preset time period can be set to a longer duration, such as several hours, one day, several days, etc. This embodiment does not limit this.

[0112] In some embodiments, the method further includes: obtaining a fourth request sent by the user to the first model service; performing a security check on the first model service based on the request content of the fourth request to obtain a security check result.

[0113] In this embodiment, a security check may also be performed on the first model service based on the request content of the fourth request to obtain a security check result.

[0114] In some embodiments, the performing security detection on the first model service based on the request content of the fourth request to obtain a security detection result includes:

[0115] Step S501: Obtain the request content of the fourth request.

[0116] In this embodiment, the request content of each fourth request may be analyzed in real time to determine whether the user attempts to use KV cache sharing to endanger the security of the first model service.

[0117] Step S503, if within the third preset time period, the request contents of the fourth requests belonging to the same user are the same or similar, and / or the number of the fourth requests is greater than the third preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

[0118] In this embodiment, by performing real-time analysis on the request content of the fourth request sent by the user, if the same user continuously sends the same or similar requests within a time period, such as the third preset time period, it may be that the user is continuously trying to reuse the KV cache, and there is a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service is failed.

[0119] For example, when the request contents of multiple fourth requests of a user are similar contents such as "How do you do?" "How do you?" "How you do?" "What do you do?", the user may continue to try to reuse the corresponding KV cache, which may lead to a security risk of KV cache sharing.

[0120] The similarity between the request contents of the plurality of fourth requests may be calculated, and when the similarity is greater than a preset threshold, it is determined that the plurality of fourth requests are similar requests.

[0121] In this embodiment, by performing real-time analysis on the request content of the fourth request sent by the user, if the same user sends a large number of requests within a time period, such as the third preset time period, it is possible that the user is continuously trying to reuse the KV cache, and there is a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service is failed.

[0122] In this embodiment, by performing real-time analysis on the request content of the fourth request sent by the user, if the same user continuously sends a large number of identical or similar requests within a time period, such as the third preset time period, it may be that the user is continuously trying to reuse the KV cache, and there is a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service is failed.

[0123] Among them, the third preset time period can be set according to the requirements of the first model service and the requirements of the user. When the first model service is a high-concurrency model service, that is, the first model service will process a large number of user requests in a short time, the third preset time period can be set to a shorter duration, such as a few minutes, one hour, two hours, etc.; when the first model service is a low-concurrency model service, the third preset time period can be set to a longer duration, such as several hours, one day, several days, etc. This embodiment does not limit this.

[0124] In some embodiments, the performing security detection on the first model service based on the request content of the fourth request to obtain a security detection result includes:

[0125] Step S601: Obtain the request content of the fourth request.

[0126] Step S603: obtaining the user setting request parameters in the request content.

[0127] Step S605: In response to the user setting request parameter satisfying a preset condition, determining that a security detection result of the cache sharing of the first model service is failed.

[0128] In this embodiment, the request content of each fourth request can be analyzed in real time to determine whether there are some unreasonable, uncommon or extreme request parameter values ​​in the request content of the fourth request; if there are unreasonable, uncommon or extreme request parameter values, it may be that the user is trying to use KV cache sharing to endanger the security of the first model service.

[0129] Therefore, in this embodiment, the user-set request parameters in the request content of the fourth request can be obtained, and the user-set request parameters can be compared with the preset conditions. If the preset conditions are met, it indicates whether there are unreasonable, uncommon or extreme request parameter values, and there is a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service is failed.

[0130] Among them, the user sets the request parameters to meet preset conditions, including at least one of the following: the length of the returned information of the fourth request is less than a preset value; the cache information generated based on the fourth request is not stored in the first model service.

[0131] Usually, users hope that the model service can provide them with more detailed, more accurate, and richer return information. If the request content of the request sent by the user contains a very small amount of return information in the attached user-set request parameter, for example, the length of the return information of the fourth request is less than the preset value, it means that the user may not be using the model service normally, but is trying to reuse the KV cache, which may lead to a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service has failed.

[0132] Normally, when users send requests to a model service, they usually do not consider whether their requests will generate a KV cache in the model service. If the request content of the request sent by the user sets the request parameter attached to the user setting that the request does not store the corresponding KV cache in the first model service, the user may be trying to reuse the KV cache and does not want the KV cache corresponding to his request to be stored, which will lead to a security risk of KV cache sharing. Therefore, it can be determined that the security test result of the cache sharing of the first model service has failed.

[0133] In some embodiments, the method further includes: generating an alarm message based on the security check result; the alarm message includes the request content of the fourth request for which the security check result fails, the return information of the third request, the user information to which the request belongs, and at least one of the alarm reasons.

[0134] In this embodiment, when the security detection result is failed, an alarm message may be generated.

[0135] The alarm information may include at least one of the request content of the fourth request whose security detection result is failed, the return information of the third request, the user information to which the request belongs, and the alarm reason. The alarm reason may include that the life cycle of the KV cache is extended due to user request or abnormal clearing, the user reuses the KV cache multiple times within a preset time, the user sends the same or similar request multiple times within a preset time to try to reuse the KV cache, and the request content includes abnormal parameters.

[0136] In this embodiment, the alarm information can provide the user with the users, content, location in the model service, specific reasons, etc. that pose security risks to KV cache sharing, so that the user can perform further analysis and processing based on the alarm information to maintain the security of the first model service.

[0137] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0138] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0139] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a security detection device for a model service.

[0140] refer to Figure 3 , the security detection device of the model service comprises:

[0141] An acquisition module 11 is configured to: acquire cache information of a first model service, where the cache information is generated based on a first request sent to the first model service;

[0142] The detection module 13 is configured to: perform a security detection on the first model service based on the life cycle of the cache information, and obtain a security detection result of the cache sharing of the first model service.

[0143] In some embodiments, the detection module 13 is configured to: if the life cycle of the cache information is extended due to user reasons, determine that the security detection result of the cache sharing of the first model service is failed.

[0144] In some embodiments, the method further includes: obtaining a second request sent to the first model service; the detection module 13 is further configured to:

[0145] Obtaining the life cycle of the cache information;

[0146] If the life cycle of the cache information is greater than a preset life cycle threshold, determining the abnormal reason for the life cycle of the cache information;

[0147] If the cause of the exception includes that the life cycle of the cache information is extended based on the second request, it is determined that the security check result of the cache sharing of the first model service is failed.

[0148] In some embodiments, the device is further configured to: in response to the abnormal reason including that the cache information is not cleared based on a preset clearing command, determine that the security detection result of the cache sharing of the first model service is failed.

[0149] In some embodiments, the detection module 13 is further configured to:

[0150] Obtaining a third request sent by the user to the first model service;

[0151] Based on the return information of the third request, a security check is performed on the first model service to obtain a security check result.

[0152] In some embodiments, the detection module 13 is further configured to:

[0153] Obtaining a request processing time of the third request;

[0154] If within a first preset time period, the number of the third requests belonging to the same user is greater than a first preset number and the corresponding request processing time is less than a preset time threshold, it is determined that the security detection result of the cache sharing of the first model service is failed.

[0155] In some embodiments, the detection module 13 is further configured to:

[0156] Obtaining return information of the third request belonging to multiple users;

[0157] If within a second preset time period, the order of the returned information of the third request belonging to the target user among multiple users changes and the number of the third requests whose order changes is greater than a second preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

[0158] In some embodiments, the detection module 13 is further configured to:

[0159] Obtaining a fourth request sent by the user to the first model service;

[0160] A security check is performed on the first model service based on the request content of the fourth request to obtain a security check result.

[0161] In some embodiments, the detection module 13 is further configured to:

[0162] Obtaining request content of the fourth request;

[0163] If within a third preset time period, the request contents of the fourth requests belonging to the same user are the same or similar, and / or the number of the fourth requests is greater than the third preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

[0164] In some embodiments, the detection module 13 is further configured to:

[0165] Obtaining request content of the fourth request;

[0166] Obtaining user-set request parameters in the request content;

[0167] In response to the user setting request parameter satisfying a preset condition, it is determined that a security detection result of the cache sharing of the first model service is failed.

[0168] In some embodiments, the user setting request parameter satisfies a preset condition, including at least one of the following:

[0169] The length of the returned information of the fourth request is smaller than a preset value;

[0170] The cache information generated based on the fourth request is not stored in the first model service.

[0171] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0172] The device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0173] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0174] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0175] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0176] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0177] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0178] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0179] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0180] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0181] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0182] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0183] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0184] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0185] Based on the same inventive concept, corresponding to the method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the method. Corresponding to the execution subject corresponding to each step in each embodiment of the method, the processor that executes the corresponding step can belong to the corresponding execution subject.

[0186] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0187] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." In addition, in some embodiments, the present disclosure may also be implemented in the form of a computer program product in one or more computer-readable media, which contains computer-readable program code.

[0188] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive examples) of computer-readable storage media may include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0189] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0190] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0191] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0192] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable model service safety protection device, thereby producing a machine, and these computer program instructions are executed by the computer or other programmable model service safety protection device to produce a device that implements the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0193] These computer program instructions can also be stored in a computer-readable medium that enables a computer or other programmable model service security protection device to operate in a specific manner. In this way, the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0194] Computer program instructions may also be loaded onto a computer, other programmable model service security protection device, or other device so that a series of operating steps are executed on the computer, other programmable model service security protection device, or other device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device can provide a process for implementing the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0195] In addition, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0196] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0197] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0198] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0199] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0200] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0201] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A security detection method for a model service, comprising: Obtain cache information of a first model service, where the cache information is generated based on a first request sent to the first model service; A security check is performed on the first model service based on the life cycle of the cache information to obtain a security check result of cache sharing for the first model service.

2. The method according to claim 1, wherein: Performing a security check on the first model service based on the life cycle of the cache information to obtain a security check result for cache sharing of the first model service includes: If the life cycle of the cache information is extended due to user reasons, it is determined that the security detection result of the cache sharing of the first model service is failed.

3. The method according to claim 2, wherein: Also includes: Obtaining a second request sent by the user to the first model service; If the life cycle of the cache information is extended due to user reasons, determining that the security detection result of the cache sharing of the first model service is failed includes: Obtaining the life cycle of the cache information; If the life cycle of the cache information is greater than a preset life cycle threshold, determining the abnormal reason for the life cycle of the cache information; If the cause of the exception includes that the life cycle of the cache information is extended based on the second request, it is determined that the security check result of the cache sharing of the first model service is failed.

4. The method according to claim 3, further comprising: In response to the abnormal reason including that the cache information is not cleared based on a preset clearing command, it is determined that a security check result of the cache sharing of the first model service is failed.

5. The method according to claim 1, further comprising: Obtaining a third request sent by the user to the first model service; Based on the return information of the third request, a security check is performed on the first model service to obtain a security check result.

6. The method according to claim 5, wherein: The performing security detection on the first model service based on the return information of the third request to obtain a security detection result includes: Obtaining a request processing time of the third request; If within a first preset time period, the number of the third requests belonging to the same user is greater than a first preset number and the corresponding request processing time is less than a preset time threshold, it is determined that the security detection result of the cache sharing of the first model service is failed.

7. The method according to claim 5, wherein: The performing security detection on the first model service based on the return information of the third request to obtain a security detection result includes: Obtaining return information of the third request belonging to multiple users; If within a second preset time period, the order of the returned information of the third request belonging to the target user among multiple users changes and the number of the third requests whose order changes is greater than a second preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

8. The method according to claim 1, further comprising: Obtaining a fourth request sent by the user to the first model service; A security check is performed on the first model service based on the request content of the fourth request to obtain a security check result.

9. The method according to claim 8, wherein: The performing security detection on the first model service based on the request content of the fourth request to obtain a security detection result includes: Obtaining request content of the fourth request; If within a third preset time period, the request contents of the fourth requests belonging to the same user are the same or similar, and / or the number of the fourth requests is greater than the third preset number, it is determined that the security check result of the cache sharing of the first model service is failed.

10. The method according to claim 8, wherein: The performing security detection on the first model service based on the request content of the fourth request to obtain a security detection result includes: Obtaining request content of the fourth request; Obtaining user-set request parameters in the request content; In response to the user setting request parameter satisfying a preset condition, it is determined that a security detection result of the cache sharing of the first model service is failed.

11. The method according to claim 10, wherein: The user setting request parameter satisfies a preset condition, including at least one of the following: The length of the returned information of the fourth request is smaller than a preset value; The cache information generated based on the fourth request is not stored in the first model service.

12. A security detection device for a model service, comprising: An acquisition module is configured to: acquire cache information of a first model service, where the cache information is generated based on a first request sent to the first model service; The detection module is configured to: perform security detection on the first model service based on the life cycle of the cache information, and obtain a security detection result of the cache sharing of the first model service.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 11 when executing the program.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 11.

15. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 11.

Images