Crawler identification method and device, vehicle and readable storage medium
By identifying and intercepting network crawlers on the vehicle, the load pressure problem caused by frequent access to the server was solved, and the data acquisition of normal users' data obtaining smoothness and privacy security.
Patent Information
- Application Number
- CN202411738382.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-01-24
- Estimated Expiration
- 2044-11-29
AI Technical Summary
Frequent access to the server by network crawlers leads to high load pressure on the server, reducing the fluency of normal users to obtain data.
Implement the reptile recognition method on the vehicle side. By obtaining the vehicle system and application data of the vehicle, the identification data request is requested by the reptile request, and intercept the request when determining the reptile request to reduce the server load.
It effectively reduces the load pressure of crawler requests on the server, ensures the smoothness of data acquisition for normal users, and prevents crawler users from maliciously obtaining user privacy information.
Smart Images

Figure CN119363475A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and more specifically, to a crawler identification method, device, vehicle and readable storage medium in the field of computer technology. Background Art
[0002] A web crawler is a computer program that searches for web pages through their link addresses and automatically obtains the content of web pages according to certain rules. Since web crawlers do not need to render pages, but only need to obtain file contents and the Uniform Resource Locator (URL) in the files, web crawlers can access servers at a very high frequency, which increases the load pressure on the servers and reduces the smoothness of data acquisition for normal users.
[0003] Therefore, how to reduce the load pressure on the server is an urgent problem that needs to be solved. Summary of the invention
[0004] The present application provides a crawler identification method, device, vehicle and readable storage medium, which can identify and intercept web crawler requests, thereby reducing the access frequency of web crawler requests and further reducing the load pressure on the server.
[0005] In a first aspect, a crawler identification method is provided, which is applied to a vehicle computer end, and the method includes: in response to a data request sent by a vehicle computer application to a server, obtaining system data of the vehicle computer system and / or application data of the vehicle computer application; identifying whether the data request is a crawler request based on the system data of the vehicle computer system and / or application data of the vehicle computer application; and when it is determined that the data request is a crawler request, intercepting the data request so that the data request is not sent to the server.
[0006] In the above technical scheme, an embodiment of the present application provides a crawler identification method: the vehicle-mounted end can respond to the data request sent by the vehicle-mounted application of the vehicle to the server, obtain the system data of the vehicle-mounted system of the vehicle and / or the application data of the vehicle-mounted application of the vehicle, and identify whether the data request is a crawler request based on the system data of the vehicle-mounted system of the vehicle and / or the application data of the vehicle-mounted application of the vehicle. Since the embodiment of the present application will intercept the data request when the vehicle-mounted end determines that the data request is a crawler request, it can effectively reduce the number of times the crawler request is sent to the server, thereby reducing the load pressure of the server, thereby ensuring the smoothness of normal users (also understood as non-crawler users) obtaining data from the server, and effectively avoiding the situation where the privacy information of normal users is maliciously obtained by crawler users.
[0007] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, identifying whether a data request is a crawler request based on system data of the vehicle's on-board system and / or application data of the vehicle's on-board application includes: obtaining at least one data combination based on the system data of the vehicle's on-board system and / or application data of the vehicle's on-board application, the data combination including at least one data; identifying whether the data request is a crawler request based on the at least one data combination and corresponding preset conditions.
[0008] In the above technical scheme, at least one data combination can be obtained based on the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application, and based on the at least one data combination and the corresponding preset conditions, it can be identified whether the data request is a crawler request. Since each data combination has its own corresponding preset condition, the accuracy of crawler identification of data requests can be guaranteed.
[0009] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the system data of the vehicle's on-board system includes the number of account switching times of the on-board system, at least one first attribute value and at least one second attribute value of the on-board system, the first attribute value is the software attribute value of the on-board system, and the second attribute value of the vehicle is the functional attribute value of the on-board system; the application data of the vehicle's on-board application includes at least one of the frequency of the on-board application sending data requests, the running time of the on-board application, and the number of restarts of the on-board application; according to the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application, at least one data combination is obtained, including: combining the frequency of the on-board application sending data requests and the number of account switching times of the on-board system to obtain a first data combination; combining the frequency of the on-board application sending data requests and the running time of the on-board application to obtain a second data combination; combining the frequency of the on-board application sending data requests and the number of restarts of the on-board application to obtain a third data combination; combining the frequency of the on-board application sending data requests and at least one first attribute value to obtain a fourth data combination; combining at least one second attribute value to obtain a fifth data combination.
[0010] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, whether a data request is a crawler request is identified based on at least one data combination and a corresponding preset condition, including: the frequency of the vehicle-machine application sending data requests and the number of account switching times of the vehicle-machine system in the first data combination are respectively in the first preset range and the second preset range; and / or, the frequency of the vehicle-machine application sending data requests and the running time of the vehicle-machine application in the second data combination are respectively in the first preset range and the third preset range; and / or, the frequency of the vehicle-machine application sending data requests and the number of restarts of the vehicle-machine application in the third data combination are respectively in the first preset range and the fourth preset range; and / or, the frequency of the vehicle-machine application sending data requests in the fourth data combination is in the first preset range, and each first attribute value is different from the corresponding initial attribute value; and / or, when at least one second attribute value in the fifth data combination does not match each other, it is determined that the data request is a crawler request.
[0011] In the above technical solution, when any data combination meets the corresponding preset conditions, it can be determined that the data request is a crawler request, thereby avoiding sending the data request to the server.
[0012] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, whether a data request is a crawler request is identified based on at least one data combination and a corresponding preset condition, including: the frequency of the vehicle-computer application sending data requests and the number of account switching times of the vehicle-computer system in the first data combination are not within the first preset range and the second preset range, respectively; and, the frequency of the vehicle-computer application sending data requests and the running time of the vehicle-computer application in the second data combination are not within the first preset range and the third preset range, respectively; and, the frequency of the vehicle-computer application sending data requests and the number of restarts of the vehicle-computer application in the third data combination are not within the first preset range and the fourth preset range, respectively; and, the frequency of the vehicle-computer application sending data requests in the fourth data combination is not within the first preset range, and each first attribute value is the same as the corresponding initial attribute value; and, when at least one second attribute value in the fifth data combination matches each other, it is determined that the data request is not a crawler request.
[0013] In the above technical solution, when all data combinations do not meet the corresponding preset conditions, it can be determined that the data request is not a crawler request, thereby ensuring the accuracy of determining that the data request is not a crawler request.
[0014] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the method also includes: determining the current operating environment of the vehicle's vehicle-computer system in response to a data request sent by the vehicle's vehicle-computer application; when the current operating environment of the vehicle's vehicle-computer system is a test environment, not performing crawler identification on the data request; when the current operating environment of the vehicle's vehicle-computer system is a production environment, performing crawler identification on the data request.
[0015] In the above technical scheme, when responding to the data request sent by the vehicle's on-board application, the current operating environment of the vehicle's on-board system will be determined first. Since the response data corresponding to the data request has no actual value when the current operating environment of the vehicle's on-board system is a test environment, the data request may not be crawled, that is, the subsequent steps of obtaining the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application are not performed, thereby reducing the vehicle's control power consumption; however, when the current operating environment of the vehicle's on-board system is a production environment, that is, when the on-board system is directly facing the user, since the response data corresponding to the data request has actual value, in this case, the data request needs to be crawled, thereby reducing the number of times the crawler request is sent to the server, thereby reducing the load pressure on the server.
[0016] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the method also includes: when it is determined that the data request is a crawler request, randomly generating a simulated response data corresponding to the data request, and sending the simulated response data to the vehicle's on-board application.
[0017] In the above technical solution, when it is determined that the data request is a crawler request, a simulated response data corresponding to the data request can be randomly generated and sent to the vehicle's on-board application to confuse the crawler user. This prevents the real data from being maliciously obtained by the crawler user and avoids the crawler user from sending the same data request again.
[0018] In a second aspect, a crawler identification device is provided, which is applied to a vehicle terminal, and includes:
[0019] An acquisition module, configured to acquire system data of the vehicle's vehicle system and / or application data of the vehicle's vehicle application in response to a data request sent by the vehicle's vehicle application to the server;
[0020] An identification module, used to identify whether a data request is a crawler request based on system data of a vehicle system and / or application data of a vehicle application;
[0021] The interception module is used to intercept the data request when it is determined that the data request is a crawler request, so that the data request is not sent to the server.
[0022] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the device also includes a combination module, which is used to obtain at least one data combination based on the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application, and the data combination includes at least one data; an identification module, which is specifically used to identify whether the data request is a crawler request based on at least one data combination and corresponding preset conditions.
[0023] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the system data of the vehicle's on-board system includes the number of account switching times of the on-board system, at least one first attribute value and at least one second attribute value of the on-board system, the first attribute value is the software attribute value of the on-board system, and the second attribute value of the vehicle is the functional attribute value of the on-board system; the application data of the vehicle's on-board application includes at least one of the frequency of the on-board application sending data requests, the running time of the on-board application, and the number of restarts of the on-board application; the combination module is specifically used to combine the frequency of the on-board application sending data requests and the number of account switching times of the on-board system to obtain a first data combination; combine the frequency of the on-board application sending data requests and the running time of the on-board application to obtain a second data combination; combine the frequency of the on-board application sending data requests and the number of restarts of the on-board application to obtain a third data combination; combine the frequency of the on-board application sending data requests and at least one first attribute value to obtain a fourth data combination; combine at least one second attribute value to obtain a fifth data combination.
[0024] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the identification module is specifically used to determine that the frequency of the vehicle-computer application sending data requests and the number of account switching times of the vehicle-computer system in the first data combination are respectively within the first preset range and the second preset range; and / or, the frequency of the vehicle-computer application sending data requests and the running time of the vehicle-computer application in the second data combination are respectively within the first preset range and the third preset range; and / or, the frequency of the vehicle-computer application sending data requests and the number of restarts of the vehicle-computer application in the third data combination are respectively within the first preset range and the fourth preset range; and / or, the frequency of the vehicle-computer application sending data requests in the fourth data combination is within the first preset range, and each first attribute value is different from the corresponding initial attribute value; and / or, when at least one second attribute value in the fifth data combination does not match each other, determine that the data request is a crawler request.
[0025] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the identification module is specifically used to: the frequency of the vehicle-computer application sending data requests and the number of account switching times of the vehicle-computer system in the first data combination are not within the first preset range and the second preset range, respectively; and, the frequency of the vehicle-computer application sending data requests and the running time of the vehicle-computer application in the second data combination are not within the first preset range and the third preset range, respectively; and, the frequency of the vehicle-computer application sending data requests and the number of restarts of the vehicle-computer application in the third data combination are not within the first preset range and the fourth preset range, respectively; and, the frequency of the vehicle-computer application sending data requests in the fourth data combination is not within the first preset range, and each first attribute value is the same as the corresponding initial attribute value; and, when at least one second attribute value in the fifth data combination matches each other, it is determined that the data request is not a crawler request.
[0026] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the device also includes a determination module for determining the current operating environment of the vehicle's vehicle-computer system in response to a data request sent by the vehicle's vehicle-computer application; when the current operating environment of the vehicle's vehicle-computer system is a test environment, crawler identification is not performed on the data request; when the current operating environment of the vehicle's vehicle-computer system is a production environment, crawler identification is performed on the data request.
[0027] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the device also includes a generation module for randomly generating a simulated response data corresponding to the data request when it is determined that the data request is a crawler request, and sending the simulated response data to the vehicle's on-board application.
[0028] In a third aspect, a vehicle is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the crawler identification method in the first aspect or any possible implementation of the first aspect.
[0029] In a fourth aspect, a computer program product is provided, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the crawler identification method in the above-mentioned first aspect or any possible implementation of the first aspect.
[0030] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the crawler identification method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a structural schematic diagram of a vehicle system provided by an embodiment of the present application;
[0032] Figure 2 is a schematic flow chart of a crawler identification method provided in an embodiment of the present application;
[0033] Figure 3 is a structural schematic diagram of a crawler identification device provided in an embodiment of the present application;
[0034] Figure 4 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION
[0035] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.
[0036] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.
[0037] A web crawler is a computer program that searches for web pages through their link addresses and automatically obtains the content of web pages according to certain rules. Since web crawlers do not need to render pages, but only need to obtain file contents and the Uniform Resource Locator (URL) in the files, web crawlers can access servers at a very high frequency, which increases the load pressure on the servers and reduces the smoothness of data acquisition for normal users.
[0038] Therefore, how to reduce the load pressure on the server is an urgent problem that needs to be solved.
[0039] Based on the above problems, the present application provides a crawler identification method, which can identify and intercept web crawler requests, thereby reducing the access frequency of web crawler requests and further reducing the load pressure on the server.
[0040] Figure 1 This is a schematic diagram of the structure of a vehicle system provided by an embodiment of the present application, see Figure 1As shown, the vehicle system 100 includes a vehicle application module 101 , an anti-crawler module 102 and a vehicle application system service module 103 .
[0041] The vehicle application module 101 is used to send a data request to the anti-crawler module 102, so that the anti-crawler module 102 performs crawler identification on the data request when receiving the data request. The data request refers to a request sent by the vehicle's vehicle application (for example, a navigation application, a multimedia entertainment application, a communication application, etc.) to a server, with the purpose of obtaining response data from the server.
[0042] In a possible implementation, upon receiving the data request, the anti-crawler module 102 can obtain the system data of the vehicle system and / or the application data of the vehicle application from the vehicle application system service module 103, and perform crawler identification on the data request based on the system data of the vehicle system and / or the application data of the vehicle application.
[0043] In one possible implementation, when the anti-crawler module 102 receives a data request sent by the anti-crawler module 102, it will first determine the current operating environment of the vehicle system, so that when the current operating environment of the vehicle system is a test environment, crawler identification will not be performed on the data request, that is, the subsequent step of obtaining the system data of the vehicle system and / or the application data of the vehicle application from the vehicle application system service module 103 will not be executed; when the current operating environment of the vehicle system is a production environment, crawler identification will be performed on the data request to reduce the control power consumption of the vehicle.
[0044] Among them, the anti-crawler module 102 is also used to randomly generate a simulated response data corresponding to the data request when it is determined that the data request is a crawler request, and send the simulated response data to the vehicle application module 101, so that the vehicle application module 101 executes the corresponding business logic according to the received simulated response data (for example, playing music, updating the route on the navigation interface, updating the music list, turning on seat heating, etc.).
[0045] Below through Figure 2 The method of the embodiment of the present application is described in detail.
[0046] Figure 2 It is a schematic flow chart of a crawler identification method provided in an embodiment of the present application.
[0047] For example, Figure 2 As shown, taking the ECU at the vehicle end as an example, the method 200 includes:
[0048] Step 201 , in response to a data request sent by a vehicle-mounted application to a server, obtain system data of the vehicle-mounted system and / or application data of the vehicle-mounted application.
[0049] Among them, the car-machine applications of the car-machine include navigation applications, multimedia entertainment applications (for example, music players, video players, and radios, etc.), communication applications (for example, Bluetooth phones, smart assistants, etc.), remote control applications (for example, remote start / stop, door lock control, air conditioning presets, and vehicle positioning, etc.), safety and assisted driving applications (for example, lane keeping assist, collision warning, blind spot monitoring, and reversing images, etc.), Internet and connection applications (for example, Wi-Fi hotspots, Over-The-Air Technology (OTA) updates, etc.) and personalized settings applications (for example, user profiles, that is, allowing users to save personal settings, such as seat position, audio preferences, etc., themes and interface customization, that is, allowing users to change screen backgrounds, font size and other interface elements, etc., shortcut key settings, that is, allowing users to customize shortcuts for commonly used functions) One or more of the above, the embodiments of the present application are not limited to this.
[0050] It can be understood that a data request refers to a request sent by the vehicle's on-board application (for example, navigation application, multimedia entertainment application, communication application, etc.) to the server, with the purpose of obtaining response data from the server or performing certain operations (for example, updating the user interface).
[0051] In an embodiment of the present application, when the vehicle's on-board application sends a data request to the server, the ECU will send the data request to the anti-crawler module, so that after receiving the data request, the anti-crawler module will obtain the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application from the on-board application system service module, and identify whether the data request is a crawler request based on the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application.
[0052] Among them, the system data of the vehicle's vehicle system includes at least one of the number of account switching times of the vehicle system, at least one first attribute value of the vehicle system, and at least one second attribute value. For example, the system data of the vehicle system includes the number of account switching times of the vehicle system, the first attribute value and the second attribute value of the vehicle system; for another example, the system data of the vehicle system includes the number of account switching times of the vehicle system and the first attribute of the vehicle system, which is not limited in the embodiment of the present application. Among them, the first attribute value is the software attribute value of the vehicle system, for example, the first attribute value is the version number and memory size of the operating system of the vehicle system. The second attribute value of the vehicle system is the functional attribute value of the vehicle system, that is, the functional configuration of the vehicle system. For example, Bluetooth connection status (Bluetooth on / off), panoramic sunroof function on / off status (panoramic sunroof on / off), seat heating function on / off status (seat on / off 1st gear heating, seat on / off 2nd gear heating, seat on / off 3rd gear heating), atmosphere light function on / off status (dynamic atmosphere light on / off and static atmosphere light on / off), etc.
[0053] It should be noted that since the response data corresponding to different vehicle system accounts and different vehicle system attribute values (first attribute value and second attribute value) are different, in order to obtain more comprehensive response data, the crawler user needs to constantly switch accounts and modify the attribute values of the vehicle system. Therefore, in an embodiment of the present application, the anti-crawler module can obtain the number of account switching times of the vehicle system, the first attribute value and the second attribute value of the vehicle system from the vehicle application system service module, so as to subsequently use the number of account switching times of the vehicle system, the first attribute value and the second attribute value of the vehicle system to determine whether the data request sent by the vehicle application is a crawler request.
[0054] The application data of the vehicle's on-board application includes at least one of the frequency of the vehicle-machine application sending data requests, the running time of the vehicle-machine application, and the number of restarts of the vehicle-machine application. For example, the application data of the vehicle-machine application includes the frequency of the vehicle-machine application sending data requests, the running time of the vehicle-machine application, and the number of restarts of the vehicle-machine application; for another example, the application data of the vehicle-machine application includes the frequency of the vehicle-machine application sending data requests and the running time of the vehicle-machine application, which is not limited in the embodiments of the present application.
[0055] It should be noted that since different data requests correspond to different response data, in order to obtain more response data, the crawler user needs to frequently send data requests through the vehicle application. Therefore, in an embodiment of the present application, the anti-crawler module can also obtain the frequency of the data request sent by the vehicle application from the vehicle application system service module, so as to subsequently use the frequency of the data request sent by the vehicle application to determine whether the data request sent by the vehicle application is a crawler request. Moreover, since the crawler user is only used to obtain response data and will not use the car application for a long time, that is, after obtaining the response data, he will immediately exit the car application. Therefore, in the embodiment of the present application, the anti-crawler module can also obtain the running time of the car application from the car application system service module, so as to subsequently use the running time of the car application to determine whether the data request sent by the car application is a crawler request; in addition, after the second attribute value of the car system is modified, it will only take effect after the corresponding car application is restarted, so the crawler user will frequently restart the car application. Therefore, in the embodiment of the present application, the anti-crawler module can also obtain the number of restarts of the car application from the car application system service module, so as to subsequently use the number of restarts of the car application to determine whether the data request sent by the car application is a crawler request.
[0056] In one possible implementation, when responding to a data request sent by the vehicle's on-board application, the anti-crawler module will first determine the current operating environment of the vehicle's on-board system. Since the response data corresponding to the data request has no actual value when the current operating environment of the vehicle's on-board system is a test environment, the anti-crawler module may not perform crawler identification on the data request, that is, it does not perform the subsequent steps of obtaining the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application, thereby reducing the vehicle's control power consumption. It should be noted that when the operating environment of the on-board system is a test environment, the developer has built a simulated server through the intranet. Therefore, even if the data request is a crawler request, since the intranet environment is isolated from the external Internet, the load pressure on the server in the external Internet will not increase at this time. However, when the current operating environment of the vehicle's on-board system is a production environment, that is, when the on-board system is directly facing the user, since the response data corresponding to the data request has actual value and the on-board system interacts with the server in the external Internet, in this case, it is necessary to perform crawler identification on the data request to reduce the number of times the crawler request is sent to the server, thereby reducing the load pressure on the server.
[0057] It should be noted that in order to facilitate the anti-crawler module to quickly determine whether the current operating environment of the vehicle's on-board system is a test environment or a production environment, in a possible implementation, the developer can set specific environment variables for the test environment and the production environment respectively. For example, set Environment = TEST in the test environment and set Environment = PROD in the production environment, so that when the anti-crawler module identifies the current operating environment of the vehicle's on-board system, it obtains the current environment variables and quickly identifies whether the current operating environment of the vehicle's on-board system is a test environment or a production environment based on the current environment variables.
[0058] Step 202 , identifying whether the data request is a crawler request based on system data of the vehicle's onboard system and / or application data of the vehicle's onboard application.
[0059] Exemplarily, after the anti-crawler module obtains the system data of the vehicle's vehicle system and / or the application data of the vehicle's vehicle application from the vehicle application system service module, the anti-crawler module can identify whether the data request is a crawler request based on the system data of the vehicle system and the application data of the vehicle application.
[0060] In a possible implementation, when the anti-crawler module identifies whether a data request is a crawler request based on the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board applications, it can first obtain at least one data combination based on the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board applications, and identify whether the data request is a crawler request based on the at least one data combination and corresponding preset conditions. Since each data combination has its own corresponding preset condition, the accuracy of crawler identification of data requests can be guaranteed.
[0061] The following describes in detail how to obtain at least one data combination based on the system data of the vehicle's onboard system and / or the application data of the vehicle's onboard application.
[0062] It should be noted that, since there may be a situation where normal users also send data requests at a high frequency through the vehicle-mounted application, in order to ensure the accuracy of data request identification, in a possible implementation, the frequency of the vehicle-mounted application sending data requests needs to be combined with other data to improve the accuracy of data request identification. Specifically, the frequency of the vehicle-mounted application sending data requests and the number of account switching times of the vehicle-mounted system can be combined to obtain a first data combination; the frequency of the vehicle-mounted application sending data requests and the running time of the vehicle-mounted application can be combined to obtain a second data combination; the frequency of the vehicle-mounted application sending data requests and the number of restarts of the vehicle-mounted application can be combined to obtain a third data combination; the frequency of the vehicle-mounted application sending data requests and at least one first attribute value of the vehicle-mounted system can be combined to obtain a fourth data combination. Here, it is worth noting that since at least one second attribute value of the vehicle-mounted system matches each other, that is, when multiple second attribute values of the vehicle-mounted system do not match each other, it means that the second attribute value of the vehicle-mounted system must have been maliciously modified, that is, the data request can be determined to be a crawler request, so all second attribute values of the vehicle-mounted system can be combined to obtain a fifth data combination.
[0063] Among them, the mismatch between multiple second attribute values of the vehicle system can be understood as, when the vehicle system does not support seat heating, the vehicle system displays that the seat heating is turned on, or, when the vehicle system supports seat heating but does not support 3-speed seat heating, the vehicle system displays that the seat heating is turned on and the 3-speed seat heating is turned on, or, when the vehicle system supports turning on the rhythmic atmosphere light (dynamic atmosphere light), the vehicle system displays that the dynamic atmosphere light is not supported, or, when the vehicle system only supports turning on the static rhythmic atmosphere light, the vehicle system displays that the rhythmic atmosphere light is turned on, etc.
[0064] After introducing how to obtain at least one data combination based on the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application, the following describes in detail how to identify whether the data request is a crawler request based on at least one data combination and corresponding preset conditions.
[0065] In a possible implementation, after the vehicle application sends a data request, if the anti-crawler module determines that any one of at least one data combination meets the corresponding preset conditions, the data request can be determined to be a crawler request, thereby avoiding sending the data request to the server.
[0066] Specifically, the frequency of the vehicle-machine application sending data requests and the number of account switching times of the vehicle-machine system in the first data combination are respectively within the first preset range and the second preset range; and / or, the frequency of the vehicle-machine application sending data requests and the running time of the vehicle-machine application in the second data combination are respectively within the first preset range and the third preset range; and / or, the frequency of the vehicle-machine application sending data requests and the number of restarts of the vehicle-machine application in the third data combination are respectively within the first preset range and the fourth preset range; and / or, the frequency of the vehicle-machine application sending data requests in the fourth data combination is within the first preset range, and each first attribute value of the vehicle-machine system (for example, the version number and memory size of the operating system) is different from the corresponding initial attribute value; and / or, in the fifth data combination, when at least one second attribute value of the vehicle-machine system does not match each other, the data request is determined to be a crawler request.
[0067] Among them, each first attribute value of the vehicle system in the fourth data combination is different from the corresponding initial attribute value, which can be understood as that the version number of the operating system of the vehicle system is different from the initial version number, and the memory size of the vehicle system is different from the initial memory size.
[0068] Exemplarily, when the frequency of data requests sent by the vehicle-computer application and the number of account switching times of the vehicle-computer system in the first data combination are respectively within the first preset range and the second preset range, but the remaining data combinations, i.e., the second data combination, the third data combination, the fourth data combination and the fifth data combination, do not meet the preset conditions, the data request is determined to be a crawler request, or, when the frequency of data requests sent by the vehicle-computer application and the number of account switching times of the vehicle-computer system in the first data combination are respectively within the first preset range and the second preset range; the frequency of data requests sent by the vehicle-computer application and the running time of the vehicle-computer application in the second data combination are respectively within the first preset range and the third preset range, but the remaining data combinations, i.e., the third data combination, the fourth data combination and the fifth data combination, do not meet the preset conditions, the data request is determined to be a crawler request.
[0069] In one possible implementation, after the vehicle application sends a data request, if the ECU determines that all data combinations do not meet the corresponding preset conditions, it can be determined that the data request is not a crawler request, thereby ensuring the accuracy of determining that the data request is not a crawler request.
[0070] Specifically, the frequency of the vehicle-machine application sending data requests and the number of account switching times of the vehicle-machine system in the first data combination are not within the first preset range and the second preset range, respectively; and, the frequency of the vehicle-machine application sending data requests and the running time of the vehicle-machine application in the second data combination are not within the first preset range and the third preset range, respectively; and, the frequency of the vehicle-machine application sending data requests and the number of restarts of the vehicle-machine application in the third data combination are not within the first preset range and the fourth preset range, respectively; and, the frequency of the vehicle-machine application sending data requests in the fourth data combination is not within the first preset range, and each first attribute value is the same as the corresponding initial attribute value; and, when at least one second attribute value in the fifth data combination matches each other, it is determined that the data request is not a crawler request.
[0071] Step 203: When it is determined that the data request is a crawler request, the data request is intercepted so that the data request is not sent to the server.
[0072] Specifically, when the anti-crawler module of the vehicle system determines that the data request is a crawler request, it will intercept the data request, that is, it will not send the data request to the server, thereby effectively reducing the number of times the crawler request is sent to the server, thereby reducing the load pressure on the server, and thus ensuring the smoothness of normal users obtaining data from the server, and effectively avoiding the situation where the privacy information of normal users is maliciously obtained by crawler users; when the anti-crawler module of the vehicle system determines that the data request is not a crawler request, it will send the data request to the server, and when receiving the response data corresponding to the data request returned by the server, it will send the response data to the vehicle application module, so that the vehicle application module executes the corresponding business logic according to the response data (for example, playing music, updating the route on the navigation interface, updating the music list, turning on seat heating, etc.).
[0073] In one possible implementation, after the anti-crawler module determines that the data request is a crawler request and intercepts the data request, it can also randomly generate a simulated response data corresponding to the data request, and send the simulated response data to the vehicle's on-board application, that is, the vehicle's on-board application module, to confuse the crawler user, so that the real data is not maliciously obtained by the crawler user, and the crawler user is prevented from sending the same data request again.
[0074] Optionally, developers can design a series of simulated JSON data or XML data according to the output format of the actual interface. These data should be as close to the real response as possible, including but not limited to successful response, error prompt, empty data scenario, etc. A database or folder system is created in the anti-crawler module to store these preset simulated response data, so that after the anti-crawler module determines that the data request is a crawler request and intercepts the data request, it can flexibly search for the corresponding simulated response data according to different data requests and return the simulated response data to the vehicle application module.
[0075] The actual interface can be understood as the server interface, in other words, a set of specifications and methods defined by the server for communication and data exchange with the vehicle or other systems. These interfaces define the functions that the vehicle can access, the data format, the request method, and the server's response to these requests.
[0076] In summary, the crawler identification method provided by the embodiment of the present application can obtain the system data of the vehicle's vehicle system and / or the application data of the vehicle's vehicle application in response to the data request sent by the vehicle's vehicle application to the server, and identify whether the data request is a crawler request based on the system data of the vehicle's vehicle system and / or the application data of the vehicle's vehicle application. Since the embodiment of the present application intercepts the data request when the vehicle determines that the data request is a crawler request, it can effectively reduce the number of times the crawler request is sent to the server, thereby reducing the load pressure of the server, thereby ensuring the smoothness of normal users (also understood as non-crawler users) obtaining data from the server, and effectively avoiding the situation where the privacy information of normal users is maliciously obtained by crawler users. In addition, when the vehicle responds to the data request sent by the vehicle's vehicle application, the vehicle will first determine the current operating environment of the vehicle's vehicle system, so that when the current operating environment of the vehicle's vehicle system is a test environment, the data request is not crawler identified, that is, the subsequent steps of obtaining the system data of the vehicle's vehicle system and / or the application data of the vehicle's vehicle application are not executed, thereby reducing the control power consumption of the vehicle. In addition, after determining that the data request is a crawler request and intercepting the data request, a simulated response data corresponding to the data request can be randomly generated and sent to the vehicle's on-board application to confuse the crawler user. This prevents the real data from being maliciously obtained by the crawler user and avoids the crawler user from sending the same data request again.
[0077] Figure 3 It is a structural schematic diagram of a crawler identification device provided in an embodiment of the present application.
[0078] For example, Figure 3 As shown, the device 300 includes:
[0079] An acquisition module 301 is used to acquire system data of the vehicle's vehicle system and / or application data of the vehicle's vehicle application in response to a data request sent by the vehicle's vehicle application to the server;
[0080] An identification module 302, configured to identify whether a data request is a crawler request based on system data of a vehicle system and / or application data of a vehicle application;
[0081] The interception module 303 is used to intercept the data request when it is determined that the data request is a crawler request, so that the data request is not sent to the server.
[0082] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the device also includes a combination module, which is used to obtain at least one data combination based on the system data of the vehicle's on-board system and / or the application data of the vehicle's on-board application, and the data combination includes at least one data; an identification module 302, which is specifically used to identify whether the data request is a crawler request based on at least one data combination and corresponding preset conditions.
[0083] In one possible implementation, the system data of the vehicle's on-board system includes the number of account switching times of the on-board system, at least one first attribute value and at least one second attribute value of the on-board system, the first attribute value is the software attribute value of the on-board system, and the second attribute value of the vehicle is the functional attribute value of the on-board system; the application data of the vehicle's on-board application includes at least one of the frequency of the on-board application sending data requests, the running time of the on-board application, and the number of restart times of the on-board application; the combination module is specifically used to combine the frequency of the on-board application sending data requests and the number of account switching times of the on-board system to obtain a first data combination; combine the frequency of the on-board application sending data requests and the running time of the on-board application to obtain a second data combination; combine the frequency of the on-board application sending data requests and the number of restart times of the on-board application to obtain a third data combination; combine the frequency of the on-board application sending data requests and at least one first attribute value to obtain a fourth data combination; combine at least one second attribute value to obtain a fifth data combination.
[0084] In one possible implementation, the identification module 302 is further specifically used to determine that the frequency of the vehicle-machine application sending data requests and the number of account switching times of the vehicle-machine system in the first data combination are respectively within the first preset range and the second preset range; and / or, the frequency of the vehicle-machine application sending data requests and the running time of the vehicle-machine application in the second data combination are respectively within the first preset range and the third preset range; and / or, the frequency of the vehicle-machine application sending data requests and the number of restarts of the vehicle-machine application in the third data combination are respectively within the first preset range and the fourth preset range; and / or, the frequency of the vehicle-machine application sending data requests in the fourth data combination is within the first preset range, and each first attribute value is different from the corresponding initial attribute value; and / or, when at least one second attribute value in the fifth data combination does not match each other, determine that the data request is a crawler request.
[0085] In one possible implementation, the identification module 302 is specifically used to determine that the frequency of the vehicle-computer application sending data requests and the number of account switching times of the vehicle-computer system in the first data combination are not within the first preset range and the second preset range, respectively; and, the frequency of the vehicle-computer application sending data requests and the running time of the vehicle-computer application in the second data combination are not within the first preset range and the third preset range, respectively; and, the frequency of the vehicle-computer application sending data requests and the number of restarts of the vehicle-computer application in the third data combination are not within the first preset range and the fourth preset range, respectively; and, the frequency of the vehicle-computer application sending data requests in the fourth data combination is not within the first preset range, and each first attribute value is the same as the corresponding initial attribute value; and, when at least one second attribute value in the fifth data combination matches each other, it is determined that the data request is not a crawler request.
[0086] In one possible implementation, the device also includes a determination module for determining the current operating environment of the vehicle's vehicle system in response to a data request sent by the vehicle's vehicle application; when the current operating environment of the vehicle's vehicle system is a test environment, crawler identification is not performed on the data request; when the current operating environment of the vehicle's vehicle system is a production environment, crawler identification is performed on the data request.
[0087] In one possible implementation, the device also includes a generation module for randomly generating a simulated response data corresponding to the data request when it is determined that the data request is a crawler request, and sending the simulated response data to the vehicle's on-board application.
[0088] Figure 4 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.
[0089] For example, Figure 4As shown, the vehicle 400 includes: a memory 401 and a processor 402, wherein the memory 401 stores an executable program code 403, and the processor 402 is used to call and execute the executable program code 403 to perform a crawler identification method.
[0090] In addition, an embodiment of the present application also protects a system, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute a crawler identification method provided by an embodiment of the present application.
[0091] In this embodiment, the system can be divided into functional modules according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0092] In the case of dividing each functional module according to each function, the system may also include a determination module, a combination module, a generation module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.
[0093] It should be understood that the system provided in this embodiment is used to execute the above-mentioned crawler identification method, and thus can achieve the same effect as the above-mentioned implementation method.
[0094] In the case of an integrated unit, the system may include a processing module and a storage module. When the system is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related program codes, etc.
[0095] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a data combination that implements a computing function, such as a data combination including one or more microprocessors, a data combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.
[0096] In addition, the system provided in the embodiments of the present application may specifically be a chip, component or module, and the chip may include a connected processor and memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a crawler identification method provided in the above embodiments.
[0097] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is run on a computer, the computer executes the above-mentioned related method steps to implement a crawler identification method provided in the above embodiment.
[0098] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a crawler identification method provided by the above embodiment.
[0099] Among them, the system, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
[0100] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.
[0101] In the embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.
[0102] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A crawler identification method, characterized in that: Applied to the vehicle terminal, the method includes: In response to a data request sent by a vehicle-mounted application to a server, obtaining system data of the vehicle-mounted system and / or application data of the vehicle-mounted application; identifying, according to system data of the vehicle system and / or application data of the vehicle application, whether the data request is a crawler request; In the case where it is determined that the data request is the crawler request, the data request is intercepted so that the data request is not sent to the server.
2. The method according to claim 1, characterized in that The identifying whether the data request is a crawler request according to the system data of the vehicle system and / or the application data of the vehicle application includes: Obtaining at least one data combination according to the system data of the vehicle system and / or the application data of the vehicle application, wherein the data combination includes at least one data; According to the at least one data combination and the corresponding preset condition, identify whether the data request is the crawler request.
3. The method according to claim 2, characterized in that The system data of the vehicle's on-board system includes the number of account switching times of the on-board system, at least one of at least one first attribute value and at least one second attribute value of the on-board system, wherein the first attribute value is a software attribute value of the on-board system and the second attribute value of the vehicle is a functional attribute value of the on-board system; the application data of the vehicle's on-board application includes at least one of the frequency of the on-board application sending the data request, the running time of the on-board application and the number of restart times of the on-board application; The obtaining of at least one data combination according to the system data of the vehicle system and / or the application data of the vehicle application includes: Combining the frequency of the vehicle computer application sending the data request and the number of account switching times of the vehicle computer system to obtain a first data combination; Combining the frequency at which the vehicle-mounted computer application sends the data request and the running time of the vehicle-mounted computer application to obtain a second data combination; Combining the frequency at which the vehicle-machine application sends the data request and the number of restarts of the vehicle-machine application to obtain a third data combination; Combining the frequency at which the vehicle computer application sends the data request and the at least one first attribute value to obtain a fourth data combination; The at least one second attribute value is combined to obtain a fifth data combination.
4. The method according to claim 3, characterized in that The step of identifying whether the data request is the crawler request according to the at least one data combination and the corresponding preset condition includes: The frequency of the vehicle computer application sending the data request and the number of account switching times of the vehicle computer system in the first data combination are respectively within a first preset range and a second preset range; and / or, the frequency of sending the data request by the vehicle-mounted application and the running time of the vehicle-mounted application in the second data combination are respectively within the first preset range and the third preset range; and / or, the frequency of sending the data request by the vehicle-machine application and the number of restarts of the vehicle-machine application in the third data combination are respectively within the first preset range and the fourth preset range; and / or, the frequency at which the vehicle-mounted application in the fourth data combination sends the data request is within the first preset range, and each first attribute value is different from the corresponding initial attribute value; And / or, when the at least one second attribute value in the fifth data combination does not match each other, determining that the data request is the crawler request.
5. The method according to claim 3, characterized in that: The step of identifying whether the data request is a crawler request according to the at least one data combination and a corresponding preset condition includes: The frequency of the vehicle computer application sending the data request and the number of account switching times of the vehicle computer system in the first data combination are not within the first preset range and the second preset range respectively; Furthermore, the frequency of sending the data request by the vehicle-mounted application and the running time of the vehicle-mounted application in the second data combination are not within the first preset range and the third preset range respectively; Furthermore, the frequency of sending the data request by the vehicle-machine application and the number of restarts of the vehicle-machine application in the third data combination are not within the first preset range and the fourth preset range respectively; Furthermore, the frequency at which the vehicle-mounted application in the fourth data combination sends the data request is not within the first preset range, and each first attribute value is the same as the corresponding initial attribute value; Furthermore, when the at least one second attribute value in the fifth data combination matches each other, it is determined that the data request is not the crawler request.
6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: In response to a data request sent by a vehicle computer application of the vehicle, determining a current operating environment of a vehicle computer system of the vehicle; When the current operating environment of the vehicle system is a test environment, no crawler identification is performed on the data request; When the current operating environment of the vehicle's onboard system is a production environment, crawler identification is performed on the data request.
7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When it is determined that the data request is the crawler request, a simulated response data corresponding to the data request is randomly generated, and the simulated response data is sent to the vehicle computer application of the vehicle.
8. A crawler identification device, characterized in that: Applied to the vehicle terminal, the device comprises: An acquisition module, configured to acquire system data of the vehicle's vehicle system and / or application data of the vehicle's vehicle application in response to a data request sent by the vehicle's vehicle application to the server; an identification module, configured to identify whether the data request is a crawler request based on system data of a vehicle system of the vehicle and / or application data of a vehicle application of the vehicle; The interception module is used to intercept the data request when it is determined that the data request is the crawler request, so that the data request is not sent to the server.
9. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the vehicle executes the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Web crawler identification method and electronic commerce system
CN115174215A
Anti-crawler headless browser detection and identification method and device
CN116389069A
Crawler method and device for dynamically adjusting sending speed, equipment and product
CN118093978A
Traffic proxy method and system, and smart terminal and storage medium
WO2023097748A1